feat(meta): add TaggedTimestamp field and preserve during re-parsing
Add TaggedTimestamp field to track when image tags were created, exposed
through GraphQL API. Previously, when zot restarted and re-parsed storage,
ResetRepoReferences would clear all tags, causing timestamp information to
be lost and reset to the service restart time for existing images.
This change adds TaggedTimestamp support and modifies ResetRepoReferences to
selectively preserve tags that still exist in storage, maintaining their
TaggedTimestamp values. Tags that no longer exist in storage are removed as
before.
Changes:
- Add TaggedTimestamp field to GraphQL ImageSummary schema
- Update GraphQL conversion functions to populate TaggedTimestamp with
fallback to PushTimestamp when unavailable
- Updated ResetRepoReferences interface to accept tagsToKeep parameter
- Modified ParseRepo to collect tags from storage before resetting
- Updated all backend implementations (Redis, DynamoDB, BoltDB) to preserve
tags in tagsToKeep instead of clearing all tags
- Updated tests and mocks to match new signature
This ensures TaggedTimestamp accurately reflects when tags were originally
created, and exposes this information through the GraphQL API.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: prevent nil pointer dereference in RemoveImageFromRepoMeta
This commit fixes a critical bug where RemoveImageFromRepoMeta crashes with
a nil pointer dereference during retention policy execution and GC operations.
Root Cause:
The function was accessing blob metadata without checking if it exists first.
During GC/retention operations, the metadata database might have stale
references to blobs that no longer exist, causing runtime panics.
Changes:
- Added nil check for descriptorBlobInfo before accessing LastUpdated field
- Added nil check for blobInfo before dereferencing Size, Vendors, Platforms, and SubBlobs
- Made the function consistent with recalculateAggregateFields which already had these checks
Impact:
- Fixes crashes during retention policy execution
- Fixes crashes during GC manifest removal
- Fixes image deletion failures via API
- Eliminates need for dryRun: true workaround in retention config
The fix gracefully handles missing blob metadata by skipping those entries
instead of crashing.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: add comprehensive tests for RemoveImageFromRepoMeta nil checks
Add test coverage for the nil pointer dereference fixes in RemoveImageFromRepoMeta.
These tests ensure the function handles missing blob metadata gracefully during
GC and retention operations.
Test cases:
- Handle nil blob info for descriptor digest (line 280 check)
- Handle nil blob info in queue traversal (line 297 check)
- Verify correct behavior with valid blob info
- Handle empty tags edge case
- Skip tags with empty digest
Coverage: RemoveImageFromRepoMeta now has 100% test coverage
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: fix RemoveImageFromRepoMeta tests to match actual usage
Address review feedback:
- Delete tag from repoMeta.Tags before calling RemoveImageFromRepoMeta
- Fix blob count expectations after tag removal
- Add assertion to verify tag was removed from metadata
- Update comments to clarify expected behavior
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: add tag removal assertion to second test case
Add missing assertion to verify tag1 was removed from resultMeta.Tags
in the 'should handle nil blob info in queue traversal' test.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* refactor: improve nil blob handling documentation and test coverage
Address Copilot review feedback:
- Expand comment at line 278 to explain implications of skipping tags
with missing blob info, clarifying that metadata inconsistency is
acceptable in GC/cleanup scenarios
- Revise 'should handle nil blob info for descriptor digest' test to
cover more realistic scenario: remove tag1 while tag2 has missing
blob info, demonstrating graceful handling of data inconsistencies
in remaining tags during removal operations
All tests pass with 49 total assertions.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* fix: prevent nil pointer in GetCandidates when statistics missing
Add defensive check in pkg/retention/candidate.go to handle cases where
a tag exists in repoMeta.Tags but has no corresponding entry in
repoMeta.Statistics. This prevents incorrect retention decisions based
on zero-value timestamps.
Changes:
- Check statistics existence before creating candidates
- Skip tags with missing statistics (retained by GetRetainedTagsFromMetaDB)
- Improve performance from O(n*m) to O(n) by using direct map lookup
- Add comprehensive test coverage for missing statistics scenarios
This addresses the concern raised in PR #3658 about metadata
inconsistencies due to non-transactional writes to blob store and metaDB.
Related: #3658
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: achieve 100% coverage for RemoveImageFromRepoMeta nil checks
Enhance test coverage for RemoveImageFromRepoMeta to address codecov failures
by adding comprehensive test cases that exercise all code paths including nil
pointer checks and continue statements.
Changes:
- Enhanced 'nil blob info for descriptor digest' test to verify processing
continues with other valid tags after skipping nil entries
- Enhanced 'nil blob info in queue traversal' test to handle mixed valid/nil
sub-blobs and verify correct processing continuation
- Added 'multiple nil blobs in deeply nested structure' test to cover complex
scenarios with multiple missing blobs at various nesting levels
- Enhanced 'skip tags with empty digest' test to verify processing continues
with valid tags after skipping empty digest entries
- Added 'combined edge cases' test to verify all edge cases work together:
empty digest, nil descriptor blob, and nil queue blob
Coverage Results:
- RemoveImageFromRepoMeta: 100.0% line coverage (was 87.50%)
- All 7 test scenarios pass with 75 total assertions
- All nil check code paths fully exercised
- All continue statement behaviors validated
Fixes codecov/patch failure on PR #3658 where 2 lines were missing coverage.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
---------
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* fix: migrate to Go module v2 for proper semantic versioning
This change updates the module path from 'zotregistry.dev/zot' to
'zotregistry.dev/zot/v2' to comply with Go's semantic versioning rules.
According to Go's module versioning requirements, major version v2+
must include the major version in the module path. The current
module path 'zotregistry.dev/zot' only supports v0.x.x and v1.x.x
versions, making existing v2.x.x tags (like v2.1.8) unusable.
Changes:
- Updated go.mod module path to zotregistry.dev/zot/v2
- Updated all internal import paths across 280+ Go source files
- Updated configuration files (golangcilint.yaml, gqlgen.yml)
- Updated README.md Go reference badge
This fix enables proper use of existing v2.x.x Git tags and allows
external packages to import zot v2+ versions without compatibility
errors.
Resolves: Go module import compatibility for v2+ versions
Fixes: #3071
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
* fix: regenerate GraphQL files with updated v2 import paths
The gqlgen tool needs to regenerate the GraphQL schema files after
the module path change to use the new v2 imports.
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
---------
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
Initial code was contributed by Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com>
Moved implementation from a separate db to repodb by Andrei Aaron <aaaron@luxoft.com>
Not done yet:
- run/test dynamodb implementation, only boltdb was tested
- add additional coverage for existing functionality
- add web-based APIs to toggle the stars/bookmarks on/off
Initially graphql mutation was discussed for the missing API but
we decided REST endpoints would be better suited for configuration
feat(userdb): complete functionality for userdb integration
- dynamodb rollback changes to user starred repos in case increasing the total star count fails
- dynamodb increment/decrement repostars in repometa when user stars/unstars a repo
- dynamodb check anonymous user permissions are working as intendend
- common test handle anonymous users
- RepoMeta2RepoSummary set IsStarred and IsBookmarked
feat(userdb): rest api calls for toggling stars/bookmarks on/off
test(userdb): blackbox tests
test(userdb): move preferences tests in a different file with specific build tags
feat(repodb): add is-starred and is-bookmarked fields to repo-meta
- removed duplicated logic for determining if a repo is starred/bookmarked
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Co-authored-by: Andrei Aaron <aaaron@luxoft.com>
* refactor(repodb): moving common utilities under pkg/meta
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
* refactor(repodb): moved update, version components under pkg/meta
- updated wrapper initialization to recieve a log object in constructor
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
* refactor(repodb): moved repodb initialization from controller to pkg/meta/repodb
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
---------
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Andrei Aaron