feat(meta): add TaggedTimestamp field and preserve during re-parsing
Add TaggedTimestamp field to track when image tags were created, exposed
through GraphQL API. Previously, when zot restarted and re-parsed storage,
ResetRepoReferences would clear all tags, causing timestamp information to
be lost and reset to the service restart time for existing images.
This change adds TaggedTimestamp support and modifies ResetRepoReferences to
selectively preserve tags that still exist in storage, maintaining their
TaggedTimestamp values. Tags that no longer exist in storage are removed as
before.
Changes:
- Add TaggedTimestamp field to GraphQL ImageSummary schema
- Update GraphQL conversion functions to populate TaggedTimestamp with
fallback to PushTimestamp when unavailable
- Updated ResetRepoReferences interface to accept tagsToKeep parameter
- Modified ParseRepo to collect tags from storage before resetting
- Updated all backend implementations (Redis, DynamoDB, BoltDB) to preserve
tags in tagsToKeep instead of clearing all tags
- Updated tests and mocks to match new signature
This ensures TaggedTimestamp accurately reflects when tags were originally
created, and exposes this information through the GraphQL API.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* ci: add a upgrade bats test
Fixes https://github.com/project-zot/zot/issues/3601
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* ci: first check existing images
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* ci: refactor into common test code
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* refactor: Refactor upgrade tests to improve readability and maintainability
- Added section headers for release tests, upgrade process, and new tests in upgrade.bats and upgrade_minimal.bats.
- Replaced inline port retrieval with a function call to get_zot_port for consistency.
- Consolidated repeated test logic into dedicated functions (e.g., test_new_existing_pull_image, test_new_push_image).
- Removed unnecessary variable assignments and streamlined test cases for clarity.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Add support for configurable identity attributes in mTLS authentication,
allowing identity extraction from CommonName, Subject DN, Email SAN,
URI SAN, or DNSName SAN with fallback chain support. Includes regex
pattern matching for URI SANs (e.g., SPIFFE workload IDs).
- Add MTLSConfig with identity attributes, URISANPattern, and index fields
- Implement extractMTLSIdentity with fallback chain logic
- Move the mtls tests in the api package to pkg/api/mtls_test.go
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: prevent nil pointer dereference in RemoveImageFromRepoMeta
This commit fixes a critical bug where RemoveImageFromRepoMeta crashes with
a nil pointer dereference during retention policy execution and GC operations.
Root Cause:
The function was accessing blob metadata without checking if it exists first.
During GC/retention operations, the metadata database might have stale
references to blobs that no longer exist, causing runtime panics.
Changes:
- Added nil check for descriptorBlobInfo before accessing LastUpdated field
- Added nil check for blobInfo before dereferencing Size, Vendors, Platforms, and SubBlobs
- Made the function consistent with recalculateAggregateFields which already had these checks
Impact:
- Fixes crashes during retention policy execution
- Fixes crashes during GC manifest removal
- Fixes image deletion failures via API
- Eliminates need for dryRun: true workaround in retention config
The fix gracefully handles missing blob metadata by skipping those entries
instead of crashing.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: add comprehensive tests for RemoveImageFromRepoMeta nil checks
Add test coverage for the nil pointer dereference fixes in RemoveImageFromRepoMeta.
These tests ensure the function handles missing blob metadata gracefully during
GC and retention operations.
Test cases:
- Handle nil blob info for descriptor digest (line 280 check)
- Handle nil blob info in queue traversal (line 297 check)
- Verify correct behavior with valid blob info
- Handle empty tags edge case
- Skip tags with empty digest
Coverage: RemoveImageFromRepoMeta now has 100% test coverage
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: fix RemoveImageFromRepoMeta tests to match actual usage
Address review feedback:
- Delete tag from repoMeta.Tags before calling RemoveImageFromRepoMeta
- Fix blob count expectations after tag removal
- Add assertion to verify tag was removed from metadata
- Update comments to clarify expected behavior
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: add tag removal assertion to second test case
Add missing assertion to verify tag1 was removed from resultMeta.Tags
in the 'should handle nil blob info in queue traversal' test.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* refactor: improve nil blob handling documentation and test coverage
Address Copilot review feedback:
- Expand comment at line 278 to explain implications of skipping tags
with missing blob info, clarifying that metadata inconsistency is
acceptable in GC/cleanup scenarios
- Revise 'should handle nil blob info for descriptor digest' test to
cover more realistic scenario: remove tag1 while tag2 has missing
blob info, demonstrating graceful handling of data inconsistencies
in remaining tags during removal operations
All tests pass with 49 total assertions.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* fix: prevent nil pointer in GetCandidates when statistics missing
Add defensive check in pkg/retention/candidate.go to handle cases where
a tag exists in repoMeta.Tags but has no corresponding entry in
repoMeta.Statistics. This prevents incorrect retention decisions based
on zero-value timestamps.
Changes:
- Check statistics existence before creating candidates
- Skip tags with missing statistics (retained by GetRetainedTagsFromMetaDB)
- Improve performance from O(n*m) to O(n) by using direct map lookup
- Add comprehensive test coverage for missing statistics scenarios
This addresses the concern raised in PR #3658 about metadata
inconsistencies due to non-transactional writes to blob store and metaDB.
Related: #3658
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
* test: achieve 100% coverage for RemoveImageFromRepoMeta nil checks
Enhance test coverage for RemoveImageFromRepoMeta to address codecov failures
by adding comprehensive test cases that exercise all code paths including nil
pointer checks and continue statements.
Changes:
- Enhanced 'nil blob info for descriptor digest' test to verify processing
continues with other valid tags after skipping nil entries
- Enhanced 'nil blob info in queue traversal' test to handle mixed valid/nil
sub-blobs and verify correct processing continuation
- Added 'multiple nil blobs in deeply nested structure' test to cover complex
scenarios with multiple missing blobs at various nesting levels
- Enhanced 'skip tags with empty digest' test to verify processing continues
with valid tags after skipping empty digest entries
- Added 'combined edge cases' test to verify all edge cases work together:
empty digest, nil descriptor blob, and nil queue blob
Coverage Results:
- RemoveImageFromRepoMeta: 100.0% line coverage (was 87.50%)
- All 7 test scenarios pass with 75 total assertions
- All nil check code paths fully exercised
- All continue statement behaviors validated
Fixes codecov/patch failure on PR #3658 where 2 lines were missing coverage.
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
---------
Signed-off-by: Gianluca Boiano <morf3089@gmail.com>
- Refactored HTTP client from global cache to struct-based approach (global state was shared between tests, including what certificates to use)
- Enhanced pkg/test/tls to support ECDSA and ED25519 key types
- Replaced static certificate files with dynamic generation in golang tests
- Fixed test cleanup issues and improved resource management
This eliminates dependency on external cert generation scripts and
improves test maintainability.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat: support mTLS-only authn/authz with AccessControl and allow combining mTLS with other auth mechanisms
Signed-off-by: Ivan Arkhipov <me@endevir.ru>
* refactor: improve authentication logic and TLS certificate generation
- Fix mTLS authentication to use only leaf certificate instead of iterating
through all certificates in the chain
- Reject Authorization headers when corresponding auth method is disabled,
regardless of mTLS status (security improvement)
- Simplify authentication switch statement ordering and logic
- Move ErrUserDataNotFound error handling into sessionAuthn method
- Refactor TLS certificate generation to use Options pattern with
CertificateOptions struct for better extensibility
- Consolidate duplicate certificate generation code into helper functions
(generateCertificate, parseCA, initializeTemplate, applyOptions)
- Rename certificate generation functions for clarity:
- GenerateCertWithCN -> GenerateClientCert
- GenerateSelfSignedCertWithCN -> GenerateClientSelfSignedCert
- Add support for SAN settings including email addresses in certificates
- Update tests to reflect new authentication behavior and certificate API
This commit improves both the security posture (rejecting disabled auth
methods) and code maintainability (consolidated certificate generation).
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: guard against multiple Authorization headers
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Ivan Arkhipov <me@endevir.ru>
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Co-authored-by: Ivan Arkhipov <me@endevir.ru>
See: https://github.com/project-zot/zot/issues/3560#issuecomment-3594856118
What happens is:
- syncRef skips the image ("skipping image because it's already synced")
- syncReferrers doesn't sync anything
- CommitAll is still called even though nothing was synced
- The temp directory exists but is empty (no index.json, no blobs)
- CommitAll fails because index.json is missing
Let's ensure we properly check for errors, and skip the log messages if some of the cases.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Replace MakeTempFile usage with MakeTempFilePath and MakeTempFileWithContent
helpers that automatically handle file lifecycle. This prevents resource
leaks by ensuring temporary files are properly closed.
Shoudld also make the tests easier to read.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Add validation to reject configuration files where stores and substores
of the same storage type (local or S3) have root directories that are
nested within each other or identical. Stores of different types (local
vs S3) are allowed to share the same root directory since they use
different storage backends.
The validation:
- Checks all stores (default + substores) for path conflicts
- Only compares stores of the same storage type
- Reports clear error messages indicating which stores conflict and why
Add comprehensive tests covering:
- Same storage types with identical/nested paths (rejected)
- Different storage types with same/nested paths (allowed)
- Various combinations of default store and substores
Fixes issues where nested or identical root directories could cause
data corruption or routing conflicts.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
fix: error handling: return nil explicitly on successful completion
Several functions in pkg/meta/redis/redis.go were returning 'err' at the
end of successful execution paths, which could lead to incorrect error
handling when 'err' was overwritten in loops or conditionals.
Changed the following functions to return nil explicitly when all
operations succeed:
- SearchRepos: return nil instead of err after successful loop
- SearchTags: return nil instead of err after successful loop
- GetRepoMeta: return nil instead of err after successful operations
- GetImageMeta: return nil instead of err after successful operations
- GetReferrersInfo: return nil instead of err after successful loop
This ensures that when functions complete successfully, they explicitly
return nil rather than relying on the last value of err, which may have
been overwritten during execution. This fixes TestRedisUnreachable which
was failing because SearchRepos was incorrectly returning nil error when
Redis was unreachable.
See failure in: https://github.com/project-zot/zot/actions/runs/19729927463/job/56528529923?pr=3599
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
fix(meta): handle cases where repositories when substores are nested
Note this is a case of bad configuration: having multiple stores
in the same tree structure. Guard against it in parse.go.
Fix getAllRepos to prevent duplicate repositories in metaDB when substore
directories are nested under the default store root directory.
The fix processes substores first, then the default store, using a
map-based deduplication approach to skip repositories that have already
been added. This ensures that when both the default store and substores
contain repositories with the same name (e.g., when a substore is nested
within the default store), only one instance is added to the repository
list.
Add test TestNoDuplicateReposWithSubstoresAndNestedRepoNames to verify
the deduplication logic works correctly with nested substores.
Also update the other tests to avoid these issues in the future
this is not a vali configuration.
This is not the intended use case for substores, and it may have caused:
https://github.com/project-zot/zot/actions/runs/19665302669/job/56320640980
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
It requires the encoding/json/jsontext
package which is only available when the goexperiment.jsonv2 build
tag is enabled. This was causing build constraint errors during
tests and builds.
Changes:
- Add GOEXPERIMENT=jsonv2 to Makefile export and all go build/test
commands that use env (since env creates a fresh environment)
- Add GOEXPERIMENT=jsonv2 to GitHub workflows that use direct go
commands (workflows using make inherit it from Makefile)
Fix other dependabot alerts.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Require blob files to follow standard OCI image layout:
rootDir/repo/blobs/algorithm/digest
- Validate grandparent directory is ImageBlobsDir
- Validate parent directory is valid digest algorithm
- Update tests to use standard OCI structure
- Add blobPath() helper to reduce duplication and fix linting
This should reduce the number of uneeded digest computations
if other non-oci specific files are present in the layout.
Fix also a race condition when picking ports in monitoring tests.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Matheus Pimenta