* fix(sync): apply tag filters before destination mapping
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* fix(sync): return stable pointer from getContentByUpstreamRepo
Iterate by index and return &cm.contents[i] so callers get the slice
element rather than a copy of the loop variable, matching the existing
GetContentByLocalRepo helper.
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
---------
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* fix(security): remove InsecureSkipVerify from metrics client (TLS-1)
Replace the unconditional InsecureSkipVerify: true TLS config in
newHTTPMetricsClient with the system cert pool (+ TLS 1.2 minimum).
Add an optional CACert field to MetricsConfig and to the exporter
ServerConfig so operators running zot with a self-signed or private
CA can point the exporter at the correct CA file instead of
disabling certificate verification entirely.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* feat(metrics): add HTTPS configuration for metrics exporter
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(security): enhance CA certificate handling in metrics client and add tests
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(security): improve CA certificate error handling in metrics client and update tests
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(tests): correct package name in minimal_client_test.go and simplify error declaration
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(tests): update package name in minimal_client_test.go for consistency
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
And default it to ["auto"] when unset, with an info log from applyDefaultValues.
Refactor CVE NewScanner to take *CVEConfig instead of separate DB repository
strings so the full Trivy block is available to the scanner.
Extend CLI and search tests for the new field and logged config; document
CVE/Trivy in examples/README and add examples/config-cve-trivy.json.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
ImageIndex2ImageSummary was missing LastPullTimestamp assignment, causing
multi-arch image queries to always return null for this field. Also adds
the PushedBy field (already stored in MetaDB) to the GraphQL schema and
both conversion paths (manifest and index).
Signed-off-by: cainydev <wajo432@gmail.com>
* feat(sync): add SyncLegacyCosignTags config to skip syncing legacy cosign/SBOM tags when disabled
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: sync on demand with referrers API should not use recursion to sync referrers of referrers
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: add tests SyncLegacyCosignTags and changes in /referrers on demand sync
Credit for the tests goes to @jzhn see:
https://github.com/project-zot/zot/pull/3840/changes
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: remove redundant syncRef logic which synced referrers both with the zot inner() implementation and with regctl native implementation
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* Fix#3823: skip OCI conversion when image is already synced
When syncRef determines an image is already synced, it now returns a
bool to signal the skip. syncImage checks this and returns early before
attempting OCI conversion, preventing misleading 'failed to convert
docker image to oci' errors caused by a non-existent temp directory.
* Keep syncReferrers and CommitAll running for already-synced images
Address review feedback: new referrers can be added upstream after
initial sync, so we must not skip syncReferrers. Only the OCI
conversion is guarded by the skipped flag, since converting an
already-stored image is both unnecessary and incorrect.
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
---------
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
1. Parse repos without metadata in ParseStorage
The timestamp check in ParseStorage was skipping repos that exist in
storage but don't have metadata. When GetRepoLastUpdated returns zero
time (no metadata), we should always parse the repo to create its
metadata. Check if metaLastUpdated is zero before comparing timestamps.
If zero, always parse regardless of storageLastUpdated.
2. Change the logic of how LastUpdated is computed in RepoSummary
It is not the latest tagged timestamp from the available images or
the last updated image created timestamp, based on whichever is the
latest.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat(meta): add TaggedTimestamp field and preserve during re-parsing
Add TaggedTimestamp field to track when image tags were created, exposed
through GraphQL API. Previously, when zot restarted and re-parsed storage,
ResetRepoReferences would clear all tags, causing timestamp information to
be lost and reset to the service restart time for existing images.
This change adds TaggedTimestamp support and modifies ResetRepoReferences to
selectively preserve tags that still exist in storage, maintaining their
TaggedTimestamp values. Tags that no longer exist in storage are removed as
before.
Changes:
- Add TaggedTimestamp field to GraphQL ImageSummary schema
- Update GraphQL conversion functions to populate TaggedTimestamp with
fallback to PushTimestamp when unavailable
- Updated ResetRepoReferences interface to accept tagsToKeep parameter
- Modified ParseRepo to collect tags from storage before resetting
- Updated all backend implementations (Redis, DynamoDB, BoltDB) to preserve
tags in tagsToKeep instead of clearing all tags
- Updated tests and mocks to match new signature
This ensures TaggedTimestamp accurately reflects when tags were originally
created, and exposes this information through the GraphQL API.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
- Refactored HTTP client from global cache to struct-based approach (global state was shared between tests, including what certificates to use)
- Enhanced pkg/test/tls to support ECDSA and ED25519 key types
- Replaced static certificate files with dynamic generation in golang tests
- Fixed test cleanup issues and improved resource management
This eliminates dependency on external cert generation scripts and
improves test maintainability.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
See: https://github.com/project-zot/zot/issues/3560#issuecomment-3594856118
What happens is:
- syncRef skips the image ("skipping image because it's already synced")
- syncReferrers doesn't sync anything
- CommitAll is still called even though nothing was synced
- The temp directory exists but is empty (no index.json, no blobs)
- CommitAll fails because index.json is missing
Let's ensure we properly check for errors, and skip the log messages if some of the cases.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Replace MakeTempFile usage with MakeTempFilePath and MakeTempFileWithContent
helpers that automatically handle file lifecycle. This prevents resource
leaks by ensuring temporary files are properly closed.
Shoudld also make the tests easier to read.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
fix(meta): handle cases where repositories when substores are nested
Note this is a case of bad configuration: having multiple stores
in the same tree structure. Guard against it in parse.go.
Fix getAllRepos to prevent duplicate repositories in metaDB when substore
directories are nested under the default store root directory.
The fix processes substores first, then the default store, using a
map-based deduplication approach to skip repositories that have already
been added. This ensures that when both the default store and substores
contain repositories with the same name (e.g., when a substore is nested
within the default store), only one instance is added to the repository
list.
Add test TestNoDuplicateReposWithSubstoresAndNestedRepoNames to verify
the deduplication logic works correctly with nested substores.
Also update the other tests to avoid these issues in the future
this is not a vali configuration.
This is not the intended use case for substores, and it may have caused:
https://github.com/project-zot/zot/actions/runs/19665302669/job/56320640980
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
When GetRepoLastUpdated fails (e.g., redis: nil), it returns time.Time{}
which gets converted to &time.Time{} (not nil). The existing nil checks
in sorting functions didn't account for zero time values, which should
also be treated as "oldest" (sorted last in descending order).
This commit:
- Adds zero time checks in addition to existing nil checks
- Treats both nil and zero time values as "oldest"
- Adds comprehensive tests for zero time edge cases
Affects ImgSortByUpdateTime, RepoSortByUpdateTime, and GetExpandedRepoInfo
sort functions.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
This commit modernizes code across multiple packages by:
- Using Go 1.18+ features (slices.IndexFunc, strings.Cut)
- Pre-allocating slices and maps with known capacity
- Consolidating defensive checks and improving code clarity
- Fixing test data and build tag issues
CLI client improvements:
- Pre-allocate slices in search functions and service methods
- Replace strings.Split with strings.Cut for username:password parsing
- Use range-based iteration instead of manual index loops
Search extension optimizations:
- Cache sort functions in pagination modules
- Pre-allocate page buffers and maps
- Consolidate defensive checks in filterBaseImages/filterDerivedImages
- Fix image bas and derived logic allowing out of sequence layers for base images
- Fix image pagination reporting images groupped by repos when sorted by update time
- Remove duplicate resolver_test.go file
Monitoring extension:
- Replace manual loops with slices.IndexFunc
- Pre-allocate bucketsFloat2String map
Sync extension:
- Pre-allocate slice in parseRegistryURLs
Test utilities:
- Fix build tags in oci_layout.go
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
- Return CommitAll errors instead of ignoring them
- Skip ErrRepoNotFound from temp sync dirs to allow other tags to sync
- Each tag uses separate temp directory, so failures are isolated
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat: Sync images with a background context
This means syncs/pulls will not be cancelled anymore when the requesting client disconnects.
The timeout used can be configured per registry
Signed-off-by: Lars Francke <git@lars-francke.de>
fix: make sure metadb statistics are initialized on image download, and minor metadb fixes for Docker v2 manifest compatibility
Looking into potential causes of https://github.com/project-zot/zot/issues/3163
1. One possible reason is the statistics were not properly initialized in the first place because of (unknown and/or unavoidable) errors on image push.
To workaround this add logic to initialize the statistics on the call to download them.
2. Some images have the download statistics while others dont, one cause could be a bug in the logic handling manifest mediatypes in the search extension.
Add compatibility checks for Docker v2 manifest types in metadb convert functions, and more tests for covering the Docker mediatype use case.
Side fixes:
- Ensure PushedBy Statistics entries are properly initialized in SetRepoReference
- Fix and issue in the image upload test functions, they were uploading docker images with oci mediatypes in call headers
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Old message example:
```
{"time":"2025-11-17T22:29:29.434073547Z","level":"debug","message":"image is not scannable","digest":"sha256:99155f2825186175c7f7e4df4f77d5d0ba68a018603afd9c014063d4ca11c0d6","mediaType":"application/vnd.oci.image.manifest.v1+json","error":"image 'nginx@sha256:99155f2825186175c7f7e4df4f77d5d0ba68a018603afd9c014063d4ca11c0d6' scanning is not supported for given image media type","caller":"zotregistry.dev/zot/v2/pkg/extensions/search/cve/cve.go:513","func":"zotregistry.dev/zot/v2/pkg/extensions/search/cve.BaseCveInfo.GetCVESummaryForImageMedia","goroutine":1246}
```
New message example:
```
{"time":"2025-11-17T22:25:10.33273421Z","level":"debug","message":"image is not scannable","digest":"sha256:99155f2825186175c7f7e4df4f77d5d0ba68a018603afd9c014063d4ca11c0d6","mediaType":"application/vnd.oci.image.manifest.v1+json","error":"image 'nginx@sha256:99155f2825186175c7f7e4df4f77d5d0ba68a018603afd9c014063d4ca11c0d6' scanning is not supported for given media type: layer media type 'application/octet-stream'","caller":"zotregistry.dev/zot/v2/pkg/extensions/search/cve/cve.go:513","func":"zotregistry.dev/zot/v2/pkg/extensions/search/cve.BaseCveInfo.GetCVESummaryForImageMedia","goroutine":1316}
```
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
GC and scrub should not stop if a manifest or index is missing from storage.
Other similar changes are also included.
WRT metadb, the missing manifests cannot be added, and the results returned from metadb
do not include the descriptors for these manifests.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat: add support for sha256 and sha512 htpasswd formats
Fixes issue #3495
We currently support only bcrypt htpasswd hashes, however bcrypt is not
FIPS-140 approved since it uses Blowfish.
This PR adds support for sha256 and sha512 formats and enforces that
bcrypt be disabled when fips140 mode is enabled.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Ramkumar Chinchani