Wait for zot PIDs (and the port) to fully shut down before restarting, and retry transient curl failures (e.g. exit 52) in wait_zot_reachable.
Send checksum verification errors to stderr for clearer failure output.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix(storage): resolve double-prefixing issue for GCS rootdirectory
Preserve double-prefixing for S3 to maintain backward compatibility with existing data. For GCS, always use "/" as rootDir to avoid double-prefixing, as GCS rootdirectory usage is a newer feature without legacy data.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(gcs): handle io.EOF correctly in Walk method
Ensure io.EOF is returned unwrapped to allow proper error handling with errors.Is() upstream.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(storage): set sensible default ("/zot") for GCS when storageDriver.rootdirectory is unset or empty or "/"
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(imagestore): avoid warning logs for expected cache miss scenarios
Refine logging to use debug level for expected cache misses, preventing unnecessary warnings.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
---------
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* ci: use zot localstack image, and consolidate on using the setup localstack GH action
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: increase zot minimal test timeout
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: setup latest localstack still available for community
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat(zb): list tests and test regex filter + misc
This change introduces the following changes to zb.
Test Filtering
===============
Allows users to selectively run tests by specifying
a standard regex that matches on the name of the test.
Test Listing
===============
Allows users to list out the available tests as well as
the matched tests when using the regex filter.
The documentation README has also been updated with
examples and the command help.
The documentation for skip cleanup has been updated.
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com>
ImageIndex2ImageSummary was missing LastPullTimestamp assignment, causing
multi-arch image queries to always return null for this field. Also adds
the PushedBy field (already stored in MetaDB) to the GraphQL schema and
both conversion paths (manifest and index).
Signed-off-by: cainydev <wajo432@gmail.com>
Validate callback_ui and default invalid values to /.
Allow absolute callback_ui only when its origin is allowlisted via http.auth.openid.callbackAllowOrigins (and externalUrl).
Add/adjust unit + controller tests and update examples/docs for relative vs allowlisted absolute redirect
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat(sync): add SyncLegacyCosignTags config to skip syncing legacy cosign/SBOM tags when disabled
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: sync on demand with referrers API should not use recursion to sync referrers of referrers
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: add tests SyncLegacyCosignTags and changes in /referrers on demand sync
Credit for the tests goes to @jzhn see:
https://github.com/project-zot/zot/pull/3840/changes
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: remove redundant syncRef logic which synced referrers both with the zot inner() implementation and with regctl native implementation
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* Fix#3823: skip OCI conversion when image is already synced
When syncRef determines an image is already synced, it now returns a
bool to signal the skip. syncImage checks this and returns early before
attempting OCI conversion, preventing misleading 'failed to convert
docker image to oci' errors caused by a non-existent temp directory.
* Keep syncReferrers and CommitAll running for already-synced images
Address review feedback: new referrers can be added upstream after
initial sync, so we must not skip syncReferrers. Only the OCI
conversion is guarded by the skipped flag, since converting an
already-stored image is both unnecessary and incorrect.
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
---------
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
feat(storage): add a GCS driver
test(storage): add unit tests for GCS driver
test(storage): add missing unit tests for GCS driver & resolve lint issues
fix: configuration validation for GCS Storage
test(storage): resolve panic by test due to setupGCS ignoring returned error
test(storage): add dummy gcs credentials
test: add darwin support for macos to run tests
ci: update workflows to pin gcs emulator version
lint: resolve long line lengths & formatting issues
test: move error for gcs mock earlier with an error
test: stop test using local google credentials and use mock instead
test: add missing dummy creds
test(storage): use storage-testbench for GCS, isolate GCS tests, fix driver Delete
- Switch GCS emulator from fake-gcs-server to storage-testbench in CI.
Run the GCS emulator only in the privileged-test job; remove it from
minimal and extended test jobs.
- Consolidate GCS tests under pkg/storage/gcs (needprivileges,linux).
Add TestMain with HTTPS proxy and /etc/hosts so tests talk to
storage-testbench; move GCS-specific cases from storage_test.go and
scrub_test.go into gcs_test.go. Run GCS tests via a second privileged-test
invocation and collect coverage in coverage-needprivileges-gcs.txt.
- Make GCS driver Delete idempotent and normalize errors. Treat
PathNotFoundError from Delete as success so that deleting an already-gone
path (e.g. after GC under eventual consistency) does not fail. Add
formatErr to map 404/not found to PathNotFoundError and use it for all
driver methods so callers get consistent storage driver errors.
- Drop GCS branches and helpers from storage_test.go and scrub_test.go so
non-privileged tests only use local/S3; GCS is tested only in
pkg/storage/gcs with storage-testbench.
- Set GCSMOCK_ENDPOINT without /storage/v1/, as the rest of the URL is set in tests.
- Show errors in case of failure to create bucket.
- Consolidate StorageDriverMock structs inside the pkg/test/mocks package.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Co-authored-by: Steven Marks <steve.marks@qomodo.io>
1. Parse repos without metadata in ParseStorage
The timestamp check in ParseStorage was skipping repos that exist in
storage but don't have metadata. When GetRepoLastUpdated returns zero
time (no metadata), we should always parse the repo to create its
metadata. Check if metaLastUpdated is zero before comparing timestamps.
If zero, always parse regardless of storageLastUpdated.
2. Change the logic of how LastUpdated is computed in RepoSummary
It is not the latest tagged timestamp from the available images or
the last updated image created timestamp, based on whichever is the
latest.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat(meta): add TaggedTimestamp field and preserve during re-parsing
Add TaggedTimestamp field to track when image tags were created, exposed
through GraphQL API. Previously, when zot restarted and re-parsed storage,
ResetRepoReferences would clear all tags, causing timestamp information to
be lost and reset to the service restart time for existing images.
This change adds TaggedTimestamp support and modifies ResetRepoReferences to
selectively preserve tags that still exist in storage, maintaining their
TaggedTimestamp values. Tags that no longer exist in storage are removed as
before.
Changes:
- Add TaggedTimestamp field to GraphQL ImageSummary schema
- Update GraphQL conversion functions to populate TaggedTimestamp with
fallback to PushTimestamp when unavailable
- Updated ResetRepoReferences interface to accept tagsToKeep parameter
- Modified ParseRepo to collect tags from storage before resetting
- Updated all backend implementations (Redis, DynamoDB, BoltDB) to preserve
tags in tagsToKeep instead of clearing all tags
- Updated tests and mocks to match new signature
This ensures TaggedTimestamp accurately reflects when tags were originally
created, and exposes this information through the GraphQL API.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* ci: add a upgrade bats test
Fixes https://github.com/project-zot/zot/issues/3601
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* ci: first check existing images
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* ci: refactor into common test code
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* refactor: Refactor upgrade tests to improve readability and maintainability
- Added section headers for release tests, upgrade process, and new tests in upgrade.bats and upgrade_minimal.bats.
- Replaced inline port retrieval with a function call to get_zot_port for consistency.
- Consolidated repeated test logic into dedicated functions (e.g., test_new_existing_pull_image, test_new_push_image).
- Removed unnecessary variable assignments and streamlined test cases for clarity.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Ramkumar Chinchani