feat: add verify-feature retention subcommand with comprehensive testing and validation
Add a `verify-feature retention` subcommand that allows users to preview and
validate retention policy changes without running the actual Zot server.
The command runs GC and retention tasks in dry-run mode for immediate feedback.
- Run verify-feature retention standalone without starting the server
- Preview retention policy decisions in dry-run mode
- Configurable GC interval override via command-line flag
- Optional timeout for task completion
- Configurable log output (stdout or file)
Basic usage:
```bash
zot verify-feature retention <config-file>
```
With log file output:
```bash
zot verify-feature retention -l /var/log/zot-retention-check.log <config-file>
```
With GC interval override (runs GC tasks every 30 seconds):
```bash
zot verify-feature retention -i 30s <config-file>
```
With timeout (wait up to 5 minutes for tasks to complete):
```bash
zot verify-feature retention -t 5m <config-file>
```
Combined flags:
```bash
zot verify-feature retention -l /var/log/zot-retention-check.log -i 1m -t 10m <config-file>
```
The command supports overriding GC settings from the config:
- `-i, --gc-interval`: Override the GC interval setting (applies to all storage paths including subpaths)
- Refactored `RunGCTasks` from `controller.go` to be reusable
- Added `checkServerRunning` validation to prevent conflicts
- Implemented signal handling for graceful shutdown
- Added configuration sanitization and logging
- Set GCMaxSchedulerDelay programmatically (not user-configurable)
Added tests for coverage on main function:
- Negative test cases (no args, bad config, GC disabled, server running)
- Both BoltDB and Redis
- Retention enabled scenarios with complex image setups
- Retention disabled scenarios
- Delete referrers functionality
- Subpaths configuration
- GC interval override validation
Run the verify-feature retention tests:
```bash
go test -v ./pkg/cli/server -run TestRetentionCheck
```
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Make the Secure flag for session cookies configurable based on Zot's
TLS settings. This allows cookies to work properly when Zot is
accessed over HTTP (without TLS).
Changes:
- Add SecureSession field to AuthConfig to allow explicit control
- Add UseSecureSession() method that returns true when TLS is
configured, or uses SecureSession setting if provided
- Update saveUserLoggedSession() to accept and use secure parameter
- Add tests for UseSecureSession() in config_test.go
- Enhance authn tests to verify cookie Secure flag behavior
- Fix TestAuthnSessionErrors by creating new client without cookies
The logic is:
- If TLS is configured, cookies always have Secure=true
- If TLS is not configured but SecureSession is explicitly set,
use that value
- Otherwise, default to Secure=false for HTTP-only deployments
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: make config read/write thread safe and fix some other similar issues
1. The config config has a lock, and safe methods to update and read the attributes
2. The config has methods to retrieve copies of specific attributes, such as the extyensions config, the auth config, and the authz config.
These are needed, as the config object may mutate in the middle of an auth/authz requests, and we avoid partial configuration being applied for that request.
3. Fix an issue with the monitoring server not stopping when the controller is shut down.
4. Fix an issue with the HTPasswdWatcher not stopping when the background tasks are supposed to finish.
5. Fix some tests using hardcoded ports.
Moved some of the methods which were on the main config to the auth, access control and extension configs
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: migrate to Go module v2 for proper semantic versioning
This change updates the module path from 'zotregistry.dev/zot' to
'zotregistry.dev/zot/v2' to comply with Go's semantic versioning rules.
According to Go's module versioning requirements, major version v2+
must include the major version in the module path. The current
module path 'zotregistry.dev/zot' only supports v0.x.x and v1.x.x
versions, making existing v2.x.x tags (like v2.1.8) unusable.
Changes:
- Updated go.mod module path to zotregistry.dev/zot/v2
- Updated all internal import paths across 280+ Go source files
- Updated configuration files (golangcilint.yaml, gqlgen.yml)
- Updated README.md Go reference badge
This fix enables proper use of existing v2.x.x Git tags and allows
external packages to import zot v2+ versions without compatibility
errors.
Resolves: Go module import compatibility for v2+ versions
Fixes: #3071
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
* fix: regenerate GraphQL files with updated v2 import paths
The gqlgen tool needs to regenerate the GraphQL schema files after
the module path change to use the new v2 imports.
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
---------
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
Most users don't make the difference between retention deleting untagged manifests vs GC deleting other blobs.
This causes confusion since the GC delay and the retention delay (used for untagged manifests and orphan referrers) have different defaults, and are set separately in the zot configuration.
Most users don't configrue retention policies, and they still expect untagged manifests to be deleted at GC time.
With this change, if retention delay is not specified in the config file, the value used is the GC delay.
If GC delay is also unspecified in the config file, the default GC delay is used for both.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Description
====================
zot currently stores session cookies in memory or in a local directory.
For cases where the session cookies should be independent of the
instance where they were created such as multiple instances of zot, or a
fully stateless zot instance, there is a need to support a remote
session storage.
This change adds support for using Redis and Redis-compatible services as a
remote session driver as well as introduces a new configuration option
for it.
What has changed
=======================
- New config added under Auth config to specify configuration for
the session driver.
- Examples README updated with details of the new Auth config.
- The config supports only 2 drivers in this change - local and redis
- Using the local driver is backwards compatible and behaves the same
way that zot currently works for local session storage.
- Omitting this config does not result in an error. In this case, zot
behaves as it normally does for local session storage.
- When configured, zot can use redis for persisting cookie
information for zot UI.
- The cookie in the store is deleted on logout or after the max
expiry time for the cookie.
- Configuration for the redis session driver accepts the same configuration
values as that of the remote meta cache.
- A separate connection is established for the session driver. An
existing connection for meta cache will not be re-used for the
session driver.
- A key prefix is configurable for the redis session driver. The value will be
converted into a string for use. If no value is provided, a default
prefix of "zotsession" will be used.
- Redis sessions does not support hash key or encryption in this change.
- New BATS test added to verify zot behavior with Redis session store.
- Github workflow updated to install valkey-tools dependency for BATS.
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com>
* fix: migrate from github.com/rs/zerolog to golang-native log/slog
We have been using zerolog for a really long time.
golang now has structured logging using slog.
Best to move to this in interests of long-term support.
This is a tech debt item.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: a few changes on top
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: address comments
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
fix: allow zot to build on a FreeBSD host (#3246)
The build works as long as the protoc package is installed on the build
host. This also fixes lint checks when building on FreeBSD, working
around common lint complaints caused by the fact that rlim_t is int64 on
FreeBSD.
Signed-off-by: Doug Rabson <dfr@rabson.org>
* feat: healthz server
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
* fix: startup and readiness probe activation points
Enable startup probe at end of Controller.Init and readiness probe at
end of Controller.Run
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
* fix: rewrote to reuse same HTTP listener
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
---------
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
Revert "feat(mcp): add MCP extension support with routes and configuration"
This reverts commit 56afa6bd42.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: get groups claim from idtokenclaims
Signed-off-by: Philipp Lange <ph.lange@pm.me>
* fix: lint
Signed-off-by: Philipp Lange <ph.lange@pm.me>
---------
Signed-off-by: Philipp Lange <ph.lange@pm.me>
Fixes#3005
Previously, changing a image's media-type was disallowed.
However, "docker buildx" appears to first push an image manifest and
then an image index for the same image tag. So, allow this.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* feat: rework token auth to allow ED25519/EC public keys
Signed-off-by: evanebb <git@evanus.nl>
* fix: shadow err variable to hopefully avoid data race
Signed-off-by: evanebb <git@evanus.nl>
* fix: apply golangci-lint feedback
Signed-off-by: evanebb <git@evanus.nl>
* fix: simplify public key loading by only supporting certificates, fixes ED25519 certificate handling
Signed-off-by: evanebb <git@evanus.nl>
* test: add golang-jwt based test auth server and test RSA/EC/ED25519 keys
Signed-off-by: evanebb <git@evanus.nl>
* fix: restrict allowed signing algorithms as recommended by library
Signed-off-by: evanebb <git@evanus.nl>
* test: add more bearer authorizer tests
Signed-off-by: evanebb <git@evanus.nl>
* fix: apply more golangci-lint feedback
Signed-off-by: evanebb <git@evanus.nl>
* test: ensure chmod calls run on test failure for authn errors test
Signed-off-by: evanebb <git@evanus.nl>
* fix: verify issued-at in given token if present
Pulls the validation in-line with the old library
Signed-off-by: evanebb <git@evanus.nl>
---------
Signed-off-by: evanebb <git@evanus.nl>
* feat: show more error information in zb output
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* chore(ci): gc stress tests to save logs as artifacts
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* chore: add benchmark results to job summaries
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* fix: count and show zb errors
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* ci: fix the flaky coverage of the redis logger
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
---------
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(htpasswd): move htpasswd processing to a helper struct and add reload
Signed-off-by: Vladimir Ermakov <vooon341@gmail.com>
* feat(htpasswd): use dedicated fsnotify reloader for htpasswd file
- rewrite htpasswd watcher not to store context
- improve logging
Signed-off-by: Vladimir Ermakov <vooon341@gmail.com>
* feat(htpasswd): add htpasswd reload test
Signed-off-by: Vladimir Ermakov <vooon341@gmail.com>
---------
Signed-off-by: Vladimir Ermakov <vooon341@gmail.com>
* feat: add redis cache support
https://github.com/project-zot/zot/pull/2005
Fixes https://github.com/project-zot/zot/issues/2004
* feat: add redis cache support
Currently, we have dynamoDB as the remote shared cache but ideal only
for the cloud use case.
For on-prem use case, add support for redis.
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* feat(redis): added blackbox tests for redis
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* feat(redis): dummy implementation of MetaDB interface for redis cache
Signed-off-by: Alexei Dodon <adodon@cisco.com>
* feat: check validity of driver configuration on metadb instantiation
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat: multiple fixes for redis cache driver implementation
- add missing method GetAllBlobs
- add redis cache tests, with and without mocking
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): redis implementation for MetaDB
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): use redsync to block concurrent write access to the redis DB
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): update .github/workflows/cluster.yaml to also test redis
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(metadb): add keyPrefix parameter for redis and remove unneeded method meta.Crate()
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): support RedisCluster configuration and add unit tests
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): more tests for redis metadb implementation
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): add more examples and update examples/README.md
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): move option parsing and redis client initialization under pkg/api/config/redis
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* chore(cachedb): move Cache interface to pkg/storage/types
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): reorganize code in pkg/storage/cache.go
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): call redis.SetLogger() with the zot logger as parameter
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): rename pkg/meta/redisdb to pkg/meta/redis
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Signed-off-by: Alexei Dodon <adodon@cisco.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: a <a@tuxpa.in>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
Co-authored-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: Alexei Dodon <adodon@cisco.com>
* feat: add support for docker images
Issue #724
A new config section under "HTTP" called "Compat" is added which
currently takes a list of possible compatible legacy media-types.
https://github.com/opencontainers/image-spec/blob/main/media-types.md#compatibility-matrix
Only "docker2s2" (Docker Manifest V2 Schema V2) is currently supported.
Garbage collection also needs to be made aware of non-OCI compatible
layer types.
feat: add cve support for non-OCI compatible layer types
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
*
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* test: add more docker compat tests
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* feat: add additional validation checks for non-OCI images
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* ci: make "full" images docker-compatible
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
In case of delete by tag only the tag is removed, the manifest itself would continue to be accessible by digest.
In case of delete by digest the manifest would be completely removed (provided it is not used by an index or another reference).
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
fix(authn): configurable hashing/encryption keys used to secure cookies
If they are not configured zot will generate a random hashing key at startup,
invalidating all cookies if zot is restarted. closes: #2526
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
when a client pushes an image zot's inline dedupe
will try to find the blob path corresponding with the blob digest
that it's currently pushed and if it's found in the cache
then zot will make a symbolic link to that cache entry and report
to the client that the blob already exists on the location.
Before this patch authorization was not applied on this process meaning
that a user could copy blobs without having permissions on the source repo.
Added a rule which says that the client should have read permissions on the source repo
before deduping, otherwise just Stat() the blob and return the corresponding status code.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: Petu Eusebiu <peusebiu@cisco.com>
This commit includes support for periodic repo sync in a scale-out
cluster.
Before this commit, all cluster members would sync all the repos as
the config is shared.
With this change, in periodic sync, the cluster member checks whether
it manages the repo. If it does not manage the repo, it will skip the
sync.
This commit also includes a unit test to test on-demand sync too, but
there are no logic changes for it as it is implicitly handled by the
proxying logic.
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
* feat(cluster): initial commit for scale-out cluster
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* feat(cluster): support shared storage scale out
This change introduces support for shared storage backed
zot cluster scale out.
New feature
Multiple stateless zot instances can run using the same shared
storage backend where each instance looks at a specific set
of repositories based on a siphash of the repository name to improve
scale as the load is distributed across multiple instances.
For a given config, there will only be one instance that can perform
dist-spec read/write on a given repository.
What's changed?
- introduced a transparent request proxy for dist-spec endpoints based on
siphash of repository name.
- new config for scale out cluster that specifies list of
cluster members.
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
Ramkumar Chinchani