Most users don't make the difference between retention deleting untagged manifests vs GC deleting other blobs.
This causes confusion since the GC delay and the retention delay (used for untagged manifests and orphan referrers) have different defaults, and are set separately in the zot configuration.
Most users don't configrue retention policies, and they still expect untagged manifests to be deleted at GC time.
With this change, if retention delay is not specified in the config file, the value used is the GC delay.
If GC delay is also unspecified in the config file, the default GC delay is used for both.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Description
====================
zot currently stores session cookies in memory or in a local directory.
For cases where the session cookies should be independent of the
instance where they were created such as multiple instances of zot, or a
fully stateless zot instance, there is a need to support a remote
session storage.
This change adds support for using Redis and Redis-compatible services as a
remote session driver as well as introduces a new configuration option
for it.
What has changed
=======================
- New config added under Auth config to specify configuration for
the session driver.
- Examples README updated with details of the new Auth config.
- The config supports only 2 drivers in this change - local and redis
- Using the local driver is backwards compatible and behaves the same
way that zot currently works for local session storage.
- Omitting this config does not result in an error. In this case, zot
behaves as it normally does for local session storage.
- When configured, zot can use redis for persisting cookie
information for zot UI.
- The cookie in the store is deleted on logout or after the max
expiry time for the cookie.
- Configuration for the redis session driver accepts the same configuration
values as that of the remote meta cache.
- A separate connection is established for the session driver. An
existing connection for meta cache will not be re-used for the
session driver.
- A key prefix is configurable for the redis session driver. The value will be
converted into a string for use. If no value is provided, a default
prefix of "zotsession" will be used.
- Redis sessions does not support hash key or encryption in this change.
- New BATS test added to verify zot behavior with Redis session store.
- Github workflow updated to install valkey-tools dependency for BATS.
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com>
* fix: migrate from github.com/rs/zerolog to golang-native log/slog
We have been using zerolog for a really long time.
golang now has structured logging using slog.
Best to move to this in interests of long-term support.
This is a tech debt item.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: a few changes on top
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: address comments
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* chore: increase/stabilize coverage for the local storage driver
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: add/stabilize coverage for soring ImageSummary objects
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: stabilize coverage in sync tests
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
- fixes#3347: removeUntaggedManifests() did not consider compatible manifest types
- add AsDockerImage() to Image and MultiarchImage for testing
- extend TestGarbageCollectAndRetentionMetaDB to test docker image and multiarch image
Signed-off-by: Stephan Merker <stephan.merker@sap.com>
fix: allow zot to build on a FreeBSD host (#3246)
The build works as long as the protoc package is installed on the build
host. This also fixes lint checks when building on FreeBSD, working
around common lint complaints caused by the fact that rlim_t is int64 on
FreeBSD.
Signed-off-by: Doug Rabson <dfr@rabson.org>
* feat: healthz server
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
* fix: startup and readiness probe activation points
Enable startup probe at end of Controller.Init and readiness probe at
end of Controller.Run
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
* fix: rewrote to reuse same HTTP listener
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
---------
Signed-off-by: Asgeir Nilsen <asgeir@twingine.no>
Using just the last repository is not enough as in the case when it is deleted
(either by GC or some other way), GetNextRepository returns empty string
causing the generator to be marked completed without any errors.
An alternative would have been to start over from the first repository,
but this can take hours if multiple repositories need to be deleted,
not to mention the processing power and I/O and S3 load this could take.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* chore: bump github.com/olekukonko/tablewriter from 0.0.5 to 1.0.7
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* fix: zli failed to connect to https server using test certificates
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
---------
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
It is to fix#3185.
This fixes the case where MetaDB is not instantiated (none of the conditions match),
and we want to retain tags only by pattern (which should not need to use MetaBD).
Without this fix you could only use retention to delete untagged manifests.
If you specified only the key "patterns" under "keepTags", zot would crash.
It was possible to not specify "keepTags" all, which would retain all tags,
but it was not possible to retains specific tags.
Basically the case quoted below, from the documentation, was broken::
https://zotregistry.dev/v2.1.4/articles/retention/#configuration-example
```
When you specify a regex pattern with no rules other than the default, all tags matching the pattern are retained.
```
This would only work if MetaDb was instantiated by an unrelated configured feature.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
fix: metrics authorization middleware bleed (#3182)
Fixes `extension_metrics_disabled.go` to correctly isolate the authz
middleware when the metrics extension is disabled.
Signed-off-by: Matthieu Mottet <m.mottet@outlook.com>
Revert "feat(mcp): add MCP extension support with routes and configuration"
This reverts commit 56afa6bd42.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: get groups claim from idtokenclaims
Signed-off-by: Philipp Lange <ph.lange@pm.me>
* fix: lint
Signed-off-by: Philipp Lange <ph.lange@pm.me>
---------
Signed-off-by: Philipp Lange <ph.lange@pm.me>
Fixes#3005
Previously, changing a image's media-type was disallowed.
However, "docker buildx" appears to first push an image manifest and
then an image index for the same image tag. So, allow this.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* feat: rework token auth to allow ED25519/EC public keys
Signed-off-by: evanebb <git@evanus.nl>
* fix: shadow err variable to hopefully avoid data race
Signed-off-by: evanebb <git@evanus.nl>
* fix: apply golangci-lint feedback
Signed-off-by: evanebb <git@evanus.nl>
* fix: simplify public key loading by only supporting certificates, fixes ED25519 certificate handling
Signed-off-by: evanebb <git@evanus.nl>
* test: add golang-jwt based test auth server and test RSA/EC/ED25519 keys
Signed-off-by: evanebb <git@evanus.nl>
* fix: restrict allowed signing algorithms as recommended by library
Signed-off-by: evanebb <git@evanus.nl>
* test: add more bearer authorizer tests
Signed-off-by: evanebb <git@evanus.nl>
* fix: apply more golangci-lint feedback
Signed-off-by: evanebb <git@evanus.nl>
* test: ensure chmod calls run on test failure for authn errors test
Signed-off-by: evanebb <git@evanus.nl>
* fix: verify issued-at in given token if present
Pulls the validation in-line with the old library
Signed-off-by: evanebb <git@evanus.nl>
---------
Signed-off-by: evanebb <git@evanus.nl>
* feat: show more error information in zb output
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* chore(ci): gc stress tests to save logs as artifacts
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* chore: add benchmark results to job summaries
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* fix: count and show zb errors
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* ci: fix the flaky coverage of the redis logger
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
---------
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Andrei Aaron