* fix(api): set blob response Content-Type from OCI descriptor
Blob HEAD responses had no Content-Type and GET responses echoed the
request's Accept header verbatim, which produced missing or malformed
media types and left multipart/byteranges parts without a per-part
Content-Type. This breaks OCI distribution-spec conformance and
consumers like stargz-snapshotter that need a well-formed layer media
type.
Add a blobResponseMediaType helper that resolves the descriptor's
MediaType via GetBlobDescriptorFromRepo and falls back to
application/octet-stream. Use it in CheckBlob (HEAD), GetBlob full
(200), GetBlob single-range (206), and per-part in
writeMultipartRanges (206 multipart). Lookup is deferred until after
the blob is known to exist.
Cover the new behaviour with mock-based unit tests in routes_test.go
and end-to-end assertions in TestPullRange.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* perf(api): stream multipart blob ranges lazily with precomputed Content-Length
writeMultipartRanges previously opened every range reader up front
and emitted no Content-Length, so an N-range request held N
concurrent storage readers (and their fds / read buffers) per
response window and forced chunked encoding on HTTP/1.1 — neither
friendly to proxies nor to fan-out scenarios like stargz lazy pulls.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix(api): support multipart range blob pulls
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* fix(api): tighten multipart range response
- Drop the redundant deferred closeRangeReaders; the deferred cleanup
registered when the slice is created already covers all paths.
- Stop copying the request Accept header into each multipart part's
Content-Type. Accept can be a list of media ranges (e.g.
"application/octet-stream,*/*"), which is not a valid Content-Type and
may confuse multipart parsers. RFC 9110 lets us omit it entirely.
- Set Docker-Content-Digest on the partial-content response so range
pulls expose the same header as a full GET.
- Drop the over-broad build tag on routes_internal_test.go; the parser
unit test does not need any extension build tags.
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
---------
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
POST /zot/auth/logout now returns an endSessionUrl in the JSON
response body when the session was established via an OIDC provider
whose discovery document advertises an endSessionEndpoint, so the
UI can navigate the browser to it and terminate the session at the
IdP in addition to clearing the local cookie.
- The OIDC callback records the provider name in the session after
login; the github OAuth2 path is untouched.
- end_session_endpoint is read from the zitadel/oidc RelyingParty
and validated as an absolute http(s) URL.
- post_logout_redirect_uri prefers http.externalUrl when set and
falls back to deriving the origin from the incoming request.
- No id_token_hint is sent; client_id identifies the RP, so the
ID token does not need to be persisted.
- Non-OIDC sessions (local/basic/LDAP/GitHub) retain the existing
200 OK, no body behavior.
Operators must register the URI zot sends as a valid post-logout
redirect URI on the IdP client.
Ref: https://openid.net/specs/openid-connect-rpinitiated-1_0.html
Signed-off-by: Nikita Vakula <programmistov.programmist@gmail.com>
Docker Compose and Buildx proxy through the Docker daemon, which sends
a User-Agent starting with "docker/<version>" rather than the
"Docker-Client/<version>" string sent by direct Docker CLI pulls.
This caused compose/buildx pulls to skip the 401 challenge on
registries with mixed anonymous/authenticated access policies,
resulting in 'unauthorized' errors.
Add strings.HasPrefix(ua, "docker/") alongside the existing
Docker-Client check so daemon-proxied requests from any upstream
tool (compose, buildx, etc.) are handled correctly.
Fixes#3991
Align closing blob upload (PUT) with the OCI Distribution Spec: invalid /
out-of-order upload ranges (ErrBadUploadRange) return 416 Requested Range Not
Satisfiable instead of 400, for both the final-chunk PutBlobChunk path and
FinishBlobUpload.
GetBlobUpload (GET upload status): fix the Range response when zero bytes have
been received—send Range: 0-0 instead of Range: 0--1, consistent with a new
session and the spec’s Location + Range upload status shape. Only map
ErrBadBlobDigest to 400 here; do not handle ErrBadUploadRange on GET (that
request carries no range; ImageStore.GetBlobUpload does not return it).
Document PUT upload failures 400 and 416 in swagger; regenerate swagger
artifacts. Update route tests (expect 416 on UpdateBlobUpload for
ErrBadUploadRange), drop the mock-only GetBlobUpload+ErrBadUploadRange case,
and assert Range: 0-0 in TestPullRange after GET on a new upload location.
Fix potential panic when parsing Content-Range (index out of range)
when accessing `tokens[0]`.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
fix(security): suppress Allow-Credentials on wildcard CORS origin (CORS-1)
Per CORS spec §3.2, Access-Control-Allow-Credentials must not be
"true" when Access-Control-Allow-Origin is the wildcard "*".
ACHeadersMiddleware (pkg/common/http_server.go) and
getUIHeadersHandler (pkg/api/routes.go) now only emit the
credentials header when an explicit, non-empty AllowOrigin is
configured. Deployments that leave AllowOrigin blank (default
wildcard) no longer produce a contradictory header pair.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Wrap req.Body with http.MaxBytesReader before io.ReadAll in
CreateAPIKey. Requests with bodies larger than MaxAPIKeyBodySize
(4 KiB) now return HTTP 413 instead of buffering arbitrary data.
Add the MaxAPIKeyBodySize constant, update the Swagger @Failure
annotation to document 413, and add a unit test.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Wrap request.Body with http.MaxBytesReader before io.ReadAll in
UpdateManifest. Bodies exceeding MaxManifestBodySize (4 MiB) now
return HTTP 413 with a MANIFEST_INVALID error body instead of
buffering unlimited data into memory.
Add the MaxManifestBodySize constant and a unit test that sends an
oversized body and asserts the 413 status.
Agent-Logs-Url: https://github.com/project-zot/zot/sessions/5eca86eb-9749-4cf8-9fb8-7b9ace2ba87f
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(auth): add workaround for Docker client auth with mixed anonymous policies
Docker client fails to authenticate to protected repositories when basic auth
(htpasswd/LDAP) is used with mixed access policies (some repos anonymous,
some requiring auth). This happens because Docker determines whether to send
credentials based on the /v2/ response - if it returns 200, Docker assumes
no auth is needed anywhere.
Add `forceDockerClientAuth` config option that, when enabled, forces 401 on
/v2/ for Docker clients, triggering Docker's authentication flow.
This workaround only affects Docker clients (detected via User-Agent).
Podman and other OCI-compliant clients are unaffected.
Refs: https://github.com/opencontainers/wg-auth/blob/main/docs/implementations/moby.md
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat: remove ForceDockerClientAuth flag and use only authz policies to determine the docker specific behavior
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Adds a configurable maximum repository count per registry instance.
When maxRepos is set on StorageConfig, manifest pushes that would create
a new repository beyond the limit are rejected with HTTP 429
TOOMANYREQUESTS. Pushes to existing repositories are always allowed.
Implemented as an always-available feature in pkg/api (not a build-tag
extension). MaxRepos is a field on StorageConfig, enabled when > 0.
- repoQuotaMiddleware on the dist-spec router intercepts manifest PUTs.
New-repo pushes are serialized with a sync.Mutex to prevent concurrent
requests from exceeding the limit.
- Adds CountRepos(ctx) to the MetaDB interface with efficient
implementations: BoltDB (Stats().KeyN), Redis (HLen), DynamoDB
(Scan with Select=COUNT).
- Config.IsQuotaEnabled() added, wired into controller.go metaDB init.
- Four integration tests (enforcement, concurrency, disabled,
unconfigured) and backend-specific CountRepos tests for BoltDB, Redis,
and DynamoDB.
Signed-off-by: Bachir Khiati <bachir.khiati@gmail.com>
Validate callback_ui and default invalid values to /.
Allow absolute callback_ui only when its origin is allowlisted via http.auth.openid.callbackAllowOrigins (and externalUrl).
Add/adjust unit + controller tests and update examples/docs for relative vs allowlisted absolute redirect
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat(storage): add a GCS driver
test(storage): add unit tests for GCS driver
test(storage): add missing unit tests for GCS driver & resolve lint issues
fix: configuration validation for GCS Storage
test(storage): resolve panic by test due to setupGCS ignoring returned error
test(storage): add dummy gcs credentials
test: add darwin support for macos to run tests
ci: update workflows to pin gcs emulator version
lint: resolve long line lengths & formatting issues
test: move error for gcs mock earlier with an error
test: stop test using local google credentials and use mock instead
test: add missing dummy creds
test(storage): use storage-testbench for GCS, isolate GCS tests, fix driver Delete
- Switch GCS emulator from fake-gcs-server to storage-testbench in CI.
Run the GCS emulator only in the privileged-test job; remove it from
minimal and extended test jobs.
- Consolidate GCS tests under pkg/storage/gcs (needprivileges,linux).
Add TestMain with HTTPS proxy and /etc/hosts so tests talk to
storage-testbench; move GCS-specific cases from storage_test.go and
scrub_test.go into gcs_test.go. Run GCS tests via a second privileged-test
invocation and collect coverage in coverage-needprivileges-gcs.txt.
- Make GCS driver Delete idempotent and normalize errors. Treat
PathNotFoundError from Delete as success so that deleting an already-gone
path (e.g. after GC under eventual consistency) does not fail. Add
formatErr to map 404/not found to PathNotFoundError and use it for all
driver methods so callers get consistent storage driver errors.
- Drop GCS branches and helpers from storage_test.go and scrub_test.go so
non-privileged tests only use local/S3; GCS is tested only in
pkg/storage/gcs with storage-testbench.
- Set GCSMOCK_ENDPOINT without /storage/v1/, as the rest of the URL is set in tests.
- Show errors in case of failure to create bucket.
- Consolidate StorageDriverMock structs inside the pkg/test/mocks package.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Co-authored-by: Steven Marks <steve.marks@qomodo.io>
Add support for configurable identity attributes in mTLS authentication,
allowing identity extraction from CommonName, Subject DN, Email SAN,
URI SAN, or DNSName SAN with fallback chain support. Includes regex
pattern matching for URI SANs (e.g., SPIFFE workload IDs).
- Add MTLSConfig with identity attributes, URISANPattern, and index fields
- Implement extractMTLSIdentity with fallback chain logic
- Move the mtls tests in the api package to pkg/api/mtls_test.go
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
- Refactored HTTP client from global cache to struct-based approach (global state was shared between tests, including what certificates to use)
- Enhanced pkg/test/tls to support ECDSA and ED25519 key types
- Replaced static certificate files with dynamic generation in golang tests
- Fixed test cleanup issues and improved resource management
This eliminates dependency on external cert generation scripts and
improves test maintainability.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat: support mTLS-only authn/authz with AccessControl and allow combining mTLS with other auth mechanisms
Signed-off-by: Ivan Arkhipov <me@endevir.ru>
* refactor: improve authentication logic and TLS certificate generation
- Fix mTLS authentication to use only leaf certificate instead of iterating
through all certificates in the chain
- Reject Authorization headers when corresponding auth method is disabled,
regardless of mTLS status (security improvement)
- Simplify authentication switch statement ordering and logic
- Move ErrUserDataNotFound error handling into sessionAuthn method
- Refactor TLS certificate generation to use Options pattern with
CertificateOptions struct for better extensibility
- Consolidate duplicate certificate generation code into helper functions
(generateCertificate, parseCA, initializeTemplate, applyOptions)
- Rename certificate generation functions for clarity:
- GenerateCertWithCN -> GenerateClientCert
- GenerateSelfSignedCertWithCN -> GenerateClientSelfSignedCert
- Add support for SAN settings including email addresses in certificates
- Update tests to reflect new authentication behavior and certificate API
This commit improves both the security posture (rejecting disabled auth
methods) and code maintainability (consolidated certificate generation).
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: guard against multiple Authorization headers
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Ivan Arkhipov <me@endevir.ru>
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Co-authored-by: Ivan Arkhipov <me@endevir.ru>
Replace MakeTempFile usage with MakeTempFilePath and MakeTempFileWithContent
helpers that automatically handle file lifecycle. This prevents resource
leaks by ensuring temporary files are properly closed.
Shoudld also make the tests easier to read.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat: add support for sha256 and sha512 htpasswd formats
Fixes issue #3495
We currently support only bcrypt htpasswd hashes, however bcrypt is not
FIPS-140 approved since it uses Blowfish.
This PR adds support for sha256 and sha512 formats and enforces that
bcrypt be disabled when fips140 mode is enabled.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
- Use custom authURL/tokenURL from config instead of hardcoded github.com endpoints
- Properly configure GitHub Enterprise API base URL from auth endpoints
Fixes OAuth2 authentication with GitHub Enterprise Server and other
self-hosted OAuth2 providers.
Signed-off-by: Mathias Bogaert <mathias.bogaert@gmail.com>
feat: add verify-feature retention subcommand with comprehensive testing and validation
Add a `verify-feature retention` subcommand that allows users to preview and
validate retention policy changes without running the actual Zot server.
The command runs GC and retention tasks in dry-run mode for immediate feedback.
- Run verify-feature retention standalone without starting the server
- Preview retention policy decisions in dry-run mode
- Configurable GC interval override via command-line flag
- Optional timeout for task completion
- Configurable log output (stdout or file)
Basic usage:
```bash
zot verify-feature retention <config-file>
```
With log file output:
```bash
zot verify-feature retention -l /var/log/zot-retention-check.log <config-file>
```
With GC interval override (runs GC tasks every 30 seconds):
```bash
zot verify-feature retention -i 30s <config-file>
```
With timeout (wait up to 5 minutes for tasks to complete):
```bash
zot verify-feature retention -t 5m <config-file>
```
Combined flags:
```bash
zot verify-feature retention -l /var/log/zot-retention-check.log -i 1m -t 10m <config-file>
```
The command supports overriding GC settings from the config:
- `-i, --gc-interval`: Override the GC interval setting (applies to all storage paths including subpaths)
- Refactored `RunGCTasks` from `controller.go` to be reusable
- Added `checkServerRunning` validation to prevent conflicts
- Implemented signal handling for graceful shutdown
- Added configuration sanitization and logging
- Set GCMaxSchedulerDelay programmatically (not user-configurable)
Added tests for coverage on main function:
- Negative test cases (no args, bad config, GC disabled, server running)
- Both BoltDB and Redis
- Retention enabled scenarios with complex image setups
- Retention disabled scenarios
- Delete referrers functionality
- Subpaths configuration
- GC interval override validation
Run the verify-feature retention tests:
```bash
go test -v ./pkg/cli/server -run TestRetentionCheck
```
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Make the Secure flag for session cookies configurable based on Zot's
TLS settings. This allows cookies to work properly when Zot is
accessed over HTTP (without TLS).
Changes:
- Add SecureSession field to AuthConfig to allow explicit control
- Add UseSecureSession() method that returns true when TLS is
configured, or uses SecureSession setting if provided
- Update saveUserLoggedSession() to accept and use secure parameter
- Add tests for UseSecureSession() in config_test.go
- Enhance authn tests to verify cookie Secure flag behavior
- Fix TestAuthnSessionErrors by creating new client without cookies
The logic is:
- If TLS is configured, cookies always have Secure=true
- If TLS is not configured but SecureSession is explicitly set,
use that value
- Otherwise, default to Secure=false for HTTP-only deployments
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: make config read/write thread safe and fix some other similar issues
1. The config config has a lock, and safe methods to update and read the attributes
2. The config has methods to retrieve copies of specific attributes, such as the extyensions config, the auth config, and the authz config.
These are needed, as the config object may mutate in the middle of an auth/authz requests, and we avoid partial configuration being applied for that request.
3. Fix an issue with the monitoring server not stopping when the controller is shut down.
4. Fix an issue with the HTPasswdWatcher not stopping when the background tasks are supposed to finish.
5. Fix some tests using hardcoded ports.
Moved some of the methods which were on the main config to the auth, access control and extension configs
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: migrate to Go module v2 for proper semantic versioning
This change updates the module path from 'zotregistry.dev/zot' to
'zotregistry.dev/zot/v2' to comply with Go's semantic versioning rules.
According to Go's module versioning requirements, major version v2+
must include the major version in the module path. The current
module path 'zotregistry.dev/zot' only supports v0.x.x and v1.x.x
versions, making existing v2.x.x tags (like v2.1.8) unusable.
Changes:
- Updated go.mod module path to zotregistry.dev/zot/v2
- Updated all internal import paths across 280+ Go source files
- Updated configuration files (golangcilint.yaml, gqlgen.yml)
- Updated README.md Go reference badge
This fix enables proper use of existing v2.x.x Git tags and allows
external packages to import zot v2+ versions without compatibility
errors.
Resolves: Go module import compatibility for v2+ versions
Fixes: #3071
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
* fix: regenerate GraphQL files with updated v2 import paths
The gqlgen tool needs to regenerate the GraphQL schema files after
the module path change to use the new v2 imports.
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
---------
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
Most users don't make the difference between retention deleting untagged manifests vs GC deleting other blobs.
This causes confusion since the GC delay and the retention delay (used for untagged manifests and orphan referrers) have different defaults, and are set separately in the zot configuration.
Most users don't configrue retention policies, and they still expect untagged manifests to be deleted at GC time.
With this change, if retention delay is not specified in the config file, the value used is the GC delay.
If GC delay is also unspecified in the config file, the default GC delay is used for both.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Description
====================
zot currently stores session cookies in memory or in a local directory.
For cases where the session cookies should be independent of the
instance where they were created such as multiple instances of zot, or a
fully stateless zot instance, there is a need to support a remote
session storage.
This change adds support for using Redis and Redis-compatible services as a
remote session driver as well as introduces a new configuration option
for it.
What has changed
=======================
- New config added under Auth config to specify configuration for
the session driver.
- Examples README updated with details of the new Auth config.
- The config supports only 2 drivers in this change - local and redis
- Using the local driver is backwards compatible and behaves the same
way that zot currently works for local session storage.
- Omitting this config does not result in an error. In this case, zot
behaves as it normally does for local session storage.
- When configured, zot can use redis for persisting cookie
information for zot UI.
- The cookie in the store is deleted on logout or after the max
expiry time for the cookie.
- Configuration for the redis session driver accepts the same configuration
values as that of the remote meta cache.
- A separate connection is established for the session driver. An
existing connection for meta cache will not be re-used for the
session driver.
- A key prefix is configurable for the redis session driver. The value will be
converted into a string for use. If no value is provided, a default
prefix of "zotsession" will be used.
- Redis sessions does not support hash key or encryption in this change.
- New BATS test added to verify zot behavior with Redis session store.
- Github workflow updated to install valkey-tools dependency for BATS.
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com>
* fix: migrate from github.com/rs/zerolog to golang-native log/slog
We have been using zerolog for a really long time.
golang now has structured logging using slog.
Best to move to this in interests of long-term support.
This is a tech debt item.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: a few changes on top
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: address comments
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Andrei Aaron