* fix(api): set blob response Content-Type from OCI descriptor
Blob HEAD responses had no Content-Type and GET responses echoed the
request's Accept header verbatim, which produced missing or malformed
media types and left multipart/byteranges parts without a per-part
Content-Type. This breaks OCI distribution-spec conformance and
consumers like stargz-snapshotter that need a well-formed layer media
type.
Add a blobResponseMediaType helper that resolves the descriptor's
MediaType via GetBlobDescriptorFromRepo and falls back to
application/octet-stream. Use it in CheckBlob (HEAD), GetBlob full
(200), GetBlob single-range (206), and per-part in
writeMultipartRanges (206 multipart). Lookup is deferred until after
the blob is known to exist.
Cover the new behaviour with mock-based unit tests in routes_test.go
and end-to-end assertions in TestPullRange.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* perf(api): stream multipart blob ranges lazily with precomputed Content-Length
writeMultipartRanges previously opened every range reader up front
and emitted no Content-Length, so an N-range request held N
concurrent storage readers (and their fds / read buffers) per
response window and forced chunked encoding on HTTP/1.1 — neither
friendly to proxies nor to fan-out scenarios like stargz lazy pulls.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix(sync): apply tag filters before destination mapping
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* fix(sync): return stable pointer from getContentByUpstreamRepo
Iterate by index and return &cm.contents[i] so callers get the slice
element rather than a copy of the loop variable, matching the existing
GetContentByLocalRepo helper.
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
---------
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* fix(api): support multipart range blob pulls
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* fix(api): tighten multipart range response
- Drop the redundant deferred closeRangeReaders; the deferred cleanup
registered when the slice is created already covers all paths.
- Stop copying the request Accept header into each multipart part's
Content-Type. Accept can be a list of media ranges (e.g.
"application/octet-stream,*/*"), which is not a valid Content-Type and
may confuse multipart parsers. RFC 9110 lets us omit it entirely.
- Set Docker-Content-Digest on the partial-content response so range
pulls expose the same header as a full GET.
- Drop the over-broad build tag on routes_internal_test.go; the parser
unit test does not need any extension build tags.
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
---------
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
POST /zot/auth/logout now returns an endSessionUrl in the JSON
response body when the session was established via an OIDC provider
whose discovery document advertises an endSessionEndpoint, so the
UI can navigate the browser to it and terminate the session at the
IdP in addition to clearing the local cookie.
- The OIDC callback records the provider name in the session after
login; the github OAuth2 path is untouched.
- end_session_endpoint is read from the zitadel/oidc RelyingParty
and validated as an absolute http(s) URL.
- post_logout_redirect_uri prefers http.externalUrl when set and
falls back to deriving the origin from the incoming request.
- No id_token_hint is sent; client_id identifies the RP, so the
ID token does not need to be persisted.
- Non-OIDC sessions (local/basic/LDAP/GitHub) retain the existing
200 OK, no body behavior.
Operators must register the URI zot sends as a valid post-logout
redirect URI on the IdP client.
Ref: https://openid.net/specs/openid-connect-rpinitiated-1_0.html
Signed-off-by: Nikita Vakula <programmistov.programmist@gmail.com>
Docker Compose and Buildx proxy through the Docker daemon, which sends
a User-Agent starting with "docker/<version>" rather than the
"Docker-Client/<version>" string sent by direct Docker CLI pulls.
This caused compose/buildx pulls to skip the 401 challenge on
registries with mixed anonymous/authenticated access policies,
resulting in 'unauthorized' errors.
Add strings.HasPrefix(ua, "docker/") alongside the existing
Docker-Client check so daemon-proxied requests from any upstream
tool (compose, buildx, etc.) are handled correctly.
Fixes#3991
feat(zb): support for measuring TTFB + sync tests
Adds supports for measuring time to first byte (TTFB)
for Pull tests for the manifest check, the manifest get,
the config get, and the blob get.
Additionally, this introduces 2 new sync tests which measure
the performance of on-demand sync.
Setup code has been refactored to accomodate a new
blob size of 1GB.
Parts of zb have been refactored to address linter errors.
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com>
Align closing blob upload (PUT) with the OCI Distribution Spec: invalid /
out-of-order upload ranges (ErrBadUploadRange) return 416 Requested Range Not
Satisfiable instead of 400, for both the final-chunk PutBlobChunk path and
FinishBlobUpload.
GetBlobUpload (GET upload status): fix the Range response when zero bytes have
been received—send Range: 0-0 instead of Range: 0--1, consistent with a new
session and the spec’s Location + Range upload status shape. Only map
ErrBadBlobDigest to 400 here; do not handle ErrBadUploadRange on GET (that
request carries no range; ImageStore.GetBlobUpload does not return it).
Document PUT upload failures 400 and 416 in swagger; regenerate swagger
artifacts. Update route tests (expect 416 on UpdateBlobUpload for
ErrBadUploadRange), drop the mock-only GetBlobUpload+ErrBadUploadRange case,
and assert Range: 0-0 in TestPullRange after GET on a new upload location.
Fix potential panic when parsing Content-Range (index out of range)
when accessing `tokens[0]`.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix(security): remove InsecureSkipVerify from metrics client (TLS-1)
Replace the unconditional InsecureSkipVerify: true TLS config in
newHTTPMetricsClient with the system cert pool (+ TLS 1.2 minimum).
Add an optional CACert field to MetricsConfig and to the exporter
ServerConfig so operators running zot with a self-signed or private
CA can point the exporter at the correct CA file instead of
disabling certificate verification entirely.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* feat(metrics): add HTTPS configuration for metrics exporter
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(security): enhance CA certificate handling in metrics client and add tests
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(security): improve CA certificate error handling in metrics client and update tests
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(tests): correct package name in minimal_client_test.go and simplify error declaration
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(tests): update package name in minimal_client_test.go for consistency
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
fix(security): suppress Allow-Credentials on wildcard CORS origin (CORS-1)
Per CORS spec §3.2, Access-Control-Allow-Credentials must not be
"true" when Access-Control-Allow-Origin is the wildcard "*".
ACHeadersMiddleware (pkg/common/http_server.go) and
getUIHeadersHandler (pkg/api/routes.go) now only emit the
credentials header when an explicit, non-empty AllowOrigin is
configured. Deployments that leave AllowOrigin blank (default
wildcard) no longer produce a contradictory header pair.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Wrap req.Body with http.MaxBytesReader before io.ReadAll in
CreateAPIKey. Requests with bodies larger than MaxAPIKeyBodySize
(4 KiB) now return HTTP 413 instead of buffering arbitrary data.
Add the MaxAPIKeyBodySize constant, update the Swagger @Failure
annotation to document 413, and add a unit test.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
Wrap request.Body with http.MaxBytesReader before io.ReadAll in
UpdateManifest. Bodies exceeding MaxManifestBodySize (4 MiB) now
return HTTP 413 with a MANIFEST_INVALID error body instead of
buffering unlimited data into memory.
Add the MaxManifestBodySize constant and a unit test that sends an
oversized body and asserts the 413 status.
Agent-Logs-Url: https://github.com/project-zot/zot/sessions/5eca86eb-9749-4cf8-9fb8-7b9ace2ba87f
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(auth): add workaround for Docker client auth with mixed anonymous policies
Docker client fails to authenticate to protected repositories when basic auth
(htpasswd/LDAP) is used with mixed access policies (some repos anonymous,
some requiring auth). This happens because Docker determines whether to send
credentials based on the /v2/ response - if it returns 200, Docker assumes
no auth is needed anywhere.
Add `forceDockerClientAuth` config option that, when enabled, forces 401 on
/v2/ for Docker clients, triggering Docker's authentication flow.
This workaround only affects Docker clients (detected via User-Agent).
Podman and other OCI-compliant clients are unaffected.
Refs: https://github.com/opencontainers/wg-auth/blob/main/docs/implementations/moby.md
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat: remove ForceDockerClientAuth flag and use only authz policies to determine the docker specific behavior
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Adds a configurable maximum repository count per registry instance.
When maxRepos is set on StorageConfig, manifest pushes that would create
a new repository beyond the limit are rejected with HTTP 429
TOOMANYREQUESTS. Pushes to existing repositories are always allowed.
Implemented as an always-available feature in pkg/api (not a build-tag
extension). MaxRepos is a field on StorageConfig, enabled when > 0.
- repoQuotaMiddleware on the dist-spec router intercepts manifest PUTs.
New-repo pushes are serialized with a sync.Mutex to prevent concurrent
requests from exceeding the limit.
- Adds CountRepos(ctx) to the MetaDB interface with efficient
implementations: BoltDB (Stats().KeyN), Redis (HLen), DynamoDB
(Scan with Select=COUNT).
- Config.IsQuotaEnabled() added, wired into controller.go metaDB init.
- Four integration tests (enforcement, concurrency, disabled,
unconfigured) and backend-specific CountRepos tests for BoltDB, Redis,
and DynamoDB.
Signed-off-by: Bachir Khiati <bachir.khiati@gmail.com>
* ci: Reduce chance of installing corrupt packages
See: https://dev.to/hsbt/should-rubygemsbundler-have-a-cooldown-feature-40cp
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent credential leakage from checkout steps
Add `persist-credentials: false` to all `actions/checkout` calls across
22 workflow files. Without this, the GitHub token used for checkout is
written into `.git/config` and remains accessible to all subsequent steps
and any uploaded artifacts (artipacked finding).
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent template injection from github context in run steps
`${{ github.* }}` expressions used directly inside `run:` blocks are
expanded before the shell sees them. A crafted value (e.g. a tag name
containing shell metacharacters) would execute arbitrary code.
Move the values into `env:` variables (e.g. GITHUB_EVENT_RELEASE_TAG_NAME,
GITHUB_ACTOR) and reference them as `${VAR}` in the shell, so the runtime
never interprets them as code (template-injection finding).
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent injection in yq commands via release tag name
`${{ github.event.release.tag_name }}` was interpolated directly into
yq `cmd:` inputs. A crafted tag name could inject shell commands since
the expression is expanded before the action runs. Use yq's `strenv()`
with an `env:` variable instead so the value is always treated as data.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: pin all GitHub Actions to full commit SHAs
Actions pinned to mutable tags (e.g. @v6, @main) can change under us if
the upstream repo is compromised or tags are moved, enabling supply-chain
attacks. Pinning to the full 40-char commit SHA locks the exact code that
runs. Version tags are preserved as inline comments (e.g. # v6.0.2) for
readability and Dependabot compatibility.
Used `pinact` for standard tagged versions; remaining branch-based
references (mikefarah/yq, jlumbroso/free-disk-space,
project-stacker/stacker-build-push-action, aquasecurity/trivy-action)
resolved manually via the GitHub API.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: slow down GitHub Actions dependency updates to biweekly
Dependabot has no native biweekly interval. Combining weekly checks with
a 14-day cooldown achieves the same effect: Dependabot scans every Monday
but won't open a PR for a new action version until 14 days after release,
giving the ecosystem time to stabilize before we adopt it.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: group all Dependabot updates into single PRs per ecosystem
Without groups, Dependabot opens one PR per dependency. With `patterns: "*"`,
all Go module bumps land in one PR and all GitHub Actions pin updates in
another, reducing review noise.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: restore credential persistence for helm-charts push
`persist-credentials: false` was too broad — the helm-charts checkout
uses HELM_PUSH_TOKEN specifically so the subsequent `git push` can
authenticate. Only the main repo checkout should have credentials disabled.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: use --password-stdin for oras login
Passing the token via `-p` exposes it in process listings and debug
logs. Piping via stdin is the standard secure pattern for CLI auth.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: pin actions to versioned release SHAs
jmgilman/actions-generate-checksum: v1 branch HEAD -> v1.0.1 release
mikefarah/yq: arbitrary master HEAD -> v4.52.5 release
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
---------
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
And default it to ["auto"] when unset, with an info log from applyDefaultValues.
Refactor CVE NewScanner to take *CVEConfig instead of separate DB repository
strings so the full Trivy block is available to the scanner.
Extend CLI and search tests for the new field and logged config; document
CVE/Trivy in examples/README and add examples/config-cve-trivy.json.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Wait for zot PIDs (and the port) to fully shut down before restarting, and retry transient curl failures (e.g. exit 52) in wait_zot_reachable.
Send checksum verification errors to stderr for clearer failure output.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix(storage): resolve double-prefixing issue for GCS rootdirectory
Preserve double-prefixing for S3 to maintain backward compatibility with existing data. For GCS, always use "/" as rootDir to avoid double-prefixing, as GCS rootdirectory usage is a newer feature without legacy data.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(gcs): handle io.EOF correctly in Walk method
Ensure io.EOF is returned unwrapped to allow proper error handling with errors.Is() upstream.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(storage): set sensible default ("/zot") for GCS when storageDriver.rootdirectory is unset or empty or "/"
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(imagestore): avoid warning logs for expected cache miss scenarios
Refine logging to use debug level for expected cache misses, preventing unnecessary warnings.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
---------
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* ci: use zot localstack image, and consolidate on using the setup localstack GH action
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: increase zot minimal test timeout
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: setup latest localstack still available for community
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat(zb): list tests and test regex filter + misc
This change introduces the following changes to zb.
Test Filtering
===============
Allows users to selectively run tests by specifying
a standard regex that matches on the name of the test.
Test Listing
===============
Allows users to list out the available tests as well as
the matched tests when using the regex filter.
The documentation README has also been updated with
examples and the command help.
The documentation for skip cleanup has been updated.
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com>
ImageIndex2ImageSummary was missing LastPullTimestamp assignment, causing
multi-arch image queries to always return null for this field. Also adds
the PushedBy field (already stored in MetaDB) to the GraphQL schema and
both conversion paths (manifest and index).
Signed-off-by: cainydev <wajo432@gmail.com>
Validate callback_ui and default invalid values to /.
Allow absolute callback_ui only when its origin is allowlisted via http.auth.openid.callbackAllowOrigins (and externalUrl).
Add/adjust unit + controller tests and update examples/docs for relative vs allowlisted absolute redirect
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat(sync): add SyncLegacyCosignTags config to skip syncing legacy cosign/SBOM tags when disabled
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: sync on demand with referrers API should not use recursion to sync referrers of referrers
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: add tests SyncLegacyCosignTags and changes in /referrers on demand sync
Credit for the tests goes to @jzhn see:
https://github.com/project-zot/zot/pull/3840/changes
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: remove redundant syncRef logic which synced referrers both with the zot inner() implementation and with regctl native implementation
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* Fix#3823: skip OCI conversion when image is already synced
When syncRef determines an image is already synced, it now returns a
bool to signal the skip. syncImage checks this and returns early before
attempting OCI conversion, preventing misleading 'failed to convert
docker image to oci' errors caused by a non-existent temp directory.
* Keep syncReferrers and CommitAll running for already-synced images
Address review feedback: new referrers can be added upstream after
initial sync, so we must not skip syncReferrers. Only the OCI
conversion is guarded by the skipped flag, since converting an
already-stored image is both unnecessary and incorrect.
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
---------
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
Andrei Aaron