Wrap request.Body with http.MaxBytesReader before io.ReadAll in
UpdateManifest. Bodies exceeding MaxManifestBodySize (4 MiB) now
return HTTP 413 with a MANIFEST_INVALID error body instead of
buffering unlimited data into memory.
Add the MaxManifestBodySize constant and a unit test that sends an
oversized body and asserts the 413 status.
Agent-Logs-Url: https://github.com/project-zot/zot/sessions/5eca86eb-9749-4cf8-9fb8-7b9ace2ba87f
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix(auth): add workaround for Docker client auth with mixed anonymous policies
Docker client fails to authenticate to protected repositories when basic auth
(htpasswd/LDAP) is used with mixed access policies (some repos anonymous,
some requiring auth). This happens because Docker determines whether to send
credentials based on the /v2/ response - if it returns 200, Docker assumes
no auth is needed anywhere.
Add `forceDockerClientAuth` config option that, when enabled, forces 401 on
/v2/ for Docker clients, triggering Docker's authentication flow.
This workaround only affects Docker clients (detected via User-Agent).
Podman and other OCI-compliant clients are unaffected.
Refs: https://github.com/opencontainers/wg-auth/blob/main/docs/implementations/moby.md
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat: remove ForceDockerClientAuth flag and use only authz policies to determine the docker specific behavior
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Adds a configurable maximum repository count per registry instance.
When maxRepos is set on StorageConfig, manifest pushes that would create
a new repository beyond the limit are rejected with HTTP 429
TOOMANYREQUESTS. Pushes to existing repositories are always allowed.
Implemented as an always-available feature in pkg/api (not a build-tag
extension). MaxRepos is a field on StorageConfig, enabled when > 0.
- repoQuotaMiddleware on the dist-spec router intercepts manifest PUTs.
New-repo pushes are serialized with a sync.Mutex to prevent concurrent
requests from exceeding the limit.
- Adds CountRepos(ctx) to the MetaDB interface with efficient
implementations: BoltDB (Stats().KeyN), Redis (HLen), DynamoDB
(Scan with Select=COUNT).
- Config.IsQuotaEnabled() added, wired into controller.go metaDB init.
- Four integration tests (enforcement, concurrency, disabled,
unconfigured) and backend-specific CountRepos tests for BoltDB, Redis,
and DynamoDB.
Signed-off-by: Bachir Khiati <bachir.khiati@gmail.com>
* ci: Reduce chance of installing corrupt packages
See: https://dev.to/hsbt/should-rubygemsbundler-have-a-cooldown-feature-40cp
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent credential leakage from checkout steps
Add `persist-credentials: false` to all `actions/checkout` calls across
22 workflow files. Without this, the GitHub token used for checkout is
written into `.git/config` and remains accessible to all subsequent steps
and any uploaded artifacts (artipacked finding).
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent template injection from github context in run steps
`${{ github.* }}` expressions used directly inside `run:` blocks are
expanded before the shell sees them. A crafted value (e.g. a tag name
containing shell metacharacters) would execute arbitrary code.
Move the values into `env:` variables (e.g. GITHUB_EVENT_RELEASE_TAG_NAME,
GITHUB_ACTOR) and reference them as `${VAR}` in the shell, so the runtime
never interprets them as code (template-injection finding).
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent injection in yq commands via release tag name
`${{ github.event.release.tag_name }}` was interpolated directly into
yq `cmd:` inputs. A crafted tag name could inject shell commands since
the expression is expanded before the action runs. Use yq's `strenv()`
with an `env:` variable instead so the value is always treated as data.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: pin all GitHub Actions to full commit SHAs
Actions pinned to mutable tags (e.g. @v6, @main) can change under us if
the upstream repo is compromised or tags are moved, enabling supply-chain
attacks. Pinning to the full 40-char commit SHA locks the exact code that
runs. Version tags are preserved as inline comments (e.g. # v6.0.2) for
readability and Dependabot compatibility.
Used `pinact` for standard tagged versions; remaining branch-based
references (mikefarah/yq, jlumbroso/free-disk-space,
project-stacker/stacker-build-push-action, aquasecurity/trivy-action)
resolved manually via the GitHub API.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: slow down GitHub Actions dependency updates to biweekly
Dependabot has no native biweekly interval. Combining weekly checks with
a 14-day cooldown achieves the same effect: Dependabot scans every Monday
but won't open a PR for a new action version until 14 days after release,
giving the ecosystem time to stabilize before we adopt it.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: group all Dependabot updates into single PRs per ecosystem
Without groups, Dependabot opens one PR per dependency. With `patterns: "*"`,
all Go module bumps land in one PR and all GitHub Actions pin updates in
another, reducing review noise.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: restore credential persistence for helm-charts push
`persist-credentials: false` was too broad — the helm-charts checkout
uses HELM_PUSH_TOKEN specifically so the subsequent `git push` can
authenticate. Only the main repo checkout should have credentials disabled.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: use --password-stdin for oras login
Passing the token via `-p` exposes it in process listings and debug
logs. Piping via stdin is the standard secure pattern for CLI auth.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: pin actions to versioned release SHAs
jmgilman/actions-generate-checksum: v1 branch HEAD -> v1.0.1 release
mikefarah/yq: arbitrary master HEAD -> v4.52.5 release
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
---------
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
And default it to ["auto"] when unset, with an info log from applyDefaultValues.
Refactor CVE NewScanner to take *CVEConfig instead of separate DB repository
strings so the full Trivy block is available to the scanner.
Extend CLI and search tests for the new field and logged config; document
CVE/Trivy in examples/README and add examples/config-cve-trivy.json.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Wait for zot PIDs (and the port) to fully shut down before restarting, and retry transient curl failures (e.g. exit 52) in wait_zot_reachable.
Send checksum verification errors to stderr for clearer failure output.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix(storage): resolve double-prefixing issue for GCS rootdirectory
Preserve double-prefixing for S3 to maintain backward compatibility with existing data. For GCS, always use "/" as rootDir to avoid double-prefixing, as GCS rootdirectory usage is a newer feature without legacy data.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(gcs): handle io.EOF correctly in Walk method
Ensure io.EOF is returned unwrapped to allow proper error handling with errors.Is() upstream.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(storage): set sensible default ("/zot") for GCS when storageDriver.rootdirectory is unset or empty or "/"
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* fix(imagestore): avoid warning logs for expected cache miss scenarios
Refine logging to use debug level for expected cache misses, preventing unnecessary warnings.
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
---------
Signed-off-by: Sebastian Thees <thees@users.noreply.github.com>
* ci: use zot localstack image, and consolidate on using the setup localstack GH action
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: increase zot minimal test timeout
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: setup latest localstack still available for community
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat(zb): list tests and test regex filter + misc
This change introduces the following changes to zb.
Test Filtering
===============
Allows users to selectively run tests by specifying
a standard regex that matches on the name of the test.
Test Listing
===============
Allows users to list out the available tests as well as
the matched tests when using the regex filter.
The documentation README has also been updated with
examples and the command help.
The documentation for skip cleanup has been updated.
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com>
ImageIndex2ImageSummary was missing LastPullTimestamp assignment, causing
multi-arch image queries to always return null for this field. Also adds
the PushedBy field (already stored in MetaDB) to the GraphQL schema and
both conversion paths (manifest and index).
Signed-off-by: cainydev <wajo432@gmail.com>
Validate callback_ui and default invalid values to /.
Allow absolute callback_ui only when its origin is allowlisted via http.auth.openid.callbackAllowOrigins (and externalUrl).
Add/adjust unit + controller tests and update examples/docs for relative vs allowlisted absolute redirect
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat(sync): add SyncLegacyCosignTags config to skip syncing legacy cosign/SBOM tags when disabled
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: sync on demand with referrers API should not use recursion to sync referrers of referrers
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: add tests SyncLegacyCosignTags and changes in /referrers on demand sync
Credit for the tests goes to @jzhn see:
https://github.com/project-zot/zot/pull/3840/changes
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: remove redundant syncRef logic which synced referrers both with the zot inner() implementation and with regctl native implementation
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* Fix#3823: skip OCI conversion when image is already synced
When syncRef determines an image is already synced, it now returns a
bool to signal the skip. syncImage checks this and returns early before
attempting OCI conversion, preventing misleading 'failed to convert
docker image to oci' errors caused by a non-existent temp directory.
* Keep syncReferrers and CommitAll running for already-synced images
Address review feedback: new referrers can be added upstream after
initial sync, so we must not skip syncReferrers. Only the OCI
conversion is guarded by the skipped flag, since converting an
already-stored image is both unnecessary and incorrect.
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
---------
Signed-off-by: Ugur Tafrali <ugur.tafrali@gmail.com>
feat(storage): add a GCS driver
test(storage): add unit tests for GCS driver
test(storage): add missing unit tests for GCS driver & resolve lint issues
fix: configuration validation for GCS Storage
test(storage): resolve panic by test due to setupGCS ignoring returned error
test(storage): add dummy gcs credentials
test: add darwin support for macos to run tests
ci: update workflows to pin gcs emulator version
lint: resolve long line lengths & formatting issues
test: move error for gcs mock earlier with an error
test: stop test using local google credentials and use mock instead
test: add missing dummy creds
test(storage): use storage-testbench for GCS, isolate GCS tests, fix driver Delete
- Switch GCS emulator from fake-gcs-server to storage-testbench in CI.
Run the GCS emulator only in the privileged-test job; remove it from
minimal and extended test jobs.
- Consolidate GCS tests under pkg/storage/gcs (needprivileges,linux).
Add TestMain with HTTPS proxy and /etc/hosts so tests talk to
storage-testbench; move GCS-specific cases from storage_test.go and
scrub_test.go into gcs_test.go. Run GCS tests via a second privileged-test
invocation and collect coverage in coverage-needprivileges-gcs.txt.
- Make GCS driver Delete idempotent and normalize errors. Treat
PathNotFoundError from Delete as success so that deleting an already-gone
path (e.g. after GC under eventual consistency) does not fail. Add
formatErr to map 404/not found to PathNotFoundError and use it for all
driver methods so callers get consistent storage driver errors.
- Drop GCS branches and helpers from storage_test.go and scrub_test.go so
non-privileged tests only use local/S3; GCS is tested only in
pkg/storage/gcs with storage-testbench.
- Set GCSMOCK_ENDPOINT without /storage/v1/, as the rest of the URL is set in tests.
- Show errors in case of failure to create bucket.
- Consolidate StorageDriverMock structs inside the pkg/test/mocks package.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Co-authored-by: Steven Marks <steve.marks@qomodo.io>
1. Parse repos without metadata in ParseStorage
The timestamp check in ParseStorage was skipping repos that exist in
storage but don't have metadata. When GetRepoLastUpdated returns zero
time (no metadata), we should always parse the repo to create its
metadata. Check if metaLastUpdated is zero before comparing timestamps.
If zero, always parse regardless of storageLastUpdated.
2. Change the logic of how LastUpdated is computed in RepoSummary
It is not the latest tagged timestamp from the available images or
the last updated image created timestamp, based on whichever is the
latest.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Ramkumar Chinchani