* ci: migrate workflows off deprecated oracle-vm runners
Switch CI jobs from oracle-vm-* runner labels to cncf-ubuntu-* as the oracle-cm-* CNCF runner labels are deprecated.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* ci: increase resources available for ecosystem tools tests
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* ci: Reduce chance of installing corrupt packages
See: https://dev.to/hsbt/should-rubygemsbundler-have-a-cooldown-feature-40cp
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent credential leakage from checkout steps
Add `persist-credentials: false` to all `actions/checkout` calls across
22 workflow files. Without this, the GitHub token used for checkout is
written into `.git/config` and remains accessible to all subsequent steps
and any uploaded artifacts (artipacked finding).
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent template injection from github context in run steps
`${{ github.* }}` expressions used directly inside `run:` blocks are
expanded before the shell sees them. A crafted value (e.g. a tag name
containing shell metacharacters) would execute arbitrary code.
Move the values into `env:` variables (e.g. GITHUB_EVENT_RELEASE_TAG_NAME,
GITHUB_ACTOR) and reference them as `${VAR}` in the shell, so the runtime
never interprets them as code (template-injection finding).
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: prevent injection in yq commands via release tag name
`${{ github.event.release.tag_name }}` was interpolated directly into
yq `cmd:` inputs. A crafted tag name could inject shell commands since
the expression is expanded before the action runs. Use yq's `strenv()`
with an `env:` variable instead so the value is always treated as data.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: pin all GitHub Actions to full commit SHAs
Actions pinned to mutable tags (e.g. @v6, @main) can change under us if
the upstream repo is compromised or tags are moved, enabling supply-chain
attacks. Pinning to the full 40-char commit SHA locks the exact code that
runs. Version tags are preserved as inline comments (e.g. # v6.0.2) for
readability and Dependabot compatibility.
Used `pinact` for standard tagged versions; remaining branch-based
references (mikefarah/yq, jlumbroso/free-disk-space,
project-stacker/stacker-build-push-action, aquasecurity/trivy-action)
resolved manually via the GitHub API.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: slow down GitHub Actions dependency updates to biweekly
Dependabot has no native biweekly interval. Combining weekly checks with
a 14-day cooldown achieves the same effect: Dependabot scans every Monday
but won't open a PR for a new action version until 14 days after release,
giving the ecosystem time to stabilize before we adopt it.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: group all Dependabot updates into single PRs per ecosystem
Without groups, Dependabot opens one PR per dependency. With `patterns: "*"`,
all Go module bumps land in one PR and all GitHub Actions pin updates in
another, reducing review noise.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: restore credential persistence for helm-charts push
`persist-credentials: false` was too broad — the helm-charts checkout
uses HELM_PUSH_TOKEN specifically so the subsequent `git push` can
authenticate. Only the main repo checkout should have credentials disabled.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: use --password-stdin for oras login
Passing the token via `-p` exposes it in process listings and debug
logs. Piping via stdin is the standard secure pattern for CLI auth.
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: pin actions to versioned release SHAs
jmgilman/actions-generate-checksum: v1 branch HEAD -> v1.0.1 release
mikefarah/yq: arbitrary master HEAD -> v4.52.5 release
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
---------
Signed-off-by: Benoit Tigeot <benoit.tigeot@lifen.fr>
* ci: use zot localstack image, and consolidate on using the setup localstack GH action
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: increase zot minimal test timeout
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: setup latest localstack still available for community
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
- fix log folder names used by nightly jobs
- fix attempt to install containerd.io in the redis pipeline (which conflicts with containerd)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(cluster): initial commit for scale-out cluster
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* feat(cluster): support shared storage scale out
This change introduces support for shared storage backed
zot cluster scale out.
New feature
Multiple stateless zot instances can run using the same shared
storage backend where each instance looks at a specific set
of repositories based on a siphash of the repository name to improve
scale as the load is distributed across multiple instances.
For a given config, there will only be one instance that can perform
dist-spec read/write on a given repository.
What's changed?
- introduced a transparent request proxy for dist-spec endpoints based on
siphash of repository name.
- new config for scale out cluster that specifies list of
cluster members.
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
There are performance improvements in recent releases of localstack.
1) install localstack via "pip install" and requires python 3.11
2) also pull a recently pushed localstack docker image to ghcr.io
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
now gc stress on s3 storage is using minio for ci/cd builds
gc stress on s3 storage is using localstack for nightly builds
fixed(gc): make sure we don't remove repo if there are blobs
being uploaded or the number of blobs gc'ed is not 0
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
New examples of running tests:
1. To run a specific bats file (with and without verbose output):
make run-blackbox-tests BATS_TEST_FILE_PATH=test/blackbox/delete_images.bats
make run-blackbox-tests BATS_TEST_FILE_PATH=test/blackbox/delete_images.bats BATS_VERBOSITY=2
2. To run the CI tests (with and without verbose output)
make run-blackbox-ci
make run-blackbox-ci BATS_VERBOSITY=2
BATS_TEST_FILE_PATH is used to pass on the test file to run using `run-blackbox-tests`
BATS_VERBOSITY controls the verbosity of the bats framework output, if unspecified the output only
contains test results and failure message in case of failures.
If BATS_VERBOSITY is 1, then also show commands as they are executed.
If BATS_VERBOSITY is 2, on top of the above it also shows output of passed tests.
Other changes in this PR:
- Update some of the tests to show logs after the run ends.
- Run the linters before the tests, as it saves time on failures when running in GH
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
unified both local and s3 ImageStore logic into a single ImageStore
added a new driver interface for common file/dirs manipulations
to be implemented by different storage types
refactor(gc): drop umoci dependency, implemented internal gc
added retentionDelay config option that specifies
the garbage collect delay for images without tags
this will also clean manifests which are part of an index image
(multiarch) that no longer exist.
fix(dedupe): skip blobs under .sync/ directory
if startup dedupe is running while also syncing is running
ignore blobs under sync's temporary storage
fix(storage): do not allow image indexes modifications
when deleting a manifest verify that it is not part of a multiarch image
and throw a MethodNotAllowed error to the client if it is.
we don't want to modify multiarch images
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Ramkumar Chinchani