This commit modernizes code across multiple packages by:
- Using Go 1.18+ features (slices.IndexFunc, strings.Cut)
- Pre-allocating slices and maps with known capacity
- Consolidating defensive checks and improving code clarity
- Fixing test data and build tag issues
CLI client improvements:
- Pre-allocate slices in search functions and service methods
- Replace strings.Split with strings.Cut for username:password parsing
- Use range-based iteration instead of manual index loops
Search extension optimizations:
- Cache sort functions in pagination modules
- Pre-allocate page buffers and maps
- Consolidate defensive checks in filterBaseImages/filterDerivedImages
- Fix image bas and derived logic allowing out of sequence layers for base images
- Fix image pagination reporting images groupped by repos when sorted by update time
- Remove duplicate resolver_test.go file
Monitoring extension:
- Replace manual loops with slices.IndexFunc
- Pre-allocate bucketsFloat2String map
Sync extension:
- Pre-allocate slice in parseRegistryURLs
Test utilities:
- Fix build tags in oci_layout.go
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
- Return CommitAll errors instead of ignoring them
- Skip ErrRepoNotFound from temp sync dirs to allow other tags to sync
- Each tag uses separate temp directory, so failures are isolated
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat: Sync images with a background context
This means syncs/pulls will not be cancelled anymore when the requesting client disconnects.
The timeout used can be configured per registry
Signed-off-by: Lars Francke <git@lars-francke.de>
GC and scrub should not stop if a manifest or index is missing from storage.
Other similar changes are also included.
WRT metadb, the missing manifests cannot be added, and the results returned from metadb
do not include the descriptors for these manifests.
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
feat: add support for sha256 and sha512 htpasswd formats
Fixes issue #3495
We currently support only bcrypt htpasswd hashes, however bcrypt is not
FIPS-140 approved since it uses Blowfish.
This PR adds support for sha256 and sha512 formats and enforces that
bcrypt be disabled when fips140 mode is enabled.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: make config read/write thread safe and fix some other similar issues
1. The config config has a lock, and safe methods to update and read the attributes
2. The config has methods to retrieve copies of specific attributes, such as the extyensions config, the auth config, and the authz config.
These are needed, as the config object may mutate in the middle of an auth/authz requests, and we avoid partial configuration being applied for that request.
3. Fix an issue with the monitoring server not stopping when the controller is shut down.
4. Fix an issue with the HTPasswdWatcher not stopping when the background tasks are supposed to finish.
5. Fix some tests using hardcoded ports.
Moved some of the methods which were on the main config to the auth, access control and extension configs
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* fix: migrate to Go module v2 for proper semantic versioning
This change updates the module path from 'zotregistry.dev/zot' to
'zotregistry.dev/zot/v2' to comply with Go's semantic versioning rules.
According to Go's module versioning requirements, major version v2+
must include the major version in the module path. The current
module path 'zotregistry.dev/zot' only supports v0.x.x and v1.x.x
versions, making existing v2.x.x tags (like v2.1.8) unusable.
Changes:
- Updated go.mod module path to zotregistry.dev/zot/v2
- Updated all internal import paths across 280+ Go source files
- Updated configuration files (golangcilint.yaml, gqlgen.yml)
- Updated README.md Go reference badge
This fix enables proper use of existing v2.x.x Git tags and allows
external packages to import zot v2+ versions without compatibility
errors.
Resolves: Go module import compatibility for v2+ versions
Fixes: #3071
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
* fix: regenerate GraphQL files with updated v2 import paths
The gqlgen tool needs to regenerate the GraphQL schema files after
the module path change to use the new v2 imports.
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
---------
Signed-off-by: Luca Muscariello <muscariello@ieee.org>
* fix: migrate from github.com/rs/zerolog to golang-native log/slog
We have been using zerolog for a really long time.
golang now has structured logging using slog.
Best to move to this in interests of long-term support.
This is a tech debt item.
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: a few changes on top
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* fix: address comments
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha.dev@gmail.com>
* chore: increase/stabilize coverage for the local storage driver
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: add/stabilize coverage for soring ImageSummary objects
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* chore: stabilize coverage in sync tests
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
---------
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
* feat: rework token auth to allow ED25519/EC public keys
Signed-off-by: evanebb <git@evanus.nl>
* fix: shadow err variable to hopefully avoid data race
Signed-off-by: evanebb <git@evanus.nl>
* fix: apply golangci-lint feedback
Signed-off-by: evanebb <git@evanus.nl>
* fix: simplify public key loading by only supporting certificates, fixes ED25519 certificate handling
Signed-off-by: evanebb <git@evanus.nl>
* test: add golang-jwt based test auth server and test RSA/EC/ED25519 keys
Signed-off-by: evanebb <git@evanus.nl>
* fix: restrict allowed signing algorithms as recommended by library
Signed-off-by: evanebb <git@evanus.nl>
* test: add more bearer authorizer tests
Signed-off-by: evanebb <git@evanus.nl>
* fix: apply more golangci-lint feedback
Signed-off-by: evanebb <git@evanus.nl>
* test: ensure chmod calls run on test failure for authn errors test
Signed-off-by: evanebb <git@evanus.nl>
* fix: verify issued-at in given token if present
Pulls the validation in-line with the old library
Signed-off-by: evanebb <git@evanus.nl>
---------
Signed-off-by: evanebb <git@evanus.nl>
* feat: add support for docker images
Issue #724
A new config section under "HTTP" called "Compat" is added which
currently takes a list of possible compatible legacy media-types.
https://github.com/opencontainers/image-spec/blob/main/media-types.md#compatibility-matrix
Only "docker2s2" (Docker Manifest V2 Schema V2) is currently supported.
Garbage collection also needs to be made aware of non-OCI compatible
layer types.
feat: add cve support for non-OCI compatible layer types
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
*
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* test: add more docker compat tests
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* feat: add additional validation checks for non-OCI images
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* ci: make "full" images docker-compatible
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
This commit includes support for periodic repo sync in a scale-out
cluster.
Before this commit, all cluster members would sync all the repos as
the config is shared.
With this change, in periodic sync, the cluster member checks whether
it manages the repo. If it does not manage the repo, it will skip the
sync.
This commit also includes a unit test to test on-demand sync too, but
there are no logic changes for it as it is implicitly handled by the
proxying logic.
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
refactor(http): refactor http client to take options struct
This commit updates the arguments for the `CreateHTTPClient`
function to consume a struct which can be extended as required.
It replaces the certPath argument with a struct of 3 paths for
client ertificate, client key, and ca cert. It also adds
a TLSEnabled option for when an HTTP Client is required
without any further TLS config.
Existing consumers of this function have been updated so that
they can work as they do today. This change is a no-op for
existing features.
This allows for certificate paths to be customised and
allows other modules to re-use the same HTTP client and get
the benefits of mTLS support and per-host certificates.
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
* fix(oras)!: remove ORAS artifact references support
ORAS artifacts/references predated OCI dist-spec 1.1.0 which now has the
same functionality and likely to see wider adoption.
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* test: update to released official images
So that they are unlikely to be deleted.
*-rc images may be cleaned up over time.
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
BREAKING CHANGE: the dist spec version in the config files needs to be bumped to 1.1.0
in order for the config verification to pass without warnings.
Also fix 1 dependabot alert for helm.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
This causes the "fair" scheduler to run it too often in the detriment of other generators.
The intention was to run it every 2 hours but the measurement unit for 7200 was not specified.
Add more logs, including showing a generator name, in order to troubleshoot this kind of issues easier in the future.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* fix(scheduler): data race when pushing new tasks
the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()
because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers
solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* refactor(scheduler): refactor into a single shutdown
before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().
simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
---------
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* feat(sync): local tmp store
Signed-off-by: a <a@tuxpa.in>
* fix(sync): various fixes for s3+remote storage feature
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
---------
Signed-off-by: a <a@tuxpa.in>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: a <a@tuxpa.in>
Andrei Aaron