We use chartmuseum lib for handling bearer requests, which is not
implementing the token spec, mainly it expects "scope" parameter
to be given on every request, even for /v2/ route which doesn't represent
a resource.
Handle this /v2/ route inside our code.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
- update cve tests
- update scrub tests
- update tests for parsing storage and loading into meta DB
- update controller tests
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
- added a new field 'IsDeletable' for graphql ImageSummary struct.
- apply cors on DeleteManifest route
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* fix(scheduler): data race when pushing new tasks
the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()
because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers
solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* refactor(scheduler): refactor into a single shutdown
before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().
simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
---------
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
wait for workers to finish before exiting
should fix tests reporting they couldn't remove rootDir because it's being
written by tasks
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
- MetaDB stores the time of the last update of a repo
- During startup we check if the layout has been updated after the last recorded change in the db
- If this is the case, the repo is parsed and updated in the DB otherwise it's skipped
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
ci(notation): update to latest notation version
fix(sync): add layers info when syncing signatures
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
Which could be imported independently. See more details:
1. "zotregistry.io/zot/pkg/test/common" - currently used as
tcommon "zotregistry.io/zot/pkg/test/common" - inside pkg/test
test "zotregistry.io/zot/pkg/test/common" - in tests
. "zotregistry.io/zot/pkg/test/common" - in tests
Decouple zb from code in test/pkg in order to keep the size small.
2. "zotregistry.io/zot/pkg/test/image-utils" - curently used as
. "zotregistry.io/zot/pkg/test/image-utils"
3. "zotregistry.io/zot/pkg/test/deprecated" - curently used as
"zotregistry.io/zot/pkg/test/deprecated"
This one will bre replaced gradually by image-utils in the future.
4. "zotregistry.io/zot/pkg/test/signature" - (cosign + notation) use as
"zotregistry.io/zot/pkg/test/signature"
5. "zotregistry.io/zot/pkg/test/auth" - (bearer + oidc) curently used as
authutils "zotregistry.io/zot/pkg/test/auth"
6. "zotregistry.io/zot/pkg/test/oci-utils" - curently used as
ociutils "zotregistry.io/zot/pkg/test/oci-utils"
Some unused functions were removed, some were replaced, and in
a few cases specific funtions were moved to the files they were used in.
Added an interface for the StoreController, this reduces the number of imports
of the entire image store, decreasing binary size for tests.
If the zb code was still coupled with pkg/test, this would have reflected in zb size.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
- implement scrub also for S3 storage by replacing umoci
- change scrub implementation for ImageIndex
- take the `Subject` into consideration when running scrub
- remove test code relying on the umoci library. Since we started
relying on images in test/data, and we create our own images using
go code we can obtain digests by other means. (cherry picked from commit 489d4e2d23c1b4e48799283f8281024bbef6123f)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
fix(gc): fix cleaning deduped blobs because they have the modTime of
the original blobs, fixed by updating the modTime when hard linking
the blobs.
fix(gc): failing to parse rootDir at zot startup when using s3 storage
because there are no files under rootDir and we can not create empty dirs
on s3, fixed by creating an empty file under rootDir.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Also fix additional issues:
- sorting of tags on calls without pagination parameters ('n' or 'last')
- if 'n' is 0 we should return an empty list and not error
Added tests accordingly
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
fix(authz): fix isAdmin not using groups to determine if a user is admin.
fix(authz): return 401 instead of 403
403 is correct as per HTTP spec
However authz is not part of dist-spec and clients know only about 401
So this is a compromise.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
unified both local and s3 ImageStore logic into a single ImageStore
added a new driver interface for common file/dirs manipulations
to be implemented by different storage types
refactor(gc): drop umoci dependency, implemented internal gc
added retentionDelay config option that specifies
the garbage collect delay for images without tags
this will also clean manifests which are part of an index image
(multiarch) that no longer exist.
fix(dedupe): skip blobs under .sync/ directory
if startup dedupe is running while also syncing is running
ignore blobs under sync's temporary storage
fix(storage): do not allow image indexes modifications
when deleting a manifest verify that it is not part of a multiarch image
and throw a MethodNotAllowed error to the client if it is.
we don't want to modify multiarch images
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
- image command is now deprecated in favor of 'images'
- cve command is now deprecated in favor of 'cves'
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
added a new config option under 'http' called externalURL which is used
by openid/oauth2 clients to redirect back to zot
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* fix: remove inline GC and set a default value of gc interval
- remove inline GC
- add a default value of GC interval
- run the GC periodically by default with the default value if no interval provided
- generate GC tasks with a random delay(0-30s) between
- add IsReady() method to scheduler.TaskGenerator interface
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
* ci: add test for gc with short interval
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
---------
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
BREAKING CHANGE: The functionality provided by the mgmt endpoint has beed redesigned - see details below
BREAKING CHANGE: The API keys endpoint has been moved - see details below
BREAKING CHANGE: The mgmt extension config has been removed - endpoint is now enabled by having both the search and the ui extensions enabled
BREAKING CHANGE: The API keys configuration has been moved from extensions to http>auth>apikey
mgmt and imagetrust extensions:
- separate the _zot/ext/mgmt into 3 separate endpoints: _zot/ext/auth, _zot/ext/notation, _zot/ext/cosign
- signature verification logic is in a separate `imagetrust` extension
- better hanling or errors in case of signature uploads: logging and error codes (more 400 and less 500 errors)
- add authz on signature uploads (and add a new middleware in common for this purpose)
- remove the mgmt extension configuration - it is now enabled if the UI and the search extensions are enabled
userprefs estension:
- userprefs are enabled if both search and ui extensions are enabled (as opposed to just search)
apikey extension is removed and logic moved into the api folder
- Move apikeys code out of pkg/extensions and into pkg/api
- Remove apikey configuration options from the extensions configuration and move it inside the http auth section
- remove the build label apikeys
other changes:
- move most of the logic adding handlers to the extensions endpoints out of routes.go and into the extensions files.
- add warnings in case the users are still using configurations with the obsolete settings for mgmt and api keys
- add a new function in the extension package which could be a single point of starting backgroud tasks for all extensions
- more clear methods for verifying specific extensions are enabled
- fix http methods paired with the UI handlers
- rebuild swagger docs
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
The ui sends the header X-ZOT-API-CLIENT=zot-ui regardless of session authentication status.
In case of new sessions zot would reject the unauthenticated call on /v2 (which is used to determine
if anonymous access is allowed by the server when the header was set) expecting all users sending
this header to be already authenticated.
Since the ui received 401 from the server, it would not show the option for anonymous login.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* fix(conformance): OCI-Filters-Applied should return a literal
https://github.com/opencontainers/distribution-spec/issues/448
Should only indicate what filter-type was applied and not what exact
values it was filtered on.
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* ci(conformance): point to temporary repo
Revert this commit once upstream changes have been merged
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
dist-spec APIs independently allow deletion of blobs and manifests.
Doing the former when in use by an image manifest or index is simply
error-prone. So disallow it.
Fixes issue #1509
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
This change introduces OpenID authn by using providers such as Github,
Gitlab, Google and Dex.
User sessions are now used for web clients to identify
and persist an authenticated users session, thus not requiring every request to
use credentials.
Another change is apikey feature, users can create/revoke their api keys and use them
to authenticate when using cli clients such as skopeo.
eg:
login:
/auth/login?provider=github
/auth/login?provider=gitlab
and so on
logout:
/auth/logout
redirectURL:
/auth/callback/github
/auth/callback/gitlab
and so on
If network policy doesn't allow inbound connections, this callback wont work!
for more info read documentation added in this commit.
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
when pushing manifests, zot will validate blobs (layers + config blob) are
present in repo, currently it opens(in case of filesystem storage) or download(
in case of cloud storage) each blob.
fixed that by adding a new method ImageStore.CheckBlobPresence() on storage
to check blobs presence without checking the cache like ImageStore.CheckBlob() method does.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
peusebiu