Commit Graph

371 Commits

Author SHA1 Message Date
Ramkumar Chinchani fb2edcc269 chore: fix dependabot alerts (#2486)
https://github.com/project-zot/zot/pull/2475
https://github.com/project-zot/zot/pull/2477
https://github.com/project-zot/zot/pull/2478
https://github.com/project-zot/zot/pull/2479
https://github.com/project-zot/zot/pull/2480
https://github.com/project-zot/zot/pull/2481
https://github.com/project-zot/zot/pull/2482
https://github.com/project-zot/zot/pull/2483
https://github.com/project-zot/zot/pull/2484
https://github.com/project-zot/zot/pull/2485

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-06-17 20:31:01 +03:00
peusebiu e023936e8e fix(ui): fix image details view (#2470)
when a UI client tries to view image details
for an image with multiple tags pointing to the same digest
we make a query to dynamodb having duplicate keys (same digest)
resulting in an error and the client is redirect back to image
overview.

closes: #2464

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-06-14 09:22:48 -07:00
Ramkumar Chinchani f5fef2384a chore: fix dependabot alerts (#2462)
* chore: fix dependabot alerts

https://github.com/project-zot/zot/pull/2451
https://github.com/project-zot/zot/pull/2452
https://github.com/project-zot/zot/pull/2453
https://github.com/project-zot/zot/pull/2454
https://github.com/project-zot/zot/pull/2455
https://github.com/project-zot/zot/pull/2456
https://github.com/project-zot/zot/pull/2457
https://github.com/project-zot/zot/pull/2458
https://github.com/project-zot/zot/pull/2459
https://github.com/project-zot/zot/pull/2460
https://github.com/project-zot/zot/pull/2461
https://github.com/project-zot/zot/pull/2463

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* chore: mockoidc has moved to github.com/go-jose/go-jose/v3

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* chore: quiet aws/s3 golang api deprecations

These need to be addressed in a separate PR.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-06-12 22:51:32 -07:00
Ramkumar Chinchani 1594852428 chore: fix dependabot alerts (#2446)
* chore: fix dependabot alerts

https://github.com/project-zot/zot/pull/2435
https://github.com/project-zot/zot/pull/2436
https://github.com/project-zot/zot/pull/2437
https://github.com/project-zot/zot/pull/2438
https://github.com/project-zot/zot/pull/2439
https://github.com/project-zot/zot/pull/2440
https://github.com/project-zot/zot/pull/2441
https://github.com/project-zot/zot/pull/2442
https://github.com/project-zot/zot/pull/2443
https://github.com/project-zot/zot/pull/2444
https://github.com/project-zot/zot/pull/2445
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* fix(zli): _schema query in zli code should not use empty parens

Fix also some tests
See https://github.com/vektah/gqlparser/issues/292 and https://github.com/vektah/gqlparser/pull/293

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Andrei Aaron <aaaron@luxoft.com>
2024-06-04 13:54:30 +03:00
Vishwas Rajashekar 767f81d4f5 feat(sync): support for periodic repo sync in scale-out cluster (#2424)
This commit includes support for periodic repo sync in a scale-out
cluster.
Before this commit, all cluster members would sync all the repos as
the config is shared.

With this change, in periodic sync, the cluster member checks whether
it manages the repo. If it does not manage the repo, it will skip the
sync.

This commit also includes a unit test to test on-demand sync too, but
there are no logic changes for it as it is implicitly handled by the
proxying logic.

Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
2024-05-31 09:25:34 -07:00
Andrei Aaron 2bb46b0562 chore: fix dependabot alerts (#2431)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-05-27 14:37:27 -07:00
Vishwas R be5ad66797 refactor(http): refactor http client to accept more customisable options (#2414)
refactor(http): refactor http client to take options struct

This commit updates the arguments for the `CreateHTTPClient`
function to consume a struct which can be extended as required.
It replaces the certPath argument with a struct of 3 paths for
client ertificate, client key, and ca cert. It also adds
a TLSEnabled option for when an HTTP Client is required
without any further TLS config.

Existing consumers of this function have been updated so that
they can work as they do today. This change is a no-op for
existing features.

This allows for certificate paths to be customised and
allows other modules to re-use the same HTTP client and get
the benefits of mTLS support and per-host certificates.

Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
2024-05-06 13:43:41 -07:00
Ramkumar Chinchani 186855b5f8 fix: additional input validation for CVE graphQL query (#2408)
It is possible to ask for a very large limit size which can exhaust
memory.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-04-24 09:23:17 +03:00
Ramkumar Chinchani 28e9aabecf chore: fix dependabot alerts (#2331)
https://github.com/project-zot/zot/pull/2324
https://github.com/project-zot/zot/pull/2325
https://github.com/project-zot/zot/pull/2326
https://github.com/project-zot/zot/pull/2327
https://github.com/project-zot/zot/pull/2328
https://github.com/project-zot/zot/pull/2329

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-03-20 07:37:29 +02:00
Ramkumar Chinchani 18235ca254 fix(oras)!: remove ORAS artifact references support (#2294)
* fix(oras)!: remove ORAS artifact references support

ORAS artifacts/references predated OCI dist-spec 1.1.0 which now has the
same functionality and likely to see wider adoption.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* test: update to released official images

So that they are unlikely to be deleted.
*-rc images may be cleaned up over time.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-03-06 12:16:42 -08:00
LaurentiuNiculae 5039128723 feat(cve): cli cve diff (#2242)
* feat(gql): add new query for diff of cves for 2 images

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(cli): add cli for cve diff

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-03-06 10:40:29 +02:00
peusebiu 6f00e843a0 fix(sync): sync generator now backs off on errors (#2272)
handle unsupported features like oci artifacts.

closes: #2238

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-03-04 09:44:11 -08:00
peusebiu 740eae8f26 fix(sync): better cleaning sync's download dir (#2273)
added cleanup in the case of copy.Image() failures.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-02-29 09:09:21 -08:00
Ramkumar Chinchani 565eca2609 chore: fix dependabot alerts (#2268)
https://github.com/project-zot/zot/pull/2258
https://github.com/project-zot/zot/pull/2259
https://github.com/project-zot/zot/pull/2260
https://github.com/project-zot/zot/pull/2261
https://github.com/project-zot/zot/pull/2262
https://github.com/project-zot/zot/pull/2263
https://github.com/project-zot/zot/pull/2264
https://github.com/project-zot/zot/pull/2265
https://github.com/project-zot/zot/pull/2266
https://github.com/project-zot/zot/pull/2267

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-02-20 21:51:40 +02:00
Andrei Aaron 4e5db84cb1 chore: update image-spec and dist spec to 1.1.0 (#2255)
BREAKING CHANGE: the dist spec version in the config files needs to be bumped to 1.1.0
in order for the config verification to pass without warnings.

Also fix 1 dependabot alert for helm.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-20 13:27:21 +02:00
Vishwas R 0aa6bf0fff feat: include PackagePath data in CVEs for image queries (#2241)
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
2024-02-15 13:19:49 -08:00
peusebiu 8e68255946 fix(sync): added bearer client for sync (#2222)
fixed ping function taking too much time

closes: #2213 #2212

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-02-14 09:18:10 -08:00
Andrei Aaron d0eb043be5 feat: Get the image LastUpdated timestamp from annotations (#2240)
Fallback to Created field and the History entries in the image config
only if the annotation "org.opencontainers.image.created" is not available

closes #2210

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-14 09:14:24 -08:00
Andreea Lupu 55acce6923 feat(graphql): filter CVEs by severity (#2246)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2024-02-14 09:11:57 -08:00
LaurentiuNiculae de90abd5dc style(metadb): use type aliases for metadb types to be easier to read (#2043)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-02-14 09:08:08 -08:00
peusebiu 5b83937d40 fix(tests): fixed inconsistent sync test (#2237)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-02-08 18:28:59 +02:00
Andrei Aaron 6a83dd47c0 fix(scheduler): the session cleanup generator is reset too often (#2220)
This causes the "fair" scheduler to run it too often in the detriment of other generators.
The intention was to run it every 2 hours but the measurement unit for 7200 was not specified.

Add more logs, including showing a generator name, in order to troubleshoot this kind of issues easier in the future.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-01 09:15:53 -08:00
Andrei Aaron ce4924f841 refactor: rename go module from zotregistry.io/zot to zotregistry.dev/zot (#2187)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-31 20:34:07 -08:00
Andrei Aaron a2b923b6fd feat(search): search for a specific tag cross-repo (#2211)
Syntax to search for `<tag_name>` accross all repos is `:<tag_name>`

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-30 09:12:41 -08:00
Andreea Lupu ddba1b7baf fix(scrub): hold locks per image not per repo while executing scrub (#2180)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2024-01-25 11:12:21 -08:00
LaurentiuNiculae 1785688b7c feat(ldap): hot reloading ldap credentials on change (#2167)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-01-25 09:15:22 -08:00
Andrei Aaron ed6be0580e refactor: replace deprecated APIs for creating images in the search tests (#2173)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-22 09:10:34 -08:00
LaurentiuNiculae 3f97f878fd feat(cve): add option to exclude string from cve search (#2163)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-01-19 12:59:42 -08:00
Andrei Aaron 8467a80a50 refactor: update tests to use the newer API for creating test images (#2168)
- update cve tests
- update scrub tests
- update tests for parsing storage and loading into meta DB
- update controller tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-17 10:20:07 -08:00
Raul Kele 59f41ac17d fix(ui): Add users route to extension_ui.go (#2141)
Add new ui route "/user" to ui router

Signed-off-by: Raul-Cristian Kele <raulkeleblk@gmail.com>
2023-12-17 21:56:06 +02:00
Andrei Aaron cff74578be fix(apikey): show api key configuration in mgmt API (#2138)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-14 09:54:51 -08:00
Andrei Aaron 18aa975ae2 feat(CVE): add CVE severity counters to returned images and CVE list calls (#2131)
For CLI output is similar to:

CRITICAL 0, HIGH 1, MEDIUM 1, LOW 0, UNKNOWN 0, TOTAL 2

ID                SEVERITY  TITLE
CVE-2023-0464     HIGH      openssl: Denial of service by excessive resou...
CVE-2023-0465     MEDIUM    openssl: Invalid certificate policies in leaf...

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-13 09:16:31 -08:00
peusebiu dbb1c3519f feat(ui): let UI delete manifests if current user has permissions to do so (#2132)
- added a new field 'IsDeletable' for graphql ImageSummary struct.
- apply cors on DeleteManifest route

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-13 09:06:08 -08:00
peusebiu 7642e5af98 fix(scheduler): fix data race (#2085)
* fix(scheduler): data race when pushing new tasks

the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()

because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers

solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

* refactor(scheduler): refactor into a single shutdown

before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().

simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

---------

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-11 10:00:34 -08:00
LaurentiuNiculae 79e14027ee refactor(test): add lint rule for messages starting with the component (#2045)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-08 10:05:02 +02:00
Andreea Lupu e3bd9a8fa8 fix(log): trimmed error level logs (#2115)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-12-05 12:01:01 +02:00
Alexei Dodon 2e733b3f4f feat(metrics): add scheduler related metrics (#2076)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-12-04 14:13:50 -08:00
Ramkumar Chinchani 8bac653dd2 chore: fix dependabot alerts (#2113)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-12-04 11:28:01 -08:00
LaurentiuNiculae 90d27ff2ac feat(cve): expand search domain to cve description and package info (#2086)
* feat(cve): add reference url for cve

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(cve): expand search domain to cve description and package info

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-29 20:59:00 +02:00
peusebiu 3c8da6e6fc Sync s3 (#2073)
* feat(sync): local tmp store

Signed-off-by: a <a@tuxpa.in>

* fix(sync): various fixes for s3+remote storage feature

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

---------

Signed-off-by: a <a@tuxpa.in>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: a <a@tuxpa.in>
2023-11-28 12:08:15 -08:00
LaurentiuNiculae 0de2210686 feat(metadb): add support for querying for images by a blob digest (#2077)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-27 08:52:52 -08:00
peusebiu 6222dae1f0 feat(scheduler): gracefully shutdown (#1951)
wait for workers to finish before exiting

should fix tests reporting they couldn't remove rootDir because it's being
written by tasks

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-24 00:40:10 -08:00
LaurentiuNiculae 83f287d1f6 feat(cli): add command to interogate the server version and other details (#1709)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-22 19:25:21 +02:00
LaurentiuNiculae 4fb1e756c4 feat(startup): update logic for metadb update on startup, skip unmodified repos (#2024)
- MetaDB stores the time of the last update of a repo
- During startup we check if the layout has been updated after the last recorded change in the db
- If this is the case, the repo is parsed and updated in the DB otherwise it's skipped

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-16 10:39:27 -08:00
Alexei Dodon dd079bf9a3 fix: TestPopulateStorageMetrics fails occasionally in CI (#2042)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-11-14 16:22:24 -08:00
LaurentiuNiculae 272eb7cc43 feat(ldap): add option to load ldap from file (#1778)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-14 16:21:36 -08:00
Andrei Aaron 38f10af8cf docs: update graphql examples to match current implementation (#2038)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-11-13 09:58:15 -08:00
LaurentiuNiculae 2db6e86fb5 fix(cov): coverage boltdb+dynamo (#2018)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-08 13:37:52 -08:00
LaurentiuNiculae c9cc5b9acb test(meta): add push-pull-read tests for metadb (#2022)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-08 13:35:51 -08:00
Andreea Lupu d5065513f5 feat: add support for oci1.1 cosign signatures(using referrers) (#1963)
- Cosign supports 2 types of signature formats:

	1. Using tag -> each new signature of the same manifest is
	added as a new layer of the signature manifest having that
	specific tag("{alghoritm}-{digest_of_signed_manifest}.sig")

	2. Using referrers -> each new signature of the same manifest is
	added as a new manifest

- For adding these cosign signature to metadb, we reserved index 0 of the
list of cosign signatures for tag-based signatures. When a new tag-based
signature is added for the same manifest, the element on first position
in its list of cosign signatures(in metadb) will be updated/overwritten.
When a new cosign signature(using referrers) will be added for the same
manifest this new signature will be appended to the list of cosign
signatures.

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-11-06 14:09:39 -08:00