Commit Graph

734 Commits

Author SHA1 Message Date
peusebiu f04e66a5e2 feat(mgmt): added mgmt extension which returns current zot configuration (#1198)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-03-09 10:43:26 -08:00
LaurentiuNiculae 4c156234cb feat(repodb): sync-repodb WIP (#1241)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-03-09 10:41:48 -08:00
Andrei Aaron fd5a2af10b feat(ui): zui can now show multiple manifests per image (#1254)
Update to a zui version with multiarch image support
Also fix some issues with the filtering on the global search page

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-03-09 09:00:01 -08:00
Lisca Ana-Roberta 336526065f feat(groups)!: added "groups" mechanism for authZ (#1123)
BREAKING CHANGE: repository paths are now specified under a new config key called "repositories" under "accessControl" section in order to handle "groups" feature. Previously the repository paths were specified directly under "accessControl".

This PR adds the ability to create groups of users which can be used for authZ policies, instead of just users.

{
"http": {
   "accessControl": {
       "groups": {

Just like the users, groups can be part of repository policies/default policies/admin policies. The 'groups' field in accessControl can be missing if there are no groups. The permissions priority is user>group>default>admin policy, verified in this order (in authz.go), and permissions are cumulative. It works with LDAP too, and the group attribute name is configurable. The DN of the group is used as the group name and the functionality is the same. All groups for the given user are added to the context in authn.go. Repository paths are now specified under a new keyword called "repositories" under "accessControl" section in order to handle "groups" feature.

Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2023-03-08 11:47:15 -08:00
peusebiu 79783b4b06 feat(sync): skip already synced images in sync ondemand (#1234)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-03-07 09:58:42 -08:00
Ramkumar Chinchani c2bec0d4a8 chore(go.mod): fix dependabot alerts (#1251)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-03-07 09:59:59 +02:00
Andrei Aaron 73b1126bbf chore(go.mod): fix dependabot alerts (#1247)
Supersedes:
- https://github.com/project-zot/zot/pull/1132
- https://github.com/project-zot/zot/pull/1243
- https://github.com/project-zot/zot/pull/1244
- https://github.com/project-zot/zot/pull/1245

Also update the AWS SDK libraries used

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-03-06 11:05:19 -08:00
Andrei Aaron c9b594d03d feat(ui): update to the latest zui version which supports new multiarch image APIs (#1246)
Note patch does not mean full support for showing multiarch images in ZUI,
a single architecture is shown, but it fixes the ZUI views which were broken
since #1147

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-03-06 09:42:05 -08:00
peusebiu e712b64c28 feat(doc): add documentation for dynamodb (#1236)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-03-06 09:25:20 -08:00
Andreea Lupu 646250736e fix(go.mod): replace opencontainers/umoci dependency with project-stacker/umoci (#1240)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-03-05 21:11:07 -08:00
Nicol 23efe24c72 refactor(test): update cve tests to stop duplicating test/data if not needed (#1232)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2023-03-02 09:45:23 -08:00
Lisca Ana-Roberta 6bbf730061 fix: trivydb update now uses task scheduler (#1204)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2023-03-02 09:43:54 -08:00
Ramkumar Chinchani 5a2fb4108d chore(go.mod): fix dependabot alerts (#1228)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-28 17:38:49 +02:00
Andrei Aaron 5968e7199f test(ui): add owasp zap scanner in ci/cd (#1224)
(cherry picked from commit 6d03ce5f2d)

Additional changes on top of: 6d03ce5f2d
- Build and use zot from the same branch
do not use a container image as scan target, use the binary
- Fix typo in rules filename
- Add the full rule list to the rules config file
- Ignore some of the specific rules and add reasons
- Add security-related headers to fix some of the issues identified by the scan
- Update UI it includes the latest fixes for zap scan issues

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-27 11:25:47 -08:00
LaurentiuNiculae d62c09e2cc feat(repodb): Multiarch Image support (#1147)
* feat(repodb): index logic + tests

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(cli): printing indexes support using the rest api

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-02-27 11:23:18 -08:00
Nicol a561d0bad5 refactor(test): remove unnecessary usage of images copied from under test/data (#1217)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2023-02-24 14:22:47 -08:00
Andrei Aaron 792f3f55b3 feat(graphql): Image() call now returns a non-nullable ImageSummary (#1216)
This is for consistency with the other calls, and should help in making
all ZUI handling of ZOT errors consistent

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-24 12:22:01 -08:00
Ramkumar Chinchani f6a540747f chore(go.mod): fix dependabot alerts (#1222)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
v2.0.0-rc3
2023-02-23 22:32:20 +02:00
Andrei Aaron ed4954ab0d build(ui): the ui is now included in the zot binary by default (#1202)
Update the default value of the EXTENSIONS variable in the makefile.
Also cleanup binary-ui and other make targets assuming the UI was not included by default.
Enable the ui by default in the zot container image
Swith back to using the distroless images, as c3 only has amd64 images.
Fix updating security events in github (permission issue)
Add an integration test for the UI extension
Rename ui extension files to use _ instead of -
feat(ui): upgrade to zui v2.0.0-rc3

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-23 22:28:08 +02:00
Ramkumar Chinchani 4a56e30cd7 chore(go.mod): fix dependabot alerts (#1218)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-23 09:09:28 +02:00
Andrei Aaron b1c6d945c6 chore(codecov): use a token to authenticate to codecov (#1212)
This should not be necessary for public repos,
but should help with rate limitting

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-17 23:10:28 -08:00
Ramkumar Chinchani be33f7b252 chore(go.mod): fix dependabot alerts (#1210)
* chore(go.mod): fix dependabot alerts

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* chore(test): update image tags

We have cleaned up older golang images in the project.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* ci(gqlgen): fix gql schema validation GH workflow after npm upgrade

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-17 13:54:49 -08:00
Andrei Aaron 1f9f178a57 ci(golang): fix syncing build image golang 1.20 (#1205)
Since the matrix values were unquoted,
the logic reading the yaml considered them numbers

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-17 10:02:30 -08:00
Ramkumar Chinchani 672a2cd384 chore: add/sync golang 1.20.x images (#1200)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-16 11:25:32 +02:00
Bogdan Bivolaru 7c3bf86a6b refactor: Centralise extensions config entries (#1177)
Except for registry sync config

Signed-off-by: Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com>
2023-02-15 22:20:28 -08:00
peusebiu 4aa0106b0a feat(scheduler): use an worker pool for scheduler (#1146)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-02-15 11:36:50 -08:00
Nicol f00a9e6e48 perf: update the ImageList queries to return PaginatedImagesResult (#1182)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2023-02-15 11:34:07 -08:00
Ramkumar Chinchani d79ad863e4 fix: set GC delay defaults for storage subPaths (#1189)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-14 09:16:37 -08:00
Andrei Aaron 0cddb2be3e feat(ui): use a Makefile variable to download a pre-existing zui build (#1196)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-13 22:55:05 +02:00
Ramkumar Chinchani b9a75b2e44 chore(go.mod): fix dependabot alerts (#1194)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-13 12:48:05 -08:00
Andreea Lupu ee95ab0ffc fix: call notation-go libs instead of using notation binary (#1104)
fix: add loading notation path

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
Co-authored-by: Roxana Nemulescu <roxana.nemulescu@gmail.com>
2023-02-13 10:43:52 -08:00
Amit Marcus 2377d62344 feat(sync): onDemand check for updated manifest (#1190)
Signed-off-by: Amit Marcus <marxus@gmail.com>
2023-02-13 10:33:48 -08:00
Andrei Aaron c0aaca8ed1 feat(ui): package zui within zot binary (#1161)
(cherry picked from commit d557da0baba819b7cd7e6b5941528776e125ac6d)

build(ui): fix stacker builds


(cherry picked from commit ba25daf02b4a9bc7ee1cb6f84b7a6b096ca7d61f)

build(ui): various fixes

- Fix metrics endpoint
- Fix unit tests unit tests
- Make the ui build optional in the makefile
before the linter lint runs in the golangci-lint workflow
- Do not attempt to include UI routes if search is enabled
- Fix authorization for search endpoint



fix: use zot tag in ui make target


(cherry picked from commit 2a6882fa23f06b2d68c6c299773a6ff50bf90e78)

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-10 14:52:54 -08:00
Andrei Aaron d12836e69c refactor(cve): improve CVE test time by mocking trivy (#1184)
- refactor(cve): remove the global of type cveinfo.CveInfo from the extensions package
  Replace it with an attribute on controller level
- refactor(controller): extract initialization logic from controller.Run()
- test(cve): mock cve scanner in cli tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-09 21:04:52 -08:00
Andrei Aaron c1de15c87b ci: Revert "test: show the execution times of the tests (#1163)" (#1186)
Exit code was not corrctly handled because of the pipes/tee,
so the job would not fail if tests were failing. Reverting this for now.

This reverts commit 74013a71af.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-09 14:28:01 +02:00
Andrei Aaron 74013a71af test: show the execution times of the tests (#1163)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-07 20:25:08 +02:00
Ramkumar Chinchani c154ab02f3 chore(deps): fix dependabot alerts (#1179)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-02-07 08:52:17 +02:00
Andrei Aaron af222de108 test: refactor CVE tests in CLI package (#1170)
Changes in this commit:

A.
There were nested conveys following the pattern:
- outer convey was a positive test
- inner conveys were negative tests
The positive and negative tests did not depend on one another, but the positive test was repeated for each negative test.

So for example if there was 1 positive test and 2 negative tests what go would do is:
1. execute the outer convey with the positive test
2. execute the 1st inner convey with the 1st negative test
3. execute the outer convey with the positive test
4. execute the 2nd inner convey with the 2nd negative test

Since there were no dependencies between these tests they can be moved on the same level.

B.
The test function bringing up the controller for `TestServerCVEResponseGQL` was the same for `TestServerCVEResponse`, so I removed `TestServerCVEResponseGQL` and moved the Conveys under `TestServerCVEResponse`. This will save time on the initial TrivyDB download, since it's done only once when the controller starts.

C. The `Convey("without flags", func() {` was actually the same as the test under which it was nested: `Convey("Test CVE invalid url port", t, func() {` no code differences so I removed it.

D. Use the new test functions to start/stop and wait for the zot contoller to be reachable

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-06 13:16:28 -08:00
Andrei Aaron 3676c5889e test: avoid running trivy tests if search build label is missing (#1172)
The 'test' makefile target runs the tests for both 'minimal' and equivalent of the former 'extended' build.
The trivy package tests were run twice, even if the trivy logic is unreachable if search is disabled.
With this update we should see a cut of about 150s of test time.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-04 23:21:45 -08:00
Andrei Aaron 3cb99f4ed9 ci(cache): split go build cache from go modules cache (#1169)
The cache should not be the same if we build for different OSs/architectures

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-02-04 09:57:07 +02:00
Andrei Aaron 78feb4b495 docs(graphql): rewrote search.md (#1130)
* docs(graphql): rewrote search.md
docs(graphql): added pagination and filter docs for gql

Squash of both commits:
(cherry picked from commit 2268fa0510b32b27f2c1f71e9889ec769877553b)
(cherry picked from commit c96adc88b2fb9edff90e7e4b01a8885511ceb0df)
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>

Some additional updates by Andrei Aaron to keep up with the changes on main.
Also add more comments to schema.graphql

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

* style: run a graphql schemma linter as a github workflow

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

---------

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2023-02-02 12:46:35 -08:00
Nicol ba3f6f7492 fix(test): update the zot tests not to use test/data as rootDir (use a temporary folder instead) (#1162)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2023-02-02 11:39:03 -08:00
Ramkumar Chinchani 863d057e43 chore(deps): fix dependabot alerts (#1153)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
v2.0.0-rc2
2023-02-01 00:21:14 -08:00
Lisca Ana-Roberta 976ccfcf0d fix: removed references to old dist-spec (#1128)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2023-01-31 09:35:33 -08:00
Ramkumar Chinchani 1b618f305e test(exporter): add unit test to cover cli pkg (#1148)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-01-27 20:35:32 -08:00
Ramkumar Chinchani b3ed92ef1a chore(deps): fix dependabot alerts (#1143)
https://github.com/project-zot/zot/pull/1119
https://github.com/project-zot/zot/pull/1142

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-01-26 22:39:18 -08:00
Andrei Aaron e04d98272c chore: update the version of go-lru we use to the latest available (#1141)
We are now using v2.0.1 in the cve cache logic.
Unfortunately we are also using v0.5.4 indirectly, as it is required for gqlgen, see:
https://github.com/99designs/gqlgen/blob/e6114a2c6af22bcdc92180660a58e6125e7946ad/go.mod#L7

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-01-26 11:14:17 -08:00
Ramkumar Chinchani e2c7a3c5ba fix(referrers): fix some conformance issues (#1134)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-01-26 10:13:12 -08:00
Andrei Aaron feb7328f50 feat(repodb): DerivedImageList and BaseImageList make use of RepoDB (#1135)
- derivedImageList and baseImageList now use FilterTags to obtain results,
each with its own filter function
- images that have the exact same manifest as the one provided as a
parameter are no longer considered base images or derived images
- both calls can be made with specific pagination parameters, and the
response will include PageInfo

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>

fix(tests): fix one of the pagination tests

The results were not reliable as the 2 returned tags were sorted by created date/time
which was not set, resulting in an unpredictable order

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
(cherry picked from commit be504200a1127371422aeb0e5c0219e2a1ead20a)
(cherry picked from commit ed8d797e639f262a63840120afe92da7db9a7600)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
2023-01-25 14:06:02 -08:00
Ramkumar Chinchani be4b8c6243 fix(referrers): annotation key is incorrect (#1139)
"If filtering is requested and applied, the response MUST include an
annotation (org.opencontainers.referrers.filtersApplied) denoting that
an artifactType filter was applied.""

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-01-25 12:50:03 -08:00