Commit Graph

1013 Commits

Author SHA1 Message Date
Alexei Dodon a79d79a03a fix: more accurate storage metrics after zot restart (#1972)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-11-01 09:09:21 -07:00
Ramkumar Chinchani 3e6053e1db chore: fix dependabot alerts (#1986)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-30 14:47:11 -07:00
LaurentiuNiculae 56ad9e6707 refactor(metadb): improve UX by speeding up metadb serialize/deserialize (#1842)
Use protocol buffers and update the metadb interface to better suit our search needs

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-30 13:06:04 -07:00
Alexei Dodon f34af3cb3f fix: Makefile 'run' target should not depend on running tests (#1969)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-27 10:40:12 -07:00
Alexei Dodon f3bb68ae15 fix: ci 'TLS check' job is broken (#1970)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-27 10:39:04 -07:00
Alexei Dodon d2fbd273ba fix: tests refactoring (#1950)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-26 11:20:39 +03:00
Ramkumar Chinchani a3d8202345 chore: fix dependabot alerts (#1961)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-24 12:17:59 +03:00
Ramkumar Chinchani 4cb7a6c755 ci: use runners provided by CNCF (#1946)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-21 12:07:32 +03:00
peusebiu 7ab2032a21 feat(api): repair corrupted blobs when pushed again (#1927)
CheckBlob() returns ErrBlobNotFound on corrupted blobs

closes #1922

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-20 13:02:31 -07:00
Ramkumar Chinchani 1675f30d4a ci: update golangci-lint version (#1834)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-20 17:27:04 +03:00
Andrei Aaron 7ce5a74598 feat: use the "zot" namespace for the authentication url (#1947)
Some other minor fixes for swaggo comments (indentation and a bad description)

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-20 15:30:56 +03:00
Alexei Dodon a345ba0823 fix: metrics should be protected behind authZ (#1895)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-20 10:33:26 +03:00
Andreea Lupu a44ca578a1 fix(tests): update imagetrust tests to use mock service (#1929)
- use secretsManagerMock and secretsManagerCacheMock to avoid failing
because of "already exists" error when running multiple times
image_trust_test on the same localstack instance

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-10-18 13:25:29 +03:00
peusebiu 7f6534a52d fix(sessions): periodically cleanup expired sessions (#1939)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-16 20:03:42 -07:00
Ramkumar Chinchani d60786c3b2 chore: fix dependabot alerts (#1942)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-16 15:19:26 -07:00
Alexei Dodon 12227b4484 fix: quick metrics setup in kind with prometheus (#1897)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-16 11:51:00 +03:00
Andreea Lupu fc2380b57b fix: add support for uploaded index when signing using notation (#1882)
ci(notation): update to latest notation version
fix(sync): add layers info when syncing signatures

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-10-12 18:45:20 -07:00
peusebiu 458d40fb48 fix(nightly): increase nightly delays and timeouts (#1877)
change the startup dedupe blackbox test source image with a lighter one

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-12 16:42:07 +03:00
peusebiu a91c0c5cfe fix(authn): create sessions only if UI header value is supplied (#1919)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-12 16:37:55 +03:00
peusebiu d1fcab421a fix(authn): apply fail delay only if credentials/sessions are supplied (#1920)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-12 16:32:36 +03:00
Ramkumar Chinchani ab45356d9a feat(ui): update UI version (#1926)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-12 12:27:30 +03:00
peusebiu 04048e5ad4 fix(sync): fix data race when pinging registries by read-locking (#1924)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-12 11:00:33 +03:00
Ramkumar Chinchani 556c0660bd chore: fix dependabot alerts (#1915)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-11 14:29:59 +03:00
peusebiu 53f97eb265 fix(cache): make dynamoDB aware of orignal/deduped blobs (#1881)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-10 10:29:07 -07:00
Alexei Dodon 039648a445 ci(licenses): check if licenses file is updated (#1898)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-09 20:23:51 -07:00
Andrei Aaron ee25985c3e chore(modules): update trivy to the tip of main (#1901)
Includes https://github.com/aquasecurity/trivy/commit/ce89d083453f2293d8176c2ac9f4efa0fdf68095

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-09 15:12:25 -07:00
Ramkumar Chinchani ed775914df chore: fix dependabot alerts (#1911)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-09 11:35:42 -07:00
Alexei Dodon 044ea85279 fix: running tests locally fails (#1879)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-05 14:34:50 +03:00
Ramkumar Chinchani e6902b937f chore: fix dependabot alerts (#1893) 2023-10-05 09:26:20 +03:00
Ramkumar Chinchani b196369ea4 docs: add logging guidelines (#1884)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-04 12:29:46 -07:00
Ramkumar Chinchani 0fb23848b4 chore: fix dependabot alerts (#1885)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-04 12:29:31 -07:00
Ramkumar Chinchani c6407354b5 build: fix container image targets (#1856)
Use stacker to build OCI image.
Use podman/docker to build docker image.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-04 09:12:56 -07:00
Andrei Aaron ca1c3288cf refactor(test): make sure cli tests are not internal unless they need to be (#1878)
As part of this change searchConfig needed to be exported,
as it was passed as a parameter to exported functions

At this moment most of the tests remaining internal depend on the mock service.
The interface it implements has unexported methods.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-03 11:15:39 -07:00
Andrei Aaron 99e29c0f46 refactor(tests): Migrate some of the older tests to the new image-utils library (#1863)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-02 11:10:43 -07:00
Alexei Dodon 2fd7bfc37a fix: metrics endpoint must be secured behind authN (#1864)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-02 16:37:21 +03:00
peusebiu 0eb984426e fix(nightly): create bucket for s3 testing with localstack (#1872)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-02 15:31:50 +03:00
Alexei Dodon 75085dcff5 fix: errors returned by zot should match the dist-spec errors (#1868)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-29 13:45:19 -07:00
Ramkumar Chinchani 56e3580687 ci: update localstack to 2.3.1 (#1869)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-29 11:32:27 -07:00
Andrei Aaron 0ec6f8acf8 ci(localstack): pin localstack python package to 2.2.0 and pull container image from ghcr (#1867)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-29 11:40:46 +03:00
Alexei Dodon 3a9a932791 fix: reduce test run time (#1832)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-28 21:59:52 +03:00
Andrei Aaron ba6f347d8d refactor(pkg/test): split logic in pkg/test/common.go into multiple packages (#1861)
Which could be imported independently. See more details:
1. "zotregistry.io/zot/pkg/test/common" - currently used as
   tcommon "zotregistry.io/zot/pkg/test/common" - inside pkg/test
   test "zotregistry.io/zot/pkg/test/common" - in tests
   . "zotregistry.io/zot/pkg/test/common" - in tests
Decouple zb from code in test/pkg in order to keep the size small.

2. "zotregistry.io/zot/pkg/test/image-utils" - curently used as
   . "zotregistry.io/zot/pkg/test/image-utils"

3. "zotregistry.io/zot/pkg/test/deprecated" -  curently used as
   "zotregistry.io/zot/pkg/test/deprecated"
This one will bre replaced gradually by image-utils in the future.

4. "zotregistry.io/zot/pkg/test/signature" - (cosign + notation) use as
   "zotregistry.io/zot/pkg/test/signature"

5. "zotregistry.io/zot/pkg/test/auth" - (bearer + oidc)  curently used as
   authutils "zotregistry.io/zot/pkg/test/auth"

 6. "zotregistry.io/zot/pkg/test/oci-utils" -  curently used as
   ociutils "zotregistry.io/zot/pkg/test/oci-utils"

Some unused functions were removed, some were replaced, and in
a few cases specific funtions were moved to the files they were used in.

Added an interface for the StoreController, this reduces the number of imports
of the entire image store, decreasing binary size for tests.
If the zb code was still coupled with pkg/test, this would have reflected in zb size.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-27 11:34:48 -07:00
peusebiu c3801dc3d3 fix(dedupe): run dedupe only for repositories found at startup (#1844)
no need to run dedupe/restore blobs for images being pushed or synced while
running dedupe task, they are already deduped/restored inline.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-27 11:15:08 -07:00
Andreea Lupu 92e382ce39 refactor(scrub): replace umoci logic in scrub implementation (#1845)
- implement scrub also for S3 storage by replacing umoci
- change scrub implementation for ImageIndex
- take the `Subject` into consideration when running scrub
- remove test code relying on the umoci library. Since we started
relying on images in test/data, and we create our own images using
go code we can obtain digests by other means. (cherry picked from commit 489d4e2d23c1b4e48799283f8281024bbef6123f)

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-09-26 11:02:11 -07:00
Márk Sági-Kazár 510b7a2e16 build: add cross-compile to dockerfile (#1816)
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2023-09-26 09:09:34 -07:00
Ramkumar Chinchani 80b580adaa ci: add a "nightly jobs" badge (#1858)
So that we know if something has failed.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-26 09:07:46 -07:00
Ramkumar Chinchani 9096031aeb chore: fix dependabot alerts (#1855)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-25 23:03:13 +03:00
Andrei Aaron 6bd7abe28b fix(tests): call ImageStore constructor with correct parameters (#1846)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-23 07:58:58 +00:00
peusebiu 1df743f173 fix(gc): sync repodb when gc'ing manifests (#1819)
fix(gc): fix cleaning deduped blobs because they have the modTime of
the original blobs, fixed by updating the modTime when hard linking
the blobs.
fix(gc): failing to parse rootDir at zot startup when using s3 storage
because there are no files under rootDir and we can not create empty dirs
on s3, fixed by creating an empty file under rootDir.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-22 11:51:20 -07:00
Andrei Aaron 7c78f80a96 feat(cve): implement CVE scanning as background tasks (#1833)
1. Move existing CVE DB download generator/task login under the cve package
2. Add a new CVE scanner task generator and task type to run in the background, as well as tests for it
3. Move the CVE cache in its own package
4. Add a CVE scanner methods to check if an entry is present in the cache, and to retreive the results
5. Modify the FilterTags MetaDB method to not exit on first error
This is needed in order to pass all tags to the generator,
instead of the generator stopping at the first set of invalid data
6. Integrate the new scanning task generator with the existing zot code.
7. Fix an issue where the CVE scan results for multiarch images was not cached
8. Rewrite some of the older CVE tests to use the new image-utils test package
9. Use the CVE scanner as attribute of the controller instead of CveInfo.
Remove functionality of CVE DB update from CveInfo, it is now responsible,
as the name states, only for providing CVE information.
10. The logic to get maximum severity and cve count for image sumaries now uses only the scanner cache.
11. Removed the GetCVESummaryForImage method from CveInfo as it was only used in tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-22 11:49:17 -07:00
Alexei Dodon 4e04be420e refactor(cli): Move cmdflags package under pkg/cli/client (#1840)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-22 16:33:18 +03:00