6f00e843a0
fix(sync): sync generator now backs off on errors ( #2272 )
...
handle unsupported features like oci artifacts.
closes : #2238
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2024-03-04 09:44:11 -08:00
740eae8f26
fix(sync): better cleaning sync's download dir ( #2273 )
...
added cleanup in the case of copy.Image() failures.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2024-02-29 09:09:21 -08:00
8e68255946
fix(sync): added bearer client for sync ( #2222 )
...
fixed ping function taking too much time
closes : #2213 #2212
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2024-02-14 09:18:10 -08:00
6a83dd47c0
fix(scheduler): the session cleanup generator is reset too often ( #2220 )
...
This causes the "fair" scheduler to run it too often in the detriment of other generators.
The intention was to run it every 2 hours but the measurement unit for 7200 was not specified.
Add more logs, including showing a generator name, in order to troubleshoot this kind of issues easier in the future.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com >
2024-02-01 09:15:53 -08:00
ce4924f841
refactor: rename go module from zotregistry.io/zot to zotregistry.dev/zot ( #2187 )
...
Signed-off-by: Andrei Aaron <aaaron@luxoft.com >
2024-01-31 20:34:07 -08:00
79e14027ee
refactor(test): add lint rule for messages starting with the component ( #2045 )
...
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com >
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
Signed-off-by: Andrei Aaron <aaaron@luxoft.com >
2023-12-08 10:05:02 +02:00
2e733b3f4f
feat(metrics): add scheduler related metrics ( #2076 )
...
Signed-off-by: Alexei Dodon <adodon@cisco.com >
2023-12-04 14:13:50 -08:00
3c8da6e6fc
Sync s3 ( #2073 )
...
* feat(sync): local tmp store
Signed-off-by: a <a@tuxpa.in >
* fix(sync): various fixes for s3+remote storage feature
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
---------
Signed-off-by: a <a@tuxpa.in >
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
Co-authored-by: a <a@tuxpa.in >
2023-11-28 12:08:15 -08:00
59dc4c3229
feat(scheduler): pass the shutdown/reload ctx to running tasks ( #1671 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-09-05 09:48:56 -07:00
76277f5ebd
fix: remove inline GC and schedule a background task instead ( #1610 )
...
* fix: remove inline GC and set a default value of gc interval
- remove inline GC
- add a default value of GC interval
- run the GC periodically by default with the default value if no interval provided
- generate GC tasks with a random delay(0-30s) between
- add IsReady() method to scheduler.TaskGenerator interface
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
* ci: add test for gc with short interval
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
---------
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
2023-08-07 12:55:19 -07:00
d4f200c2e1
fix: changing default numWorkers, making it customizable and refactoring scheduler ( #1563 )
...
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com >
2023-07-04 11:03:29 +03:00
612a12e5a8
refactor(sync): use task scheduler ( #1301 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-05-31 10:26:23 -07:00
a3f355c278
refactor(storage): refactoring storage ( #1459 )
...
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2023-05-26 11:08:19 -07:00
c169698c95
feat: remove usage of zerolog.Logger.Msgf() from zot code ( #1382 )
...
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
2023-04-27 19:44:22 -07:00
3dd3c46ee3
test: stop task scheduler between test runs ( #1311 )
...
sync: remove sync WaitGroup, it's stopped with context
sync: onDemand will always try to sync newest image when a tag is used
if a digest is used then onDemand will serve local image
test(sync): fix flaky coverage in sync package
closes #1294
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-03-29 11:37:58 -07:00
4c156234cb
feat(repodb): sync-repodb WIP ( #1241 )
...
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2023-03-09 10:41:48 -08:00
79783b4b06
feat(sync): skip already synced images in sync ondemand ( #1234 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-03-07 09:58:42 -08:00
7c3bf86a6b
refactor: Centralise extensions config entries ( #1177 )
...
Except for registry sync config
Signed-off-by: Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com >
2023-02-15 22:20:28 -08:00
4aa0106b0a
feat(scheduler): use an worker pool for scheduler ( #1146 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-02-15 11:36:50 -08:00
ee95ab0ffc
fix: call notation-go libs instead of using notation binary ( #1104 )
...
fix: add loading notation path
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
Co-authored-by: Roxana Nemulescu <roxana.nemulescu@gmail.com >
2023-02-13 10:43:52 -08:00
14238d4a8d
fix: removed resty calls from sync ( #1016 )
...
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com >
2022-12-22 10:19:42 -08:00
024b13efe6
fix(sync): syncing OCI artifacts with distribution package fails ( #1013 )
...
sync OCI artifacts using REST APIs
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-12-09 11:38:00 -08:00
e96c80c344
feat(sync,s3): added s3 logic for ORAS and OCI artifacts ( #985 )
...
added sync logic for OCI artifacts
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-11-14 22:21:49 -08:00
2d877aaea1
fix(sync): also sync on demand digests, not only tags, closes #902 ( #932 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-27 09:39:59 -07:00
ac6c6a844c
refactor(digests): standardise representation of digests to digest.Digest ( #898 )
...
- Digests were represented by different ways
- We needed a uniform way to represent the digests and enforce a format
- also replace usage of github.com/google/go-containerregistry/pkg/v1
with github.com/opencontainers/image-spec/specs-go/v1
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
(cherry picked from commit 96b2f29d6d57070a913ce419149cd481c0723815)
(cherry picked from commit 3d41b583daea654c98378ce3dcb78937d71538e8)
Co-authored-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2022-10-22 13:46:13 -07:00
5f99f9a445
fix(sync): fixed broken logic to get tags for repo ( #900 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-22 00:26:14 -07:00
38b00e3507
chore(lint): gci to separate zot from other imports ( #870 )
...
Signed-off-by: Andrei Aaron <andaaron@cisco.com >
2022-10-20 09:39:20 -07:00
3c0c51fcbe
fix(sync): also sync image index mediatype ( #847 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-06 09:41:16 -07:00
c146448f01
fix(sync): revert code which removed image destination feature ( #840 )
...
Added an end to end test for this feature, closes #793
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-05 11:03:24 -07:00
33a431ef43
Update go version to 1.19 ( #829 )
...
* ci: Update go version to 1.19
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
* ci: Fix lint issues
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
* ci: Added needprivileges to lint, made needprivileges pass lint
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
Co-authored-by: Catalin Hofnar <catalin.hofnar@gmail.com >
2022-10-05 13:21:14 +03:00
885f139e0e
Remove forking logger ( #825 )
...
- no longer needed, the race conditions were fixed
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2022-09-29 13:28:39 -07:00
87fc941b3c
image level lint: enforce manifest mandatory annotations
...
closes #536
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com >
2022-07-27 11:48:04 +03:00
a8a65a6c37
Modified sync log calls to include error type ( #336 )
...
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
2022-06-15 09:45:49 -07:00
da4acaf178
sync: preserve upstream digests after syncing images
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-05-25 10:19:36 -07:00
bd730150a8
sync: allow HTTP redirects when GETing signatures blobs
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-05-25 10:19:36 -07:00
36c9631000
ext: use distribution spec route prefix for extension api
...
Following the spec defined here https://github.com/opencontainers/distribution-spec/tree/main/extensions
Signed-off-by: Shivam Mishra <shimish2@cisco.com >
2022-05-22 16:35:16 -07:00
20a60cbad4
Enhance sync logic - stop blob redownloads and re-pushes ( #479 #480 )
...
Changed imagesToCopyFromUpstream to return a map[string][]types.ImageReference from just an array of refs
Rewrote some logic in sync.go to use the new signature of imagesToCopyFromUpstream
Split getLocalImageRef by adding function getLocalCachePath
Adapted tests for new changes, added some tests
Merged #481
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
2022-05-16 10:05:01 -07:00
d325c8b5f4
Fix problems signaled by new linter version v1.45.2
...
PR (linter: upgrade linter version #405 ) triggered lint job which failed
with many errors generated by various linters. Configurations were added to
golangcilint.yaml and several refactorings were made in order to improve the
results of the linter.
maintidx linter disabled
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro >
2022-04-27 09:55:44 -07:00
f53dc9eb8d
sync: Add a new flag to enforce syncing only signed images, closes #455
...
sync: When checking if a image is already synced also check for changes in upstream signatures.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-03-24 10:50:01 -07:00
0d148e1d6b
new config option for sync-destination
...
Signed-off-by: laurentiuNiculae <themelopeus@gmail.com >
2022-03-21 08:12:34 -07:00
6d04ab3cdc
sync: support reloading sync config when the config file changes
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-03-18 11:24:11 -07:00
45968e0bb7
sync: fix inconsistent test, used inject error fw for hard to reach test cases
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-03-01 09:56:27 -08:00
35eeedb22a
sync: pull only missing images, not everything, closes #335
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-02-14 08:09:18 -08:00
47c9b6244e
Added config enable=true/false for extensions with default value as enabled closes #258
...
Signed-off-by: Alexei Dodon <adodon@cisco.com >
2022-02-09 09:53:49 -08:00
f89925fb27
sync: periodically retry if on-demand fails inline, closes #281
...
sync: don't return error on sync signatures, just skip them, closes #375
sync: sync signatures on demand
sync on demand: in case of parallel requests pull image just once, closes #344
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-02-07 09:04:13 -08:00
a0e65379c8
sync: for a prefix, allow multiple registries as a list instead of only one, closes #343
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-02-01 09:45:09 -08:00
1109bb4dde
sync: Added support for syncing notary/cosign signatures, closes #261
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-01-13 08:45:59 -08:00
ac3801ea2d
lint: upgrade golangci-lint
...
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com >
2021-12-20 17:20:35 -08:00
5f04092e71
Fix negative WaitGroup counter at runtime
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2021-12-17 16:52:34 -08:00
c86f44cc53
Disable sync periodically polling when pollInterval is not configured
...
Filtering out sync on demand images based on content configuration
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2021-12-14 08:59:50 -08:00
peusebiu