e2aa088e0d
feat(sync): initial commit for streaming sync
...
initial working prototype for sync
fix: pre-load chunk readers on manifest fetch
feat: make chunkSize configurable
fix minimal build
fix: linter errors
Signed-off-by: Vishwas Rajashekar <dev@vrajashkr.com >
2026-05-22 17:40:25 +05:30
ed40bfd689
fix: more logging for sync extension ( #3656 )
...
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com >
2025-12-16 00:03:24 +02:00
da426850e7
chore: update golangci-lint and fix all issues ( #3575 )
...
* chore: Update golangci-lint
Signed-off-by: Lars Francke <git@lars-francke.de >
* chore: fix all golangci-lint issues
- Remove deprecated `// +build` tags
- Fix godoclint, modernize, wsl_v5, govet, lll, gci, noctx issues
- Update linter configuration
- Modernize code to use Go 1.22+ features (for range N, slices.Contains, etc.)
- Update make check lint the privileged tests
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com >
---------
Signed-off-by: Lars Francke <git@lars-francke.de >
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com >
Co-authored-by: Lars Francke <git@lars-francke.de >
2025-11-22 23:36:48 +02:00
7fa53f5b0f
Sync images with a background context ( #3537 )
...
feat: Sync images with a background context
This means syncs/pulls will not be cancelled anymore when the requesting client disconnects.
The timeout used can be configured per registry
Signed-off-by: Lars Francke <git@lars-francke.de >
2025-11-20 08:52:27 -08:00
2402296e9a
fix: migrate to Go module v2 for proper semantic versioning ( #3462 )
...
* fix: migrate to Go module v2 for proper semantic versioning
This change updates the module path from 'zotregistry.dev/zot' to
'zotregistry.dev/zot/v2' to comply with Go's semantic versioning rules.
According to Go's module versioning requirements, major version v2+
must include the major version in the module path. The current
module path 'zotregistry.dev/zot' only supports v0.x.x and v1.x.x
versions, making existing v2.x.x tags (like v2.1.8) unusable.
Changes:
- Updated go.mod module path to zotregistry.dev/zot/v2
- Updated all internal import paths across 280+ Go source files
- Updated configuration files (golangcilint.yaml, gqlgen.yml)
- Updated README.md Go reference badge
This fix enables proper use of existing v2.x.x Git tags and allows
external packages to import zot v2+ versions without compatibility
errors.
Resolves: Go module import compatibility for v2+ versions
Fixes : #3071
Signed-off-by: Luca Muscariello <muscariello@ieee.org >
* fix: regenerate GraphQL files with updated v2 import paths
The gqlgen tool needs to regenerate the GraphQL schema files after
the module path change to use the new v2 imports.
Signed-off-by: Luca Muscariello <muscariello@ieee.org >
---------
Signed-off-by: Luca Muscariello <muscariello@ieee.org >
2025-10-16 22:43:47 -07:00
0e2aa81439
feat(sync): use regclient for sync extension ( #2903 )
...
* feat(sync): use regclient for sync extension
replaced containers/image package with regclient/regclient package
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* fix(sync): fixed converting innner docker list mediatype
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* feat(sync): added option to preserve digest
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* fix(sync): added coverage and various fixes
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* fix(metadb): fixed converting manifest list not setting platform and annotations
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* fix(sync): remove read lock on storage, not used concurrently
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* feat(sync): added cache for repo tags
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* fix(sync): fixed Makefile
removed opengpg tag
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
* fix(sync): add test for on demand referrer
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
---------
Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com >
2025-04-15 16:58:15 -07:00
d0de12d2d3
[Feat]: Support for AWS ECR Authentication with Temporary Tokens ( #2907 )
...
feat: add support for aws ecr authentication
Signed-off-by: K Tamil Vanan <vanan@arcesium.com >
2025-01-26 11:38:22 +02:00
f618b1d4ef
ci(deps): upgrade golangci-lint ( #2556 )
...
* ci(deps): upgrade golangci-lint
Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de >
* build(deps): removed disabled linters
Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de >
* build(deps): go run github.com/daixiang0/gci@latest write .
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* build(deps): go run golang.org/x/tools/cmd/goimports@latest -l -w .
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* build(deps): go run github.com/bombsimon/wsl/v4/cmd...@latest -strict-append -test=true -fix ./...
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* build(deps): go run github.com/catenacyber/perfsprint@latest -fix ./...
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* build(deps): replace gomnd by mnd
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* build(deps): make gqlgen
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* build: Revert "build(deps): go run github.com/daixiang0/gci@latest write ."
This reverts commit 5bf8c42e1fjoe@cloudeteer.de >
* build(deps): go run github.com/daixiang0/gci@latest write -s 'standard' -s default -s 'prefix(zotregistry.dev/zot)' .
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* build(deps): make gqlgen
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: wsl issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: check-log issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: gci issues
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
* fix: tests
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
---------
Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de >
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de >
2024-07-29 10:32:51 -07:00
6f00e843a0
fix(sync): sync generator now backs off on errors ( #2272 )
...
handle unsupported features like oci artifacts.
closes : #2238
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2024-03-04 09:44:11 -08:00
740eae8f26
fix(sync): better cleaning sync's download dir ( #2273 )
...
added cleanup in the case of copy.Image() failures.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2024-02-29 09:09:21 -08:00
8e68255946
fix(sync): added bearer client for sync ( #2222 )
...
fixed ping function taking too much time
closes : #2213 #2212
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2024-02-14 09:18:10 -08:00
6a83dd47c0
fix(scheduler): the session cleanup generator is reset too often ( #2220 )
...
This causes the "fair" scheduler to run it too often in the detriment of other generators.
The intention was to run it every 2 hours but the measurement unit for 7200 was not specified.
Add more logs, including showing a generator name, in order to troubleshoot this kind of issues easier in the future.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com >
2024-02-01 09:15:53 -08:00
ce4924f841
refactor: rename go module from zotregistry.io/zot to zotregistry.dev/zot ( #2187 )
...
Signed-off-by: Andrei Aaron <aaaron@luxoft.com >
2024-01-31 20:34:07 -08:00
79e14027ee
refactor(test): add lint rule for messages starting with the component ( #2045 )
...
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com >
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
Signed-off-by: Andrei Aaron <aaaron@luxoft.com >
2023-12-08 10:05:02 +02:00
2e733b3f4f
feat(metrics): add scheduler related metrics ( #2076 )
...
Signed-off-by: Alexei Dodon <adodon@cisco.com >
2023-12-04 14:13:50 -08:00
3c8da6e6fc
Sync s3 ( #2073 )
...
* feat(sync): local tmp store
Signed-off-by: a <a@tuxpa.in >
* fix(sync): various fixes for s3+remote storage feature
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
---------
Signed-off-by: a <a@tuxpa.in >
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
Co-authored-by: a <a@tuxpa.in >
2023-11-28 12:08:15 -08:00
59dc4c3229
feat(scheduler): pass the shutdown/reload ctx to running tasks ( #1671 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-09-05 09:48:56 -07:00
76277f5ebd
fix: remove inline GC and schedule a background task instead ( #1610 )
...
* fix: remove inline GC and set a default value of gc interval
- remove inline GC
- add a default value of GC interval
- run the GC periodically by default with the default value if no interval provided
- generate GC tasks with a random delay(0-30s) between
- add IsReady() method to scheduler.TaskGenerator interface
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
* ci: add test for gc with short interval
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
---------
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
2023-08-07 12:55:19 -07:00
d4f200c2e1
fix: changing default numWorkers, making it customizable and refactoring scheduler ( #1563 )
...
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com >
2023-07-04 11:03:29 +03:00
612a12e5a8
refactor(sync): use task scheduler ( #1301 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-05-31 10:26:23 -07:00
a3f355c278
refactor(storage): refactoring storage ( #1459 )
...
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2023-05-26 11:08:19 -07:00
c169698c95
feat: remove usage of zerolog.Logger.Msgf() from zot code ( #1382 )
...
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
2023-04-27 19:44:22 -07:00
3dd3c46ee3
test: stop task scheduler between test runs ( #1311 )
...
sync: remove sync WaitGroup, it's stopped with context
sync: onDemand will always try to sync newest image when a tag is used
if a digest is used then onDemand will serve local image
test(sync): fix flaky coverage in sync package
closes #1294
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-03-29 11:37:58 -07:00
4c156234cb
feat(repodb): sync-repodb WIP ( #1241 )
...
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2023-03-09 10:41:48 -08:00
79783b4b06
feat(sync): skip already synced images in sync ondemand ( #1234 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-03-07 09:58:42 -08:00
7c3bf86a6b
refactor: Centralise extensions config entries ( #1177 )
...
Except for registry sync config
Signed-off-by: Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com >
2023-02-15 22:20:28 -08:00
4aa0106b0a
feat(scheduler): use an worker pool for scheduler ( #1146 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2023-02-15 11:36:50 -08:00
ee95ab0ffc
fix: call notation-go libs instead of using notation binary ( #1104 )
...
fix: add loading notation path
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com >
Co-authored-by: Roxana Nemulescu <roxana.nemulescu@gmail.com >
2023-02-13 10:43:52 -08:00
14238d4a8d
fix: removed resty calls from sync ( #1016 )
...
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com >
2022-12-22 10:19:42 -08:00
024b13efe6
fix(sync): syncing OCI artifacts with distribution package fails ( #1013 )
...
sync OCI artifacts using REST APIs
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-12-09 11:38:00 -08:00
e96c80c344
feat(sync,s3): added s3 logic for ORAS and OCI artifacts ( #985 )
...
added sync logic for OCI artifacts
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-11-14 22:21:49 -08:00
2d877aaea1
fix(sync): also sync on demand digests, not only tags, closes #902 ( #932 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-27 09:39:59 -07:00
ac6c6a844c
refactor(digests): standardise representation of digests to digest.Digest ( #898 )
...
- Digests were represented by different ways
- We needed a uniform way to represent the digests and enforce a format
- also replace usage of github.com/google/go-containerregistry/pkg/v1
with github.com/opencontainers/image-spec/specs-go/v1
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
(cherry picked from commit 96b2f29d6d57070a913ce419149cd481c0723815)
(cherry picked from commit 3d41b583daea654c98378ce3dcb78937d71538e8)
Co-authored-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2022-10-22 13:46:13 -07:00
5f99f9a445
fix(sync): fixed broken logic to get tags for repo ( #900 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-22 00:26:14 -07:00
38b00e3507
chore(lint): gci to separate zot from other imports ( #870 )
...
Signed-off-by: Andrei Aaron <andaaron@cisco.com >
2022-10-20 09:39:20 -07:00
3c0c51fcbe
fix(sync): also sync image index mediatype ( #847 )
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-06 09:41:16 -07:00
c146448f01
fix(sync): revert code which removed image destination feature ( #840 )
...
Added an end to end test for this feature, closes #793
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-10-05 11:03:24 -07:00
33a431ef43
Update go version to 1.19 ( #829 )
...
* ci: Update go version to 1.19
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
* ci: Fix lint issues
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
* ci: Added needprivileges to lint, made needprivileges pass lint
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
Signed-off-by: Nicol Draghici <idraghic@cisco.com >
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
Co-authored-by: Catalin Hofnar <catalin.hofnar@gmail.com >
2022-10-05 13:21:14 +03:00
885f139e0e
Remove forking logger ( #825 )
...
- no longer needed, the race conditions were fixed
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com >
2022-09-29 13:28:39 -07:00
87fc941b3c
image level lint: enforce manifest mandatory annotations
...
closes #536
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com >
2022-07-27 11:48:04 +03:00
a8a65a6c37
Modified sync log calls to include error type ( #336 )
...
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
2022-06-15 09:45:49 -07:00
da4acaf178
sync: preserve upstream digests after syncing images
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-05-25 10:19:36 -07:00
bd730150a8
sync: allow HTTP redirects when GETing signatures blobs
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-05-25 10:19:36 -07:00
36c9631000
ext: use distribution spec route prefix for extension api
...
Following the spec defined here https://github.com/opencontainers/distribution-spec/tree/main/extensions
Signed-off-by: Shivam Mishra <shimish2@cisco.com >
2022-05-22 16:35:16 -07:00
20a60cbad4
Enhance sync logic - stop blob redownloads and re-pushes ( #479 #480 )
...
Changed imagesToCopyFromUpstream to return a map[string][]types.ImageReference from just an array of refs
Rewrote some logic in sync.go to use the new signature of imagesToCopyFromUpstream
Split getLocalImageRef by adding function getLocalCachePath
Adapted tests for new changes, added some tests
Merged #481
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com >
2022-05-16 10:05:01 -07:00
d325c8b5f4
Fix problems signaled by new linter version v1.45.2
...
PR (linter: upgrade linter version #405 ) triggered lint job which failed
with many errors generated by various linters. Configurations were added to
golangcilint.yaml and several refactorings were made in order to improve the
results of the linter.
maintidx linter disabled
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro >
2022-04-27 09:55:44 -07:00
f53dc9eb8d
sync: Add a new flag to enforce syncing only signed images, closes #455
...
sync: When checking if a image is already synced also check for changes in upstream signatures.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-03-24 10:50:01 -07:00
0d148e1d6b
new config option for sync-destination
...
Signed-off-by: laurentiuNiculae <themelopeus@gmail.com >
2022-03-21 08:12:34 -07:00
6d04ab3cdc
sync: support reloading sync config when the config file changes
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-03-18 11:24:11 -07:00
45968e0bb7
sync: fix inconsistent test, used inject error fw for hard to reach test cases
...
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com >
2022-03-01 09:56:27 -08:00
Vishwas Rajashekar