Commit Graph

161 Commits

Author SHA1 Message Date
Tanmay Naik ad684ac44b cli: add config and images command
Extends the existing zot CLI to add commands for listing all images and
their details on a zot server.
Listing all images introduces the need for configurations.

Each configuration has a name and URL at the least. Check 'zot config
-h' for more details.

The user can specify the URL of zot server explicitly while running the
command or configure a URL and pass it directly.

Adding a configuration:
zot config add aci-zot <zot-url>

Run 'zot config --help' for more.

Listing all images:
zot images --url <zot-url>

Pass a config instead of the url:
zot images <config-name>

Filter the list of images by image name:
zot images <config-name> --name <image-name>

Run 'zot images --help' for all details

- Stores configurations in '$HOME/.zot' file

Add CLI README
2020-07-02 14:30:35 -04:00
Ramkumar Chinchani 4a1519bb1d Merge pull request #107 from shimish2/BuildPipelineUpgrade
Upgraded build pipeline
2020-06-26 15:20:13 -07:00
Shivam Mishra af77876306 Upgraded build pipeline
Go version changed to 1.14.4
Golangci-lint changed to 1.26.0
Bazel version changed to 3.0.0
Bazel rules_go version changed to 0.23.3
Bazel gazelle version changed to v0.21.0
Bazel build tools version changed to 0.25.1
Bazel skylib version changed to 1.0.2
2020-06-25 23:43:31 -07:00
Ramkumar Chinchani ff4a300057 Merge pull request #109 from shimish2/OpensuseChange
Changed umoci import path
2020-06-25 17:49:26 -07:00
Shivam Mishra 85d3e1db4b Changed umoci import path 2020-06-25 17:04:32 -07:00
Ramkumar Chinchani 3dc9885ee9 update the size field when existing manifest entry is updated
An existing manifest descriptor in index.json can be updated with
different manifest contents for the same/existing tag. We were updating
the digest but not the size field causing GC to report an error.

Add a unit test case to cover this.

Add logs.
v1.1.1
2020-06-18 16:20:43 -07:00
Ramkumar Chinchani c374f9dbcb Merge pull request #98 from tsnaik/fix-htpasswd
fix: the bug when htpasswd has multiple creds
2020-06-18 15:57:25 -07:00
Tanmay Naik 3f3f7e3f8c tests: add better tests for 3cfb2b3 2020-06-17 20:17:49 -04:00
Tanmay Naik 904ae763d7 tests: add unit tests for fix 3cfb2b3 2020-06-09 19:18:33 -04:00
Tanmay Naik 3cfb2b30a6 fix: the bug when htpasswd has multiple creds
earlier, when you had more than one creds in htpasswd file separated by
newline, it used to only read the first cred in the file and ignore the
rest.
2020-06-09 17:19:01 -04:00
Tanmay Naik d16cbb0b10 .gitignore: add .vscode/ 2020-06-09 17:18:30 -04:00
Ramkumar Chinchani dbb0a2d57a Merge pull request #93 from rchincha/d2
dedupe: record relative path for cache entries
2020-05-28 12:16:54 -07:00
Ramkumar Chinchani cb9e773a3e dedupe: record relative path for cache entries
In a production use case we found that the actual rootdir can be moved.
Currently, cache entries for dedupe record the full blob path which
doesn't work in the move use case.

Only for dedupe cache entries, record relative blob paths.
2020-05-27 22:11:26 -07:00
Ramkumar Chinchani 36efa17915 Merge pull request #91 from rchamarthy/fix/conformance
fix the conformance link
2020-05-20 11:53:42 -07:00
Ravi Chamarthy 7caf748341 fix the conformance link
Signed-off-by: Ravi Chamarthy <ravchama@cisco.com>
2020-05-20 11:37:54 -07:00
Ramkumar Chinchani afa52bc54d Merge pull request #90 from rchincha/helm
helm: relax Content-Type checks
v1.1.0
2020-05-19 17:27:23 -07:00
Ramkumar Chinchani e83999ae0d helm: relax Content-Type checks
OCI registries are moving towards a more generic artifact storage
mechanism. In short-term, at the very least support helm charts.
2020-05-19 16:53:05 -07:00
Tycho Andersen 6017be5bfd Merge pull request #89 from rchincha/client-compat
compat: when in "world-readable" mode, return the WWW-Authenticate
2020-05-19 15:48:34 -06:00
Ramkumar Chinchani 026b009dbb compat: when in "world-readable" mode, return the WWW-Authenticate
header

containers/image is the dominant client library to interact with
registries.

It detects which authentication to use based on the WWW-Authenticate
header returned when pinging "/v2/" end-point. If we didn't return this
header, then creds are not used for other write-protected end-points.
Hence, the compatibility fix.
2020-05-19 13:54:45 -07:00
Tycho Andersen b2338b4819 Merge pull request #83 from rchincha/compl
conformance: align with upstream conformance tests
2020-04-16 17:41:51 -06:00
Ramkumar Chinchani dd1fc1e866 config: add gc and dedupe as configurable params (default to enabled)
Since we want to conform to dist-spec, sometimes the gc and dedupe
optimizations conflict with the conformance tests that are being run.
So allow them to be turned off via configuration params.
2020-04-16 16:01:53 -07:00
Ramkumar Chinchani b1f882e1b8 conformance: align with upstream conformance tests
Upstream conformance tests are being updated, so we need to align along
with our internal GC and dedupe features.

Add a new example config file which plays nice with conformance tests.

DeleteImageManifest() updated to deal with the case where the same
manifest can be created with multiple tags and deleted with the same
digest - so all entries must be deleted.

DeleteBlob() delete the digest key (bucket) when last reference is
dropped
2020-04-16 16:01:53 -07:00
Serge Hallyn ea6018b168 Merge pull request #76 from rchincha/dedupe
dedupe: use hard links to dedupe blobs
2020-04-03 13:28:16 -05:00
Ramkumar Chinchani 25f5a45296 dedupe: use hard links to dedupe blobs
As the number of repos and layers increases, the greater the probability
that layers are duplicated. We dedupe using hard links when content is
the same. This is intended to be purely a storage layer optimization.
Access control when available is orthogonal this optimization.

Add a durable cache to help speed up layer lookups.

Update README.

Add more unit tests.
2020-04-03 09:29:12 -07:00
Serge Hallyn 365145d2cd Merge pull request #81 from rchincha/ldap
ldap: prune unused code
2020-03-31 22:00:19 -05:00
Ramkumar Chinchani 9fa185f2bb ldap: prune unused code
We don't use this method. Remove it so CI/CD coverage is better
reported.
2020-03-30 23:12:16 -07:00
Ramkumar Chinchani 9ecb5cedf3 Merge pull request #80 from rchincha/compl
conformance: fix error msg for DELETE MANIFEST
2020-03-25 14:06:54 -07:00
Ramkumar Chinchani b141d4ff9b lint: relax agressive linter settings 2020-03-25 13:21:55 -07:00
Ramkumar Chinchani 8ff60f9138 conformance: fix error msg for DELETE MANIFEST
---
Ran 27 of 27 Specs in 0.120 seconds
SUCCESS! -- 27 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
---
2020-03-25 12:53:15 -07:00
Tycho Andersen bbed38cfa0 Merge pull request #79 from rchincha/tycho-run-gc-after-manifest-removal
Tycho run gc after manifest removal
2020-03-20 14:53:22 -06:00
Ramkumar Chinchani dfe023225f pkg/api: use a rwlock when accessing storage
The original patch used a mutex, however, the workload patterns are
likely to be read-heavy, so use a rwlock instead.
2020-03-20 12:23:36 -07:00
Ramkumar Chinchani 2fd87b6a86 pkg/api: use a rwlock when accessing storage
The original patch used a mutex, however, the workload patterns are
likely to be read-heavy, so use a rwlock instead.
2020-03-20 10:58:21 -07:00
Ramkumar Chinchani fe471a3c35 gc: fix test cases since umoci GC is more strict
umoci GC enforces a valid index.json and current tests were a little
lax.
2020-03-20 10:58:21 -07:00
Tycho Andersen 4774aa81b3 compliance: don't leak test directory
Signed-off-by: Tycho Andersen <tycho@tycho.ws>
2020-03-20 10:58:21 -07:00
Ramkumar Chinchani 404c83fbc1 bazel: fix bazel build failures 2020-03-20 10:58:21 -07:00
Tycho Andersen 94afc30661 routes: lock everything that modifies blobs
Now that we're GCing blobs on delete/update manifest, we should lock the
blob queries so that they don't race with each other.

This is a pretty coarse grained lock, there's probably a better way to do
this.

Signed-off-by: Tycho Andersen <tycho@tycho.ws>
2020-03-20 10:58:21 -07:00
Tycho Andersen 95d4a7ce04 zot: run GC after manifest removal
Clients today expect the repo to clean up if there are unused blobs, not to
manually delete things they think are unused. Let's do that, and use
umoci's code to do it since it's tested and works.

v2: also run GC on update as well as delete

v3: fix up error return paths needing two args

Signed-off-by: Serge Hallyn <shallyn@cisco.com>
Signed-off-by: Tycho Andersen <tycho@tycho.ws>
2020-03-20 10:58:21 -07:00
Tycho Andersen 3349e8cb71 Merge pull request #78 from rchincha/compl
conformance: fix http status codes for MANIFEST DELETE failures.
2020-03-18 17:07:07 -06:00
Ramkumar Chinchani 941dffe2c7 conformance: fix http status codes for MANIFEST DELETE failures.
Previously returning 404s as failure code, dist-spec says 400s.
2020-03-18 15:50:40 -07:00
Serge Hallyn 355a3b4f0e Merge pull request #75 from rchincha/issue-74
routes: CheckManifest should return 404 when repo is unknown
v1.0.1
2020-02-13 13:44:26 -06:00
Ramkumar Chinchani 7287700ceb coverage: fix coverage to track packages better 2020-02-13 11:22:04 -08:00
Ramkumar Chinchani 6e494942d4 routes: CheckManifest should return 404 when repo is unknown
Previously, CheckManifest() was not checking for repo not found
condition and would default to 500 status code.

Add the check now to return 404.

Fixes issue #74
2020-02-13 11:00:11 -08:00
Ramkumar Chinchani 910e2730fe Merge pull request #72 from bloodorangeio/auth-fix
Fix auth scope on endpoints without repo name
2020-02-04 09:56:26 -08:00
Peter Engelbert b636ce2da1 Fix auth scope on endpoints without repo name
Resolves #71

Signed-off-by: Peter Engelbert <pmengelbert@gmail.com>
2020-01-31 18:04:38 -06:00
Ramkumar Chinchani 92241d17cb Merge pull request #70 from rchincha/compl
fix compliance
2020-01-31 13:47:35 -08:00
Ramkumar Chinchani 58040f4562 check: add unit tests to cover the new code, fix linter errors 2020-01-31 13:21:43 -08:00
Ramkumar Chinchani f9a1a0fe48 routes: handle compliance requirements
- that errors be returned a certain way using the new NewErrorList()
  method and the string enum constants

- allow for full blob upload without a session with repo name and digest
2020-01-31 11:49:15 -08:00
Ramkumar Chinchani d9fcf713ca auth: compliance requires error codes be returned a certain way
use the new NewErrorList() method and the enum constants as strings
2020-01-31 11:49:15 -08:00
Ramkumar Chinchani 909a97b922 storage: compliance allows for a full blob upload without a session
implement a new method which just takes the repo name, body and digest
and creates a blob out of this
2020-01-31 11:49:15 -08:00
Ramkumar Chinchani 48fb4967a2 errors: compliance requires error codes to be string enum constants. 2020-01-30 23:54:05 -08:00