mirror of
https://github.com/project-zot/zot.git
synced 2026-06-17 12:58:02 +08:00
95ce68ccb0
* refactor(test/blackbox): extract shared push/pull helpers Move duplicated push/pull/regclient/oras/helm helper functions out of fips140.bats, pushpull.bats and helpers_upgrade.bash into a single helpers_pushpull.bash, then have all three suites load from there. The helpers now share verify_prerequisites, get_zot_port, common assertion helpers (catalog/tag presence, OCI index ref name) and the regclient pagination listing. helpers_upgrade.bash keeps only the test_release_* and test_new_* wrappers that compose those helpers. Net effect: ~1000 lines of duplicated test scaffolding removed across the four files; behavior of the existing test cases is preserved. Refs: #3727 Signed-off-by: Akash Kumar <meakash7902@gmail.com> * refactor(test/blackbox): share pushpull lifecycle and dedupe authn helpers Build on the shared helpers_pushpull.bash to remove more duplicated blackbox scaffolding: - Add pushpull_setup_file/pushpull_teardown/pushpull_teardown_file keyed on PUSHPULL_FIPS_MODE so pushpull.bats and fips140.bats only set the flag and delegate lifecycle, instead of each carrying a near-identical setup_file/teardown. - Add helpers_pushpull_authn.bash (loaded by pushpull_authn.bats and fips140_authn.bats) for shared htpasswd setup, FIPS vs non-FIPS config and teardown, and the regclient/OCI/ML test helpers; both authn suites collapse to one-line @test bodies keyed on PUSHPULL_AUTHN_FIPS_MODE. - Make each authn helper self-sufficient by performing its own regctl login, removing the hidden dependency on the first test having logged in. - Split the implicit manifest delete out of helper_pull_image_index_and_delete into a standalone helper_delete_manifest, surfaced as its own "delete image index" @test in the pushpull, fips140 and upgrade suites, so the delete is explicit and no longer an order-fragile side effect of a pull. Refs: #3727 Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test/blackbox): harden flaky oras pull and sync signature tests Set org.opencontainers.image.title on oras artifact push and verify pull both via oras pull and manifest/blob fetch. Add retry_until_success and use it for periodic notation/cosign signature sync checks on slow CI. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test/blackbox): address Copilot review feedback in push/pull helpers Use curl --fail for manifest deletes so HTTP errors fail the test. Remove the duplicate regctl --format flag in helper_push_manifest_with_regclient. Harden helper_authn_ml_artifacts with run and status checks, using a binary-safe shell redirect for the ONNX artifact round-trip. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test/blackbox): isolate pushpull helper temp files and prerequisites Write oras artifact and docker build files under BATS_TEST_TMPDIR instead of the test working directory, and check git/docker in verify_prerequisites for clearer failures when running bats directly. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test): use verify_prerequisites exit status in bats setup Replace `$(verify_prerequisites)` with a direct call across blackbox and scale-out suites. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test/blackbox): unify blackbox log path to zot/zot-log.json Drop the FIPS vs non-FIPS split between zot-log.json and zot.log in pushpull, authn, and upgrade helpers. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * refactor(test/blackbox): loop over tools in verify_prerequisites Replace repeated command -v checks with a single loop over curl, jq, git, and docker. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * refactor(test/blackbox): fix exit code for retry_until_success Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * refactor(test/blackbox): inline upgrade tests and drop helpers_upgrade Call helper_* directly from upgrade.bats and upgrade_minimal.bats, load helpers_pushpull in those suites, and move teardown there. Remove helpers_upgrade.bash now that the release/new wrappers are gone. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * refactor(test/blackbox): require explicit args on pushpull helpers Drop parameter defaults from shared push/pull helpers and pass image, repository, and pagination values explicitly at each call site. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test/blackbox): clarify docker push/pull negative test naming Rename helper_push_docker_image to helper_build_docker_image_push_and_pull and update test titles to reflect build plus expected push/pull failures without the docker compatibility extension. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test/blackbox): isolate regctl config per BATS suite regctl persists login and TLS settings on disk, so push/pull blackbox tests could leak state into ~/.regctl/config.json across suites. Point REGCTL_CONFIG at BATS_FILE_TMPDIR for pushpull, authn, and upgrade suites, configure TLS once in authn setup, and drop redundant logout and per-login TLS setup. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> * fix(test/blackbox): harden upgrade suite log dump on failure Touch zot-log.json during upgrade setup and only cat it in teardown when the file exists, so a missing log cannot mask the underlying test failure. Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> --------- Signed-off-by: Akash Kumar <meakash7902@gmail.com> Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com> Co-authored-by: Akash Kumar <meakash7902@gmail.com>
408 lines
12 KiB
Bash
Executable File
408 lines
12 KiB
Bash
Executable File
# Note: Intended to be run as "make run-blackbox-tests" or "make run-blackbox-ci"
|
|
# Makefile target installs & checks all necessary tooling
|
|
# Extra tools that are not covered in Makefile target needs to be added in verify_prerequisites()
|
|
|
|
load helpers_zot
|
|
load helpers_wait
|
|
load ../port_helper
|
|
|
|
function verify_prerequisites {
|
|
if [ ! $(command -v curl) ]; then
|
|
echo "you need to install curl as a prerequisite to running the tests" >&3
|
|
return 1
|
|
fi
|
|
|
|
if [ ! $(command -v jq) ]; then
|
|
echo "you need to install jq as a prerequisite to running the tests" >&3
|
|
return 1
|
|
fi
|
|
|
|
if [ ! $(command -v docker) ]; then
|
|
echo "you need to install docker as a prerequisite to running the tests" >&3
|
|
return 1
|
|
fi
|
|
|
|
return 0
|
|
}
|
|
|
|
function assert_oidc_extract_identity_log() {
|
|
local log_file=${1:?}
|
|
local expect_fell_back=${2:?}
|
|
local expect_claim=${3:?}
|
|
|
|
grep '"message":"extracted identity"' "${log_file}" \
|
|
| grep "\"claim\":\"${expect_claim}\"" \
|
|
| grep "\"fellBackToDefaultClaim\":${expect_fell_back}" \
|
|
| grep -q .
|
|
}
|
|
|
|
IDP_PID=""
|
|
|
|
function setup_file() {
|
|
# Verify prerequisites are available
|
|
if ! verify_prerequisites; then
|
|
exit 1
|
|
fi
|
|
|
|
# Pre-allocate all ports we'll need
|
|
idp_port=$(get_free_port_for_service "idp")
|
|
echo ${idp_port} > ${BATS_FILE_TMPDIR}/idp.port
|
|
|
|
# Setup oidc provider
|
|
npm link oidc-provider express
|
|
node ${ROOT_DIR}/test/scripts/panva-idp.js ${idp_port} > ${BATS_FILE_TMPDIR}/idp.log 2>&1 &
|
|
IDP_PID=$!
|
|
|
|
# Wait for oidc provider to be ready
|
|
echo "Waiting for idp to be ready on port ${idp_port}..." >&3
|
|
local idp_url=http://127.0.0.1:${idp_port}/.well-known/openid-configuration
|
|
local max_attempts=60
|
|
local attempt=0
|
|
while [ $attempt -lt $max_attempts ]; do
|
|
if curl -f -s -o /dev/null ${idp_url} 2>&1; then
|
|
echo "oidc idp is ready!" >&3
|
|
break
|
|
fi
|
|
attempt=$((attempt + 1))
|
|
sleep 2
|
|
done
|
|
|
|
if [ $attempt -eq $max_attempts ]; then
|
|
echo "idp failed to start within timeout" >&3
|
|
return 1
|
|
fi
|
|
}
|
|
|
|
function teardown_file() {
|
|
zot_stop_all
|
|
kill ${IDP_PID} 2>/dev/null || true
|
|
if [ -f "${BATS_FILE_TMPDIR}/idp.log" ]; then
|
|
echo "--- IDP LOG ---"
|
|
cat ${BATS_FILE_TMPDIR}/idp.log
|
|
echo "--- END IDP LOG ---"
|
|
fi
|
|
}
|
|
|
|
function teardown() {
|
|
# conditionally printing on failure is possible from teardown but not from teardown_file
|
|
if [ -f "${BATS_FILE_TMPDIR}/zot-preferred-username.log" ]; then
|
|
cat ${BATS_FILE_TMPDIR}/zot-preferred-username.log
|
|
fi
|
|
if [ -f "${BATS_FILE_TMPDIR}/zot-email.log" ]; then
|
|
cat ${BATS_FILE_TMPDIR}/zot-email.log
|
|
fi
|
|
if [ -f "${BATS_FILE_TMPDIR}/zot-sub.log" ]; then
|
|
cat ${BATS_FILE_TMPDIR}/zot-sub.log
|
|
fi
|
|
if [ -f "${BATS_FILE_TMPDIR}/zot-default.log" ]; then
|
|
cat ${BATS_FILE_TMPDIR}/zot-default.log
|
|
fi
|
|
if [ -f "${BATS_FILE_TMPDIR}/idp.log" ]; then
|
|
echo "--- IDP LOG (teardown) ---"
|
|
cat ${BATS_FILE_TMPDIR}/idp.log
|
|
echo "--- END IDP LOG ---"
|
|
fi
|
|
}
|
|
|
|
function idp_session() {
|
|
local zot_port=${1}
|
|
local idp_port=$(cat ${BATS_FILE_TMPDIR}/idp.port)
|
|
|
|
# With the auto-login enabled IDP, we just need to hit the login endpoint and follow redirects.
|
|
# We use a cookie jar to handle the session cookies during the redirects.
|
|
curl -v -L -c ${BATS_FILE_TMPDIR}/cookies "http://127.0.0.1:${zot_port}/zot/auth/login?provider=oidc"
|
|
}
|
|
|
|
@test "test OIDC claim mapping with preferred_username" {
|
|
local zot_root_dir=${BATS_FILE_TMPDIR}/zot-preferred-username
|
|
local zot_config_file=${BATS_FILE_TMPDIR}/zot_config_preferred_username.json
|
|
local zot_credentials_file=${BATS_FILE_TMPDIR}/zot_credentials_preferred_username.json
|
|
local idp_port=$(cat ${BATS_FILE_TMPDIR}/idp.port)
|
|
zot_port=$(get_free_port_for_service "zot_preferred_username")
|
|
|
|
mkdir -p ${zot_root_dir}
|
|
|
|
cat > ${zot_credentials_file}<<EOF
|
|
{
|
|
"clientid": "zot-client",
|
|
"clientsecret": "ZXhhbXBsZS1hcHAtc2VjcmV0"
|
|
}
|
|
EOF
|
|
|
|
cat > ${zot_config_file}<<EOF
|
|
{
|
|
"distSpecVersion": "1.1.1",
|
|
"storage": {
|
|
"rootDirectory": "${zot_root_dir}",
|
|
"dedupe": true
|
|
},
|
|
"http": {
|
|
"address": "127.0.0.1",
|
|
"port": "${zot_port}",
|
|
"realm": "zot",
|
|
"auth": {
|
|
"openid": {
|
|
"providers": {
|
|
"oidc": {
|
|
"name": "panva",
|
|
"issuer": "http://127.0.0.1:${idp_port}/",
|
|
"credentialsFile": "${zot_credentials_file}",
|
|
"scopes": ["openid", "profile", "email", "groups"],
|
|
"claimMapping": {
|
|
"username": "preferred_username"
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"failDelay": 5
|
|
},
|
|
"accessControl": {
|
|
"repositories": {
|
|
"**": {
|
|
"policies": [
|
|
{
|
|
"users": ["alice"],
|
|
"actions": ["read", "create", "update", "delete"]
|
|
}
|
|
],
|
|
"defaultPolicy": ["read"]
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"log": {
|
|
"level": "debug",
|
|
"output": "${BATS_FILE_TMPDIR}/zot-preferred-username.log"
|
|
}
|
|
}
|
|
EOF
|
|
|
|
zot_serve ${ZOT_PATH} ${zot_config_file}
|
|
wait_zot_reachable ${zot_port}
|
|
|
|
run idp_session ${zot_port}
|
|
[ "$status" -eq 0 ]
|
|
echo "$output"
|
|
|
|
# Verify structured extract identity debug line (preferred_username mapped)
|
|
assert_oidc_extract_identity_log ${BATS_FILE_TMPDIR}/zot-preferred-username.log false preferred_username
|
|
}
|
|
|
|
@test "test OIDC claim mapping with email" {
|
|
local zot_root_dir=${BATS_FILE_TMPDIR}/zot-email
|
|
local zot_config_file=${BATS_FILE_TMPDIR}/zot_config_email.json
|
|
local zot_credentials_file=${BATS_FILE_TMPDIR}/zot_credentials_email.json
|
|
local idp_port=$(cat ${BATS_FILE_TMPDIR}/idp.port)
|
|
zot_port=$(get_free_port_for_service "zot_email")
|
|
|
|
mkdir -p ${zot_root_dir}
|
|
|
|
cat > ${zot_credentials_file}<<EOF
|
|
{
|
|
"clientid": "zot-client",
|
|
"clientsecret": "ZXhhbXBsZS1hcHAtc2VjcmV0"
|
|
}
|
|
EOF
|
|
|
|
cat > ${zot_config_file}<<EOF
|
|
{
|
|
"distSpecVersion": "1.1.1",
|
|
"storage": {
|
|
"rootDirectory": "${zot_root_dir}",
|
|
"dedupe": true
|
|
},
|
|
"http": {
|
|
"address": "127.0.0.1",
|
|
"port": "${zot_port}",
|
|
"realm": "zot",
|
|
"auth": {
|
|
"openid": {
|
|
"providers": {
|
|
"oidc": {
|
|
"name": "panva",
|
|
"issuer": "http://127.0.0.1:${idp_port}/",
|
|
"credentialsFile": "${zot_credentials_file}",
|
|
"scopes": ["openid", "profile", "email", "groups"],
|
|
"claimMapping": {
|
|
"username": "email"
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"failDelay": 5
|
|
},
|
|
"accessControl": {
|
|
"repositories": {
|
|
"**": {
|
|
"policies": [
|
|
{
|
|
"users": ["alice@example.com"],
|
|
"actions": ["read", "create", "update", "delete"]
|
|
}
|
|
],
|
|
"defaultPolicy": ["read"]
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"log": {
|
|
"level": "debug",
|
|
"output": "${BATS_FILE_TMPDIR}/zot-email.log"
|
|
}
|
|
}
|
|
EOF
|
|
|
|
zot_serve ${ZOT_PATH} ${zot_config_file}
|
|
wait_zot_reachable ${zot_port}
|
|
|
|
run idp_session ${zot_port}
|
|
[ "$status" -eq 0 ]
|
|
echo "$output"
|
|
|
|
# Verify structured extract identity debug line (explicit email username claim)
|
|
assert_oidc_extract_identity_log ${BATS_FILE_TMPDIR}/zot-email.log false email
|
|
}
|
|
|
|
@test "test OIDC claim mapping with sub" {
|
|
local zot_root_dir=${BATS_FILE_TMPDIR}/zot-sub
|
|
local zot_config_file=${BATS_FILE_TMPDIR}/zot_config_sub.json
|
|
local zot_credentials_file=${BATS_FILE_TMPDIR}/zot_credentials_sub.json
|
|
local idp_port=$(cat ${BATS_FILE_TMPDIR}/idp.port)
|
|
zot_port=$(get_free_port_for_service "zot_sub")
|
|
|
|
mkdir -p ${zot_root_dir}
|
|
|
|
cat > ${zot_credentials_file}<<EOF
|
|
{
|
|
"clientid": "zot-client",
|
|
"clientsecret": "ZXhhbXBsZS1hcHAtc2VjcmV0"
|
|
}
|
|
EOF
|
|
|
|
cat > ${zot_config_file}<<EOF
|
|
{
|
|
"distSpecVersion": "1.1.1",
|
|
"storage": {
|
|
"rootDirectory": "${zot_root_dir}",
|
|
"dedupe": true
|
|
},
|
|
"http": {
|
|
"address": "127.0.0.1",
|
|
"port": "${zot_port}",
|
|
"realm": "zot",
|
|
"auth": {
|
|
"openid": {
|
|
"providers": {
|
|
"oidc": {
|
|
"name": "panva",
|
|
"issuer": "http://127.0.0.1:${idp_port}/",
|
|
"credentialsFile": "${zot_credentials_file}",
|
|
"scopes": ["openid", "profile", "email", "groups"],
|
|
"claimMapping": {
|
|
"username": "sub"
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"failDelay": 5
|
|
},
|
|
"accessControl": {
|
|
"repositories": {
|
|
"**": {
|
|
"defaultPolicy": ["read"]
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"log": {
|
|
"level": "debug",
|
|
"output": "${BATS_FILE_TMPDIR}/zot-sub.log"
|
|
}
|
|
}
|
|
EOF
|
|
|
|
zot_serve ${ZOT_PATH} ${zot_config_file}
|
|
wait_zot_reachable ${zot_port}
|
|
|
|
run idp_session ${zot_port}
|
|
[ "$status" -eq 0 ]
|
|
echo "$output"
|
|
|
|
# Verify structured extract identity debug line (sub mapped)
|
|
assert_oidc_extract_identity_log ${BATS_FILE_TMPDIR}/zot-sub.log false sub
|
|
}
|
|
|
|
@test "test OIDC with no claim mapping (default to email)" {
|
|
local zot_root_dir=${BATS_FILE_TMPDIR}/zot-default
|
|
local zot_config_file=${BATS_FILE_TMPDIR}/zot_config_default.json
|
|
ZOT_LOG_FILE=${BATS_FILE_TMPDIR}/zot-default.log
|
|
local zot_credentials_file=${BATS_FILE_TMPDIR}/zot_credentials_default.json
|
|
local idp_port=$(cat ${BATS_FILE_TMPDIR}/idp.port)
|
|
zot_port=$(get_free_port_for_service "zot_default")
|
|
|
|
mkdir -p ${zot_root_dir}
|
|
|
|
cat > ${zot_credentials_file}<<EOF
|
|
{
|
|
"clientid": "zot-client",
|
|
"clientsecret": "ZXhhbXBsZS1hcHAtc2VjcmV0"
|
|
}
|
|
EOF
|
|
|
|
# Note: No claimMapping section - should default to email
|
|
cat > ${zot_config_file}<<EOF
|
|
{
|
|
"distSpecVersion": "1.1.1",
|
|
"storage": {
|
|
"rootDirectory": "${zot_root_dir}",
|
|
"dedupe": true
|
|
},
|
|
"http": {
|
|
"address": "127.0.0.1",
|
|
"port": "${zot_port}",
|
|
"realm": "zot",
|
|
"auth": {
|
|
"openid": {
|
|
"providers": {
|
|
"oidc": {
|
|
"name": "panva",
|
|
"issuer": "http://127.0.0.1:${idp_port}/",
|
|
"credentialsFile": "${zot_credentials_file}",
|
|
"scopes": ["openid", "profile", "email", "groups"]
|
|
}
|
|
}
|
|
},
|
|
"failDelay": 5
|
|
},
|
|
"accessControl": {
|
|
"repositories": {
|
|
"**": {
|
|
"policies": [
|
|
{
|
|
"users": ["alice@example.com"],
|
|
"actions": ["read", "create", "update", "delete"]
|
|
}
|
|
],
|
|
"defaultPolicy": ["read"]
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"log": {
|
|
"level": "debug",
|
|
"output": "${ZOT_LOG_FILE}"
|
|
}
|
|
}
|
|
EOF
|
|
|
|
zot_serve ${ZOT_PATH} ${zot_config_file}
|
|
wait_zot_reachable ${zot_port}
|
|
|
|
run idp_session ${zot_port}
|
|
[ "$status" -eq 0 ]
|
|
echo "$output"
|
|
|
|
# Verify structured extract identity debug line (default email claim, no claimMapping)
|
|
assert_oidc_extract_identity_log ${ZOT_LOG_FILE} false email
|
|
}
|