Files
zot/test/blackbox/helpers_pushpull_authn.bash
T
Andrei Aaron 95ce68ccb0 refactor(test/blackbox): extract shared push/pull helpers (#4132)
* refactor(test/blackbox): extract shared push/pull helpers

Move duplicated push/pull/regclient/oras/helm helper functions out of
fips140.bats, pushpull.bats and helpers_upgrade.bash into a single
helpers_pushpull.bash, then have all three suites load from there.

The helpers now share verify_prerequisites, get_zot_port, common assertion
helpers (catalog/tag presence, OCI index ref name) and the regclient
pagination listing. helpers_upgrade.bash keeps only the test_release_*
and test_new_* wrappers that compose those helpers.

Net effect: ~1000 lines of duplicated test scaffolding removed across
the four files; behavior of the existing test cases is preserved.

Refs: #3727
Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* refactor(test/blackbox): share pushpull lifecycle and dedupe authn helpers

Build on the shared helpers_pushpull.bash to remove more duplicated
blackbox scaffolding:

- Add pushpull_setup_file/pushpull_teardown/pushpull_teardown_file keyed
  on PUSHPULL_FIPS_MODE so pushpull.bats and fips140.bats only set the
  flag and delegate lifecycle, instead of each carrying a near-identical
  setup_file/teardown.
- Add helpers_pushpull_authn.bash (loaded by pushpull_authn.bats and
  fips140_authn.bats) for shared htpasswd setup, FIPS vs non-FIPS config
  and teardown, and the regclient/OCI/ML test helpers; both authn suites
  collapse to one-line @test bodies keyed on PUSHPULL_AUTHN_FIPS_MODE.
- Make each authn helper self-sufficient by performing its own regctl
  login, removing the hidden dependency on the first test having logged
  in.
- Split the implicit manifest delete out of helper_pull_image_index_and_delete
  into a standalone helper_delete_manifest, surfaced as its own
  "delete image index" @test in the pushpull, fips140 and upgrade suites,
  so the delete is explicit and no longer an order-fragile side effect of
  a pull.

Refs: #3727
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test/blackbox): harden flaky oras pull and sync signature tests

Set org.opencontainers.image.title on oras artifact push and verify pull
both via oras pull and manifest/blob fetch. Add retry_until_success and
use it for periodic notation/cosign signature sync checks on slow CI.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test/blackbox): address Copilot review feedback in push/pull helpers

Use curl --fail for manifest deletes so HTTP errors fail the test. Remove
the duplicate regctl --format flag in helper_push_manifest_with_regclient.
Harden helper_authn_ml_artifacts with run and status checks, using a
binary-safe shell redirect for the ONNX artifact round-trip.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test/blackbox): isolate pushpull helper temp files and prerequisites

Write oras artifact and docker build files under BATS_TEST_TMPDIR
instead of the test working directory, and check git/docker in
verify_prerequisites for clearer failures when running bats directly.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test): use verify_prerequisites exit status in bats setup

Replace `$(verify_prerequisites)` with a direct call across blackbox
and scale-out suites.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test/blackbox): unify blackbox log path to zot/zot-log.json

Drop the FIPS vs non-FIPS split between zot-log.json and zot.log in
pushpull, authn, and upgrade helpers.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* refactor(test/blackbox): loop over tools in verify_prerequisites

Replace repeated command -v checks with a single loop over curl, jq,
git, and docker.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* refactor(test/blackbox): fix exit code for retry_until_success

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* refactor(test/blackbox): inline upgrade tests and drop helpers_upgrade

Call helper_* directly from upgrade.bats and upgrade_minimal.bats,
load helpers_pushpull in those suites, and move teardown there. Remove
helpers_upgrade.bash now that the release/new wrappers are gone.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* refactor(test/blackbox): require explicit args on pushpull helpers

Drop parameter defaults from shared push/pull helpers and pass image,
repository, and pagination values explicitly at each call site.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test/blackbox): clarify docker push/pull negative test naming

Rename helper_push_docker_image to helper_build_docker_image_push_and_pull
and update test titles to reflect build plus expected push/pull failures
without the docker compatibility extension.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test/blackbox): isolate regctl config per BATS suite

regctl persists login and TLS settings on disk, so push/pull blackbox
tests could leak state into ~/.regctl/config.json across suites. Point
REGCTL_CONFIG at BATS_FILE_TMPDIR for pushpull, authn, and upgrade
suites, configure TLS once in authn setup, and drop redundant logout
and per-login TLS setup.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

* fix(test/blackbox): harden upgrade suite log dump on failure

Touch zot-log.json during upgrade setup and only cat it in teardown when
the file exists, so a missing log cannot mask the underlying test failure.

Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>

---------

Signed-off-by: Akash Kumar <meakash7902@gmail.com>
Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
Co-authored-by: Akash Kumar <meakash7902@gmail.com>
2026-06-15 10:56:11 +03:00

317 lines
9.4 KiB
Bash

# Common helper functions for authenticated push/pull blackbox tests.
# Used by pushpull_authn.bats and fips140_authn.bats.
load helpers_pushpull
function verify_authn_prerequisites() {
if ! verify_prerequisites; then
return 1
fi
if ! command -v htpasswd >/dev/null; then
echo "you need to install htpasswd as a prerequisite to running the tests" >&3
return 1
fi
if [ "${PUSHPULL_AUTHN_FIPS_MODE:-0}" = 1 ]; then
if ! command -v mkpasswd >/dev/null; then
echo "you need to install mkpasswd as a prerequisite to running the tests" >&3
return 1
fi
fi
return 0
}
function authn_write_htpasswd_file() {
local htpasswd_file=${1}
htpasswd -Bbn "${AUTH_USER}" "${AUTH_PASS}" >>"${htpasswd_file}"
if [ "${PUSHPULL_AUTHN_FIPS_MODE:-0}" = 1 ]; then
echo "${AUTH_USER2}:$(echo "${AUTH_PASS2}" | mkpasswd -s -R 1 -m sha-256)" >>"${htpasswd_file}"
echo "${AUTH_USER3}:$(echo "${AUTH_PASS3}" | mkpasswd -s -R 1 -m sha-512)" >>"${htpasswd_file}"
echo "${AUTH_USER4}:$(echo "${AUTH_PASS4}" | mkpasswd -s -R 0 -m sha-256)" >>"${htpasswd_file}"
echo "${AUTH_USER5}:$(echo "${AUTH_PASS5}" | mkpasswd -s -R 0 -m sha-512)" >>"${htpasswd_file}"
fi
}
function authn_write_zot_config() {
local zot_config_file=${1}
local zot_root_dir=${2}
local zot_port=${3}
local zot_htpasswd_file=${4}
local log_file=${5}
cat >"${zot_config_file}" <<EOF
{
"distSpecVersion":"1.1.1",
"storage":{
"dedupe": true,
"gc": true,
"gcDelay": "1h",
"gcInterval": "6h",
"rootDirectory": "${zot_root_dir}"
},
"http": {
"address": "127.0.0.1",
"port": "${zot_port}",
"realm":"zot",
"auth": {
"htpasswd": {
"path": "${zot_htpasswd_file}"
},
"failDelay": 5
},
"accessControl": {
"repositories": {
"**": {
"anonymousPolicy": ["read"],
"defaultPolicy": ["read", "create"]
}
},
"adminPolicy": {
"users": ["admin"],
"actions": ["read", "create", "update", "delete"]
}
}
},
"log":{
"level":"debug",
"output": "${log_file}"
}
}
EOF
}
function authn_setup_file() {
if ! verify_authn_prerequisites; then
exit 1
fi
pushpull_isolate_regctl_config
skopeo --insecure-policy copy --format=oci \
docker://ghcr.io/project-zot/test-images/busybox:1.36 \
oci:${TEST_DATA_DIR}/busybox:1.36
local zot_root_dir=${BATS_FILE_TMPDIR}/zot
local zot_config_file=${BATS_FILE_TMPDIR}/zot_config.json
local zot_htpasswd_file=${BATS_FILE_TMPDIR}/zot_htpasswd
local log_file=${zot_root_dir}/zot-log.json
zot_port=$(get_free_port_for_service "zot")
echo "${zot_port}" >"${BATS_FILE_TMPDIR}/zot.port"
authn_write_htpasswd_file "${zot_htpasswd_file}"
echo "${zot_root_dir}" >&3
mkdir -p "${zot_root_dir}"
touch "${log_file}"
authn_write_zot_config "${zot_config_file}" "${zot_root_dir}" "${zot_port}" \
"${zot_htpasswd_file}" "${log_file}"
if [ "${PUSHPULL_AUTHN_FIPS_MODE:-0}" = 1 ]; then
export GODEBUG="fips140=only"
fi
zot_serve "${ZOT_PATH}" "${zot_config_file}"
wait_zot_reachable "${zot_port}"
run regctl registry set "localhost:${zot_port}" --tls disabled
[ "${status}" -eq 0 ]
if [ "${PUSHPULL_AUTHN_FIPS_MODE:-0}" = 1 ]; then
log_output | jq 'contains("fips140 is currently enabled")?' | grep true
fi
}
function authn_teardown() {
cat "${BATS_FILE_TMPDIR}/zot/zot-log.json"
}
function authn_teardown_file() {
zot_stop_all
if [ "${PUSHPULL_AUTHN_FIPS_MODE:-0}" = 1 ]; then
unset GODEBUG
fi
}
function helper_authn_regctl_login() {
local user=${1}
local pass=${2}
run regctl registry login "localhost:$(get_zot_port)" -u "${user}" -p "${pass}"
[ "${status}" -eq 0 ]
}
# Args: $1 = source reference, $2 = destination repository
function helper_authn_push_image_with_regclient() {
local source_ref=${1}
local dest_repo=${2}
helper_authn_regctl_login "${AUTH_USER}" "${AUTH_PASS}"
run regctl image copy "${source_ref}" "localhost:$(get_zot_port)/${dest_repo}"
[ "${status}" -eq 0 ]
}
# Args: $1 = source repository, $2 = destination reference
function helper_authn_pull_image_with_regclient() {
local source_repo=${1}
local dest_ref=${2}
helper_authn_regctl_login "${AUTH_USER}" "${AUTH_PASS}"
run regctl image copy "localhost:$(get_zot_port)/${source_repo}" "${dest_ref}"
[ "${status}" -eq 0 ]
}
# Args: $1 = artifact reference
function helper_authn_push_oci_artifact_with_regclient() {
local ref=${1}
helper_authn_regctl_login "${AUTH_USER}" "${AUTH_PASS}"
helper_push_oci_artifact_with_regclient "${ref}"
}
# Args: $1 = artifact reference, $2 = expected artifact content
function helper_authn_pull_oci_artifact_with_regclient() {
local ref=${1}
local expected_content=${2}
helper_authn_regctl_login "${AUTH_USER}" "${AUTH_PASS}"
helper_pull_oci_artifact_with_regclient "${ref}" "${expected_content}"
}
function helper_authn_push_oci_artifact_references_with_regclient() {
helper_authn_regctl_login "${AUTH_USER}" "${AUTH_PASS}"
helper_push_oci_artifact_references_with_regclient 0
}
function helper_authn_list_oci_artifact_references_with_regclient() {
helper_authn_regctl_login "${AUTH_USER}" "${AUTH_PASS}"
helper_pull_oci_artifact_references_with_regclient 1
}
# Args: $1=username, $2=password
function helper_authn_ml_artifacts() {
local user=${1}
local pass=${2}
local zot_port sha256_in sha256_out
helper_authn_regctl_login "${user}" "${pass}"
zot_port=$(get_zot_port)
run curl --fail -L -0 \
https://github.com/tarilabs/demo20231212/raw/main/v1.nb20231206162408/mnist.onnx \
-o "${BATS_FILE_TMPDIR}/mnist.onnx"
[ "${status}" -eq 0 ]
run sha256sum "${BATS_FILE_TMPDIR}/mnist.onnx"
[ "${status}" -eq 0 ]
sha256_in=$(echo "${output}" | awk '{print $1}')
[ -n "${sha256_in}" ]
run regctl artifact put \
--annotation description="used for demo purposes" \
--annotation model_format_name="onnx" \
--annotation model_format_version="1" \
--artifact-type "application/vnd.model.type" \
"localhost:${zot_port}/models/my-model-from-gh:v1" \
-f "${BATS_FILE_TMPDIR}/mnist.onnx"
[ "${status}" -eq 0 ]
run regctl artifact list "localhost:${zot_port}/models/my-model-from-gh:v1" \
--format '{{jsonPretty .}}'
[ "${status}" -eq 0 ]
run regctl artifact list --filter-artifact-type "application/vnd.model.type" \
"localhost:${zot_port}/models/my-model-from-gh:v1" \
--format '{{jsonPretty .}}'
[ "${status}" -eq 0 ]
run bash -c "regctl artifact get 'localhost:${zot_port}/models/my-model-from-gh:v1' >'${BATS_FILE_TMPDIR}/mnist.onnx.check'"
[ "${status}" -eq 0 ]
run sha256sum "${BATS_FILE_TMPDIR}/mnist.onnx.check"
[ "${status}" -eq 0 ]
sha256_out=$(echo "${output}" | awk '{print $1}')
[ -n "${sha256_out}" ]
[ "${sha256_in}" = "${sha256_out}" ]
}
# Args: $1=username, $2=password, $3=hash_type, $4=should_succeed (true/false)
function helper_authn_verify_auth_and_push() {
local user=${1}
local pass=${2}
local hash_type=${3}
local should_succeed=${4}
helper_authn_regctl_login "${user}" "${pass}"
run regctl image copy "ocidir://${TEST_DATA_DIR}/busybox:1.36" \
"localhost:$(get_zot_port)/test-${hash_type}"
if [ "${should_succeed}" = true ]; then
[ "${status}" -eq 0 ]
else
[ "${status}" -eq 1 ]
log_output | jq 'contains("htpasswd bcrypt failed since fips140 is enabled")?' | grep true
fi
}
# Args: $1=username, $2=password, $3=source_repo, $4=dest_ref
function helper_authn_pull_image_with_auth() {
local user=${1}
local pass=${2}
local source_repo=${3}
local dest_ref=${4}
helper_authn_regctl_login "${user}" "${pass}"
run regctl image copy "localhost:$(get_zot_port)/${source_repo}" \
"ocidir://${TEST_DATA_DIR}/busybox:${dest_ref}"
[ "${status}" -eq 0 ]
}
# Args: $1=username, $2=password, $3=artifact_ref, $4=artifact_body
function helper_authn_push_oci_artifact_with_auth() {
local user=${1}
local pass=${2}
local artifact_ref=${3}
local artifact_body=${4}
helper_authn_regctl_login "${user}" "${pass}"
run regctl artifact put "localhost:$(get_zot_port)/${artifact_ref}" <<TXT
${artifact_body}
TXT
[ "${status}" -eq 0 ]
}
# Args: $1=username, $2=password, $3=artifact_ref, $4=expected_body
function helper_authn_pull_oci_artifact_with_auth() {
local user=${1}
local pass=${2}
local artifact_ref=${3}
local expected_body=${4}
helper_authn_regctl_login "${user}" "${pass}"
run regctl manifest get "localhost:$(get_zot_port)/${artifact_ref}"
[ "${status}" -eq 0 ]
run regctl artifact get "localhost:$(get_zot_port)/${artifact_ref}"
[ "${status}" -eq 0 ]
[ "${lines[-1]}" = "${expected_body}" ]
}
function helper_authn_push_oci_artifact_references_with_auth() {
helper_authn_regctl_login "${AUTH_USER2}" "${AUTH_PASS2}"
helper_push_oci_artifact_references_with_regclient 0
}
function helper_authn_list_oci_artifact_references_with_auth() {
helper_authn_regctl_login "${AUTH_USER2}" "${AUTH_PASS2}"
helper_pull_oci_artifact_references_with_regclient 1
}
function helper_authn_ml_artifacts_with_auth() {
helper_authn_ml_artifacts "${AUTH_USER3}" "${AUTH_PASS3}"
}