refactor(makefile): consolidate the make targets used for bats tests (#1746)

New examples of running tests:

1. To run a specific bats file (with and without verbose output):
make run-blackbox-tests BATS_TEST_FILE_PATH=test/blackbox/delete_images.bats
make run-blackbox-tests BATS_TEST_FILE_PATH=test/blackbox/delete_images.bats BATS_VERBOSITY=2

2. To run the CI tests (with and without verbose output)
make run-blackbox-ci
make run-blackbox-ci BATS_VERBOSITY=2

BATS_TEST_FILE_PATH is used to pass on the test file to run using `run-blackbox-tests`
BATS_VERBOSITY controls the verbosity of the bats framework output, if unspecified the output only
contains test results and failure message in case of failures.
If BATS_VERBOSITY is 1, then also show commands as they are executed.
If BATS_VERBOSITY is 2, on top of the above it also shows output of passed tests.

Other changes in this PR:
- Update some of the tests to show logs after the run ends.
- Run the linters before the tests, as it saves time on failures when running in GH

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
This commit is contained in:
Andrei Aaron
2023-09-07 21:06:21 +03:00
committed by GitHub
parent f5b63963be
commit 6115eed4ec
22 changed files with 199 additions and 211 deletions
+8 -2
View File
@@ -1,4 +1,4 @@
# Note: Intended to be run as "make test-push-pull" or "test-push-pull-verbose"
# Note: Intended to be run as "make run-blackbox-tests" or "make run-blackbox-ci"
# Makefile target installs & checks all necessary tooling
# Extra tools that are not covered in Makefile target needs to be added in verify_prerequisites()
@@ -42,7 +42,8 @@ function setup_file() {
"port": "8080"
},
"log": {
"level": "debug"
"level": "debug",
"output": "${BATS_FILE_TMPDIR}/zot.log"
}
}
EOF
@@ -51,6 +52,11 @@ EOF
wait_zot_reachable 8080
}
function teardown() {
# conditionally printing on failure is possible from teardown but not from from teardown_file
cat ${BATS_FILE_TMPDIR}/zot.log
}
function teardown_file() {
zot_stop_all
}