wayland/tests/scanner-test.sh

79 lines
2.5 KiB
Bash
Raw Permalink Normal View History

tests: add scanner tests Add tests that ensure that wayland-scanner output for a given input does not change unexpectedly. This makes it very easy to review wayland-scanner patches. Before, when patches were proposed for wayland-scanner, I had to build wayland without the patches, save the generated files into a temporary directory, apply the patches, build again, and diff the old vs. new generated file. No more. Now whenever someone makes intentional changes to wayland-scanner's output, he will also have to patch the example output files to match. That means that reviewers see the diff of the generated files straight from the patch itself. Verifying the diff is true is as easy as 'make check'. The tests use separate example XML files instead of wayland.xml directly, so that wayland.xml can be updated without fixing scanner tests, avoiding the churn. example.xml starts as a copy of wayland.xml. If wayland.xml starts using new wayland-scanner features, they should be copied into example.xml again to be covered by the tests. This patch relies on the previous patch to actually add all the data files for input and reference output. The scanner output is fed through sed to remove parts that are allowed to vary: the scanner version string. v2: no need for scanner-test.sh to depend on the test data Task: https://phabricator.freedesktop.org/T3313 Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk> Reviewed-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk> (v1) Reviewed-by: Yong Bakos <ybakos@humanoriented.com> Tested-by: Yong Bakos <ybakos@humanoriented.com>
2016-09-30 20:12:47 +08:00
#!/bin/sh
echo "srcdir: $srcdir"
echo "scanner: $WAYLAND_SCANNER"
echo "test_data_dir: $TEST_DATA_DIR"
echo "test_output_dir: $TEST_OUTPUT_DIR"
echo "pwd: $PWD"
echo "sed: $SED"
RETCODE=0
hard_fail() {
echo "$@" "ERROR"
exit 99
}
fail() {
echo "$@" "FAIL"
RETCODE=1
}
mkdir -p "$TEST_OUTPUT_DIR" || hard_fail "setup"
generate_and_compare() {
echo
echo "Testing $1 generation: $2 -> $3"
"$WAYLAND_SCANNER" $1 < "$TEST_DATA_DIR/$2" > "$TEST_OUTPUT_DIR/$3" || \
hard_fail "$2 -> $3"
"$SED" -i -e 's/Generated by wayland-scanner [0-9.]*/SCANNER TEST/' \
"$TEST_OUTPUT_DIR/$3" || hard_fail "$2 -> $3"
diff -q "$TEST_DATA_DIR/$3" "$TEST_OUTPUT_DIR/$3" && \
echo "$2 -> $3 PASS" || \
fail "$2 -> $3"
}
verify_error() {
echo
echo "Checking that reading $1 gives an error on line $3"
[ -f "$TEST_DATA_DIR/$1" ] || hard_fail "$1 not present"
# Confirm failure error code
"$WAYLAND_SCANNER" server-header < "$TEST_DATA_DIR/$1" >/dev/null 2>"$TEST_OUTPUT_DIR/$2" && \
fail "$1 return code check"
# Verify that an error is produced at the correct line
grep -q "<stdin>:$3: error:" "$TEST_OUTPUT_DIR/$2" && echo "$1 PASS" || fail "$1 line number check"
}
tests: add scanner tests Add tests that ensure that wayland-scanner output for a given input does not change unexpectedly. This makes it very easy to review wayland-scanner patches. Before, when patches were proposed for wayland-scanner, I had to build wayland without the patches, save the generated files into a temporary directory, apply the patches, build again, and diff the old vs. new generated file. No more. Now whenever someone makes intentional changes to wayland-scanner's output, he will also have to patch the example output files to match. That means that reviewers see the diff of the generated files straight from the patch itself. Verifying the diff is true is as easy as 'make check'. The tests use separate example XML files instead of wayland.xml directly, so that wayland.xml can be updated without fixing scanner tests, avoiding the churn. example.xml starts as a copy of wayland.xml. If wayland.xml starts using new wayland-scanner features, they should be copied into example.xml again to be covered by the tests. This patch relies on the previous patch to actually add all the data files for input and reference output. The scanner output is fed through sed to remove parts that are allowed to vary: the scanner version string. v2: no need for scanner-test.sh to depend on the test data Task: https://phabricator.freedesktop.org/T3313 Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk> Reviewed-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk> (v1) Reviewed-by: Yong Bakos <ybakos@humanoriented.com> Tested-by: Yong Bakos <ybakos@humanoriented.com>
2016-09-30 20:12:47 +08:00
generate_and_compare "code" "example.xml" "example-code.c"
generate_and_compare "client-header" "example.xml" "example-client.h"
generate_and_compare "server-header" "example.xml" "example-server.h"
generate_and_compare "code" "small.xml" "small-code.c"
generate_and_compare "client-header" "small.xml" "small-client.h"
generate_and_compare "server-header" "small.xml" "small-server.h"
generate_and_compare "-c code" "small.xml" "small-code-core.c"
generate_and_compare "-c client-header" "small.xml" "small-client-core.h"
generate_and_compare "-c server-header" "small.xml" "small-server-core.h"
# The existing "code" must produce result identical to "public-code"
generate_and_compare "code" "small.xml" "small-code.c"
generate_and_compare "public-code" "small.xml" "small-code.c"
generate_and_compare "private-code" "small.xml" "small-private-code.c"
verify_error "bad-identifier-arg.xml" "bad-identifier-arg.log" 7
verify_error "bad-identifier-entry.xml" "bad-identifier-entry.log" 8
verify_error "bad-identifier-enum.xml" "bad-identifier-enum.log" 6
verify_error "bad-identifier-event.xml" "bad-identifier-event.log" 6
verify_error "bad-identifier-interface.xml" "bad-identifier-interface.log" 3
verify_error "bad-identifier-protocol.xml" "bad-identifier-protocol.log" 2
verify_error "bad-identifier-request.xml" "bad-identifier-request.log" 6
tests: add scanner tests Add tests that ensure that wayland-scanner output for a given input does not change unexpectedly. This makes it very easy to review wayland-scanner patches. Before, when patches were proposed for wayland-scanner, I had to build wayland without the patches, save the generated files into a temporary directory, apply the patches, build again, and diff the old vs. new generated file. No more. Now whenever someone makes intentional changes to wayland-scanner's output, he will also have to patch the example output files to match. That means that reviewers see the diff of the generated files straight from the patch itself. Verifying the diff is true is as easy as 'make check'. The tests use separate example XML files instead of wayland.xml directly, so that wayland.xml can be updated without fixing scanner tests, avoiding the churn. example.xml starts as a copy of wayland.xml. If wayland.xml starts using new wayland-scanner features, they should be copied into example.xml again to be covered by the tests. This patch relies on the previous patch to actually add all the data files for input and reference output. The scanner output is fed through sed to remove parts that are allowed to vary: the scanner version string. v2: no need for scanner-test.sh to depend on the test data Task: https://phabricator.freedesktop.org/T3313 Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk> Reviewed-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk> (v1) Reviewed-by: Yong Bakos <ybakos@humanoriented.com> Tested-by: Yong Bakos <ybakos@humanoriented.com>
2016-09-30 20:12:47 +08:00
exit $RETCODE