* [PATCH olang 0/4] comment based integration tests
@ 2024-05-12 14:30 Carlos Maniero
2024-05-12 14:30 ` [PATCH olang 1/4] tests: add comment based integration tests mechanism Carlos Maniero
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Carlos Maniero @ 2024-05-12 14:30 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
The old munit based integration tests was replaced by a new mechanism
that makes testing more tasteful and easy to perform.
To create an integration tests you just need to create a olang file at
*tests/integration/tests* directory. The assertions are performed
thought a couple of comments.
# spec: some text to describe what are you testing
# compiler_exit: the compilers exit code
# compiler_output: the compilers output, %empty% for empty output
# program_exit: the program exit code
# ast: the ast output
# tokens: the tokens
This patchset also includes parser tests errors and creates a TODO
mechanism that can be used to track what need to be implemented.
Carlos Maniero (4):
tests: add comment based integration tests mechanism
tests: remove previous integration tests structure
tests: include integration tests for function parser errors
tests: print integration tests TODOs
Makefile | 1 -
tests/integration/Makefile | 33 +--
tests/integration/cli_runner.c | 100 ---------
tests/integration/cli_runner.h | 32 ---
tests/integration/cli_test.c | 132 ------------
tests/integration/proc_exec.c | 63 ------
tests/integration/proc_exec.h | 37 ----
tests/integration/test.sh | 195 ++++++++++++++++++
tests/integration/tests/0001_main_exit.ol | 27 +++
.../tests/0002_fn_without_fn_keyword.ol | 8 +
.../integration/tests/0003_fn_without_type.ol | 8 +
.../tests/0004_fn_without_type_colon.ol | 8 +
.../0005_fn_without_open_curly_brackets.ol | 8 +
.../0006_fn_without_close_curly_brackets.ol | 7 +
.../tests/0007_fn_without_return.ol | 8 +
15 files changed, 277 insertions(+), 390 deletions(-)
delete mode 100644 tests/integration/cli_runner.c
delete mode 100644 tests/integration/cli_runner.h
delete mode 100644 tests/integration/cli_test.c
delete mode 100644 tests/integration/proc_exec.c
delete mode 100644 tests/integration/proc_exec.h
create mode 100755 tests/integration/test.sh
create mode 100644 tests/integration/tests/0001_main_exit.ol
create mode 100644 tests/integration/tests/0002_fn_without_fn_keyword.ol
create mode 100644 tests/integration/tests/0003_fn_without_type.ol
create mode 100644 tests/integration/tests/0004_fn_without_type_colon.ol
create mode 100644 tests/integration/tests/0005_fn_without_open_curly_brackets.ol
create mode 100644 tests/integration/tests/0006_fn_without_close_curly_brackets.ol
create mode 100644 tests/integration/tests/0007_fn_without_return.ol
--
2.34.1
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH olang 1/4] tests: add comment based integration tests mechanism 2024-05-12 14:30 [PATCH olang 0/4] comment based integration tests Carlos Maniero @ 2024-05-12 14:30 ` Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 2/4] tests: remove previous integration tests structure Carlos Maniero ` (4 subsequent siblings) 5 siblings, 0 replies; 8+ messages in thread From: Carlos Maniero @ 2024-05-12 14:30 UTC (permalink / raw) To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero The old munit based integration tests was replaced by a new mechanism that makes testing more tasteful and easy to perform. To create an integration tests you just need to create a olang file at *tests/integration/tests* directory. The assertions are performed thought a couple of comments. spec: some text to describe what are you testing compiler_exit: the compilers exit code compiler_output: the compilers output, %empty% for empty output program_exit: the program exit code ast: the ast output tokens: the tokens Since the language has the line_feed token it is impossible to test the EOF once when added a comment the EOF will be located at line + 1. So that the tokens test the first n-lines defined. The AST compares the entire AST tough. Signed-off-by: Carlos Maniero <carlos@maniero.me> --- Makefile | 1 - tests/integration/Makefile | 33 +--- tests/integration/test.sh | 182 ++++++++++++++++++++++ tests/integration/tests/0001_main_exit.ol | 27 ++++ 4 files changed, 217 insertions(+), 26 deletions(-) create mode 100755 tests/integration/test.sh create mode 100644 tests/integration/tests/0001_main_exit.ol diff --git a/Makefile b/Makefile index cdfc8e1..a0130f5 100644 --- a/Makefile +++ b/Makefile @@ -19,7 +19,6 @@ $(BUILD_DIR): .PHONY: linter linter: $(SRCS) $(HEADERS) clang-format --dry-run --Werror $? - $(MAKE) -C tests/integration/ linter $(MAKE) -C tests/unit/ linter .PHONY: linter-fix diff --git a/tests/integration/Makefile b/tests/integration/Makefile index db2b7d9..567ce88 100644 --- a/tests/integration/Makefile +++ b/tests/integration/Makefile @@ -1,31 +1,14 @@ -SRCS := $(wildcard *_test.c) -OBJS := $(patsubst %_test.c, %_test.o, $(SRCS)) -CFLAGS := -I../../src -I../shared -TESTS := $(patsubst %_test.c, %_test, $(SRCS)) -EXEC_TESTS := $(patsubst %_test, ./%_test, $(TESTS)) -MUNIT_SRC := ../shared/munit.c -MUNIT := ./munit.o +TESTER_SRC := ./test.sh +TESTS := $(wildcard ./tests/*.ol) .PHONY: all -all: $(MUNIT) proc_exec.o cli_runner.o $(TESTS) - @for file in $(EXEC_TESTS); do \ - ./"$$file"; \ +all: + @set -e; \ + for file in $(TESTS); do \ + $(TESTER_SRC) "$$file"; \ + echo; \ done .PHONY: clean clean: - $(RM) *.o *_test - -.PHONY: linter -linter: $(SRCS) - clang-format --dry-run --Werror $? - -.PHONY: linter-fix -linter-fix: $(SRCS) - clang-format -i $? - -cli_test: $(MUNIT) proc_exec.o cli_runner.o cli_test.o - $(CC) $? $(CFLAGS) -o $@ - -$(MUNIT): - $(CC) -c $(MUNIT_SRC) $(CFLAGS) -o $(MUNIT) + @rm -rf ./tests/*.ol.test* diff --git a/tests/integration/test.sh b/tests/integration/test.sh new file mode 100755 index 0000000..6ab03de --- /dev/null +++ b/tests/integration/test.sh @@ -0,0 +1,182 @@ +COMPILER_PATH="../../olang" +TEST_FILE="$1" + +PROGRAM="$TEST_FILE.test" + +COMPILER_OUTPUT_FILE="$TEST_FILE.test.compiler_output" +COMPILER_EXPECT_OUTPUT_FILE="$TEST_FILE.test.expected_compiler_output" + +AST_OUTPUT_FILE="$TEST_FILE.test.ast_output" +AST_EXPECT_OUTPUT_FILE="$TEST_FILE.test.expected_ast_output" + +TOKENS_OUTPUT_FILE="$TEST_FILE.test.tokens_output" +TOKENS_EXPECT_OUTPUT_FILE="$TEST_FILE.test.expected_tokens_output" + +TOKENS_OUTPUT_FILE="$TEST_FILE.test.tokens_output" +TOKENS_EXPECT_OUTPUT_FILE="$TEST_FILE.test.expected_tokens_output" + +PROGRAM_OUTPUT="$TEST_FILE.test.program_output" + +extract_comment() { + tag="$1" + comment="# $tag: " + cat $TEST_FILE | grep "$comment" | sed -e "s/$comment//" +} + +# UI +COLOR_RED=1 +COLOR_GREEN=2 +COLOR_YELLOW=3 +COLOR_BLUE=4 +COLOR_CYAN=6 +COLOR_GRAY=7 + +colored() { + text="$1" + color="$2" + if [ -t 1 ]; then + if tput setaf 1 &> /dev/null; then + printf "$(tput setaf $color)$text$(tput sgr0)" + return + fi + fi + + printf "$text" +} + +print_test_description() { + colored "$TEST_FILE: \n" $COLOR_CYAN + colored "$(extract_comment spec)\n\n" $COLOR_GRAY +} + +print_passed() { + context="$1" + printf "$context: " + colored "passed" $COLOR_GREEN + echo +} + +print_failed() { + context="$1" + printf "$context: " + colored "failed" $COLOR_RED + echo +} + +print_skiped() { + context="$1" + printf "$context: " + colored "not set" $COLOR_GRAY + echo +} + +# Assertions +expect_exit_code() { + context="$1" + expected="$2" + actual="$3" + + if [ -z "$expected" ]; then + print_skiped "$context" + echo "no expected exit code set, it exited with \"$actual\"." + else + if [ "$expected" = "$actual" ]; then + print_passed "$context" + else + print_failed "$context" + colored "Expected exit code: " $COLOR_YELLOW + colored "$expected" $COLOR_GRAY + colored ", actual: " $COLOR_YELLOW + colored "$actual\n" $COLOR_GRAY + exit 1 + fi + fi +} + +expect_output() { + context="$1" + expected_file="$2" + actual_file="$3" + + if [ "$(cat $expected_file | wc -l)" = "0" ]; then + print_skiped "$context" + return + fi + + if cmp -s "$expected_file" "$actual_file"; then + print_passed "$context" + else + if [ "$(cat "$expected_file")" = "%empty%" ]; then + if [ "$(cat $actual_file | wc -c)" = "0" ]; then + print_passed "$context" + return + fi + fi + print_failed "$context" + colored "$context not match:\n" $COLOR_YELLOW + diff "$actual_file" "$expected_file" -u --color + exit 1 + fi +} + +cleanup() { + rm -f $PROGRAM* +} + +test_compiler() { + $COMPILER_PATH $TEST_FILE -o $PROGRAM > "$COMPILER_OUTPUT_FILE" 2>&1 + + EXIT=$? + EXPECTED_EXIT=$(extract_comment compiler_exit) + + expect_exit_code compiler_exit "$EXPECTED_EXIT" "$EXIT" + + extract_comment compiler_output > $COMPILER_EXPECT_OUTPUT_FILE + expect_output compiler_output "$COMPILER_EXPECT_OUTPUT_FILE" "$COMPILER_OUTPUT_FILE" + + if [ "$EXIT" = "1" ]; then + echo "program execution skiped since the compiler failed as expected." + cleanup + exit 0 + fi +} + +test_ast() { + $COMPILER_PATH $TEST_FILE --dump-ast > "$AST_OUTPUT_FILE" 2>&1 + + extract_comment ast > $AST_EXPECT_OUTPUT_FILE + expect_output "ast" "$AST_EXPECT_OUTPUT_FILE" "$AST_OUTPUT_FILE" +} + +test_tokens() { + extract_comment tokens > $TOKENS_EXPECT_OUTPUT_FILE + TOKEN_LINES=$(cat $TOKENS_EXPECT_OUTPUT_FILE | wc -l) + + $COMPILER_PATH $TEST_FILE --dump-tokens | head -n $TOKEN_LINES > "$TOKENS_OUTPUT_FILE" 2>&1 + + expect_output tokens $TOKENS_OUTPUT_FILE $TOKENS_EXPECT_OUTPUT_FILE + + print_passed tokens +} + +test_program() { + ./$PROGRAM + + EXIT=$? + EXPECTED_EXIT=$(extract_comment program_exit) + expect_exit_code "program_exit" "$EXPECTED_EXIT" "$EXIT" +} + + +main() { + print_test_description + + test_compiler + test_program + test_tokens + test_ast + + cleanup +} + +main diff --git a/tests/integration/tests/0001_main_exit.ol b/tests/integration/tests/0001_main_exit.ol new file mode 100644 index 0000000..c30492f --- /dev/null +++ b/tests/integration/tests/0001_main_exit.ol @@ -0,0 +1,27 @@ +# spec: A minimal olang program +fn main(): u32 { + return 0 +} +# compiler_exit: 0 +# compiler_output: %empty% +# program_exit: 0 +# +# ast: Translation_Unit +# ast: `-Function_Definition <name:main> <return:0> +# ast: `-Block +# ast: `-Return_Statement +# ast: `-Literal <kind:u32> <value:0> +# +# tokens: ./tests/0001_main_exit.ol:1:32: <line_feed> +# tokens: ./tests/0001_main_exit.ol:2:1: <fn> +# tokens: ./tests/0001_main_exit.ol:2:4: <identifier> +# tokens: ./tests/0001_main_exit.ol:2:8: <(> +# tokens: ./tests/0001_main_exit.ol:2:9: <)> +# tokens: ./tests/0001_main_exit.ol:2:10: <:> +# tokens: ./tests/0001_main_exit.ol:2:12: <identifier> +# tokens: ./tests/0001_main_exit.ol:2:16: <{> +# tokens: ./tests/0001_main_exit.ol:2:17: <line_feed> +# tokens: ./tests/0001_main_exit.ol:3:3: <return> +# tokens: ./tests/0001_main_exit.ol:3:10: <number> +# tokens: ./tests/0001_main_exit.ol:3:11: <line_feed> +# tokens: ./tests/0001_main_exit.ol:4:1: <}> -- 2.34.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH olang 2/4] tests: remove previous integration tests structure 2024-05-12 14:30 [PATCH olang 0/4] comment based integration tests Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 1/4] tests: add comment based integration tests mechanism Carlos Maniero @ 2024-05-12 14:30 ` Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 3/4] tests: include integration tests for function parser errors Carlos Maniero ` (3 subsequent siblings) 5 siblings, 0 replies; 8+ messages in thread From: Carlos Maniero @ 2024-05-12 14:30 UTC (permalink / raw) To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero Signed-off-by: Carlos Maniero <carlos@maniero.me> --- tests/integration/cli_runner.c | 100 ------------------------- tests/integration/cli_runner.h | 32 -------- tests/integration/cli_test.c | 132 --------------------------------- tests/integration/proc_exec.c | 63 ---------------- tests/integration/proc_exec.h | 37 --------- tests/integration/test.sh | 62 ++++++++-------- 6 files changed, 33 insertions(+), 393 deletions(-) delete mode 100644 tests/integration/cli_runner.c delete mode 100644 tests/integration/cli_runner.h delete mode 100644 tests/integration/cli_test.c delete mode 100644 tests/integration/proc_exec.c delete mode 100644 tests/integration/proc_exec.h diff --git a/tests/integration/cli_runner.c b/tests/integration/cli_runner.c deleted file mode 100644 index 636abfc..0000000 --- a/tests/integration/cli_runner.c +++ /dev/null @@ -1,100 +0,0 @@ -/* - * Copyright (C) 2024 olang maintainers - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see <https://www.gnu.org/licenses/>. - */ -#include "cli_runner.h" -#include "proc_exec.h" -#include <assert.h> -#include <stdbool.h> -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <sys/wait.h> -#include <unistd.h> - -#define OLANG_COMPILER_PATH "../../olang" - -static int compiler_exists_already_checked = 0; - -static void -assert_compiler_exists() -{ - { - if (compiler_exists_already_checked == 1) { - return; - } - - compiler_exists_already_checked = 1; - } - - FILE *file = fopen(OLANG_COMPILER_PATH, "r"); - - if (file != NULL) { - fclose(file); - return; - } - - perror("Build the compiler before executing tests"); - exit(1); -} - -void -create_tmp_file_name(char *file_name) -{ - sprintf(file_name, "%s/olang_programXXXXXX", P_tmpdir); - int fd = mkstemp(file_name); - - if (fd == -1) { - perror("Could not create a tmp file. Check your P_tmpdir permission."); - exit(1); - } - close(fd); -} - -void -cli_runner_compiler(cli_result_t* result, char *args[]) -{ - assert_compiler_exists(); - - proc_exec_command_t command = { - .path = OLANG_COMPILER_PATH, - .args = args - }; - - proc_exec(&command); - - result->exec = command.result; -} - -cli_result_t -cli_runner_compiler_dump_tokens(char *src) -{ - cli_result_t result = { 0 }; - - char *program_args[] = { "olang", "--dump-tokens", src, NULL }; - cli_runner_compiler(&result, program_args); - return result; -} - -cli_result_t -cli_runner_compiler_compile(char *src) -{ - cli_result_t result = { 0 }; - create_tmp_file_name(result.binary_path); - - char *program_args[] = { "olang", src, "-o", result.binary_path, NULL }; - cli_runner_compiler(&result, program_args); - return result; -} diff --git a/tests/integration/cli_runner.h b/tests/integration/cli_runner.h deleted file mode 100644 index 785cd34..0000000 --- a/tests/integration/cli_runner.h +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Copyright (C) 2024 olang maintainers - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see <https://www.gnu.org/licenses/>. - */ -#ifndef CLI_RUNNER_H -#define CLI_RUNNER_H -#include "proc_exec.h" - -typedef struct cli_result_t -{ - char binary_path[255]; - proc_exec_result_t exec; -} cli_result_t; - -cli_result_t -cli_runner_compiler_dump_tokens(char *src); - -cli_result_t -cli_runner_compiler_compile(char *src); -#endif diff --git a/tests/integration/cli_test.c b/tests/integration/cli_test.c deleted file mode 100644 index e7ae059..0000000 --- a/tests/integration/cli_test.c +++ /dev/null @@ -1,132 +0,0 @@ -/* - * Copyright (C) 2024 olang maintainers - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see <https://www.gnu.org/licenses/>. - */ -#define MUNIT_ENABLE_ASSERT_ALIASES -#include "cli_runner.h" -#include "munit.h" -#include <stdio.h> - -static MunitResult -test_cli_dump_tokens_example_main_exit(const MunitParameter params[], void *user_data_or_fixture) -{ - cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/main_exit.ol"); - munit_assert_int(compilation_result.exec.exit_code, ==, 0); - munit_assert_string_equal(compilation_result.exec.stdout_buf, - "../../examples/main_exit.ol:1:12: <line_feed>\n" - "../../examples/main_exit.ol:2:16: <line_feed>\n" - "../../examples/main_exit.ol:3:1: <line_feed>\n" - "../../examples/main_exit.ol:4:1: <fn>\n" - "../../examples/main_exit.ol:4:4: <identifier>\n" - "../../examples/main_exit.ol:4:8: <(>\n" - "../../examples/main_exit.ol:4:9: <)>\n" - "../../examples/main_exit.ol:4:10: <:>\n" - "../../examples/main_exit.ol:4:12: <identifier>\n" - "../../examples/main_exit.ol:4:16: <{>\n" - "../../examples/main_exit.ol:4:17: <line_feed>\n" - "../../examples/main_exit.ol:5:3: <return>\n" - "../../examples/main_exit.ol:5:10: <number>\n" - "../../examples/main_exit.ol:5:11: <line_feed>\n" - "../../examples/main_exit.ol:6:1: <}>\n" - "../../examples/main_exit.ol:6:2: <line_feed>\n" - "../../examples/main_exit.ol:7:1: <EOF>\n"); - return MUNIT_OK; -} - -static MunitResult -test_cli_dump_tokens_example_expression(const MunitParameter params[], void *user_data_or_fixture) -{ - cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/expression.ol"); - munit_assert_int(compilation_result.exec.exit_code, ==, 0); - munit_assert_string_equal(compilation_result.exec.stdout_buf, - "../../examples/expression.ol:1:1: <fn>\n" - "../../examples/expression.ol:1:4: <identifier>\n" - "../../examples/expression.ol:1:8: <(>\n" - "../../examples/expression.ol:1:9: <)>\n" - "../../examples/expression.ol:1:10: <:>\n" - "../../examples/expression.ol:1:12: <identifier>\n" - "../../examples/expression.ol:1:16: <{>\n" - "../../examples/expression.ol:1:17: <line_feed>\n" - "../../examples/expression.ol:2:3: <return>\n" - "../../examples/expression.ol:2:10: <(>\n" - "../../examples/expression.ol:2:11: <number>\n" - "../../examples/expression.ol:2:14: <+>\n" - "../../examples/expression.ol:2:16: <number>\n" - "../../examples/expression.ol:2:18: <*>\n" - "../../examples/expression.ol:2:20: <number>\n" - "../../examples/expression.ol:2:21: <)>\n" - "../../examples/expression.ol:2:23: <->\n" - "../../examples/expression.ol:2:25: <(>\n" - "../../examples/expression.ol:2:26: <number>\n" - "../../examples/expression.ol:2:29: <->\n" - "../../examples/expression.ol:2:31: <(>\n" - "../../examples/expression.ol:2:32: <number>\n" - "../../examples/expression.ol:2:34: <+>\n" - "../../examples/expression.ol:2:36: <number>\n" - "../../examples/expression.ol:2:37: <)>\n" - "../../examples/expression.ol:2:39: </>\n" - "../../examples/expression.ol:2:41: <number>\n" - "../../examples/expression.ol:2:42: <)>\n" - "../../examples/expression.ol:2:43: <line_feed>\n" - "../../examples/expression.ol:3:1: <}>\n" - "../../examples/expression.ol:3:2: <line_feed>\n" - "../../examples/expression.ol:4:1: <EOF>\n"); - return MUNIT_OK; -} - -static MunitResult -test_cli_compile_minimal_program(const MunitParameter params[], void *user_data_or_fixture) -{ - cli_result_t compilation_result = cli_runner_compiler_compile("../../examples/main_exit.ol"); - munit_assert_int(compilation_result.exec.exit_code, ==, 0); - - char *command_args[] = { compilation_result.binary_path, NULL }; - - proc_exec_command_t command = { .path = command_args[0], .args = command_args }; - - proc_exec(&command); - - remove(command_args[0]); - - munit_assert_int(command.result.exit_code, ==, 0); - - return MUNIT_OK; -} - -static MunitTest tests[] = { - { "/test_cli_dump_tokens_example_main_exit", - test_cli_dump_tokens_example_main_exit, - NULL, - NULL, - MUNIT_TEST_OPTION_NONE, - NULL }, - { "/test_cli_dump_tokens_example_expression", - test_cli_dump_tokens_example_expression, - NULL, - NULL, - MUNIT_TEST_OPTION_NONE, - NULL }, - { "/test_cli_compile_minimal_program", test_cli_compile_minimal_program, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL }, - { NULL, NULL, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL } -}; - -static const MunitSuite suite = { "/cli_test", tests, NULL, 1, MUNIT_SUITE_OPTION_NONE }; - -int -main(int argc, char *argv[]) -{ - return munit_suite_main(&suite, NULL, argc, argv); - return EXIT_SUCCESS; -} diff --git a/tests/integration/proc_exec.c b/tests/integration/proc_exec.c deleted file mode 100644 index c22dd1e..0000000 --- a/tests/integration/proc_exec.c +++ /dev/null @@ -1,63 +0,0 @@ -/* - * Copyright (C) 2024 olang maintainers - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see <https://www.gnu.org/licenses/>. - */ -#include "proc_exec.h" -#include <assert.h> -#include <stdbool.h> -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <sys/wait.h> -#include <unistd.h> - - -void -proc_exec(proc_exec_command_t* command) -{ - int fd_pipe[2]; - - if (pipe(fd_pipe) == -1) { - perror("pipe error."); - exit(1); - } - - pid_t pid = fork(); - - if (pid == -1) { - perror("fork error."); - exit(1); - } - - if (pid == 0) { - dup2(fd_pipe[1], STDOUT_FILENO); - close(fd_pipe[0]); - close(fd_pipe[1]); - - execv(command->path, command->args); - perror("execl error."); - exit(127); - } else { - close(fd_pipe[1]); - // TODO: stop truncating the output. - if (read(fd_pipe[0], command->result.stdout_buf, sizeof(command->result.stdout_buf)) == -1) { - perror("read error."); - exit(1); - } - int status; - waitpid(pid, &status, 0); - command->result.exit_code = WEXITSTATUS(status); - } -} diff --git a/tests/integration/proc_exec.h b/tests/integration/proc_exec.h deleted file mode 100644 index 45c2977..0000000 --- a/tests/integration/proc_exec.h +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright (C) 2024 olang maintainers - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see <https://www.gnu.org/licenses/>. - */ -#ifndef PROC_EXEC_H -#define PROC_EXEC_H -#include <stdlib.h> - -typedef struct proc_exec_result -{ - int exit_code; - // FIXME: output buffer shouldn't be fixed size - char stdout_buf[2048]; -} proc_exec_result_t; - -typedef struct proc_exec_command -{ - char* path; - char** args; - proc_exec_result_t result; -} proc_exec_command_t; - -void -proc_exec(proc_exec_command_t* command); -#endif diff --git a/tests/integration/test.sh b/tests/integration/test.sh index 6ab03de..9fdabef 100755 --- a/tests/integration/test.sh +++ b/tests/integration/test.sh @@ -1,3 +1,4 @@ +#!/bin/sh COMPILER_PATH="../../olang" TEST_FILE="$1" @@ -15,57 +16,63 @@ TOKENS_EXPECT_OUTPUT_FILE="$TEST_FILE.test.expected_tokens_output" TOKENS_OUTPUT_FILE="$TEST_FILE.test.tokens_output" TOKENS_EXPECT_OUTPUT_FILE="$TEST_FILE.test.expected_tokens_output" -PROGRAM_OUTPUT="$TEST_FILE.test.program_output" - +# Misc extract_comment() { tag="$1" - comment="# $tag: " - cat $TEST_FILE | grep "$comment" | sed -e "s/$comment//" + comment="# $tag:" + grep "$comment" "$TEST_FILE" | sed -e "s/$comment //" +} + +cleanup() { + rm -f "$PROGRAM*" } # UI COLOR_RED=1 COLOR_GREEN=2 COLOR_YELLOW=3 -COLOR_BLUE=4 COLOR_CYAN=6 COLOR_GRAY=7 colored() { text="$1" - color="$2" + color=$(tput setaf "$2") + reset=$(tput sgr0) + if [ -t 1 ]; then - if tput setaf 1 &> /dev/null; then - printf "$(tput setaf $color)$text$(tput sgr0)" + if tput setaf 1 > /dev/null 2>&1; then + printf "%s%s%s" "$color" "$text" "$reset" return fi fi - printf "$text" + printf "%s" "$text" } print_test_description() { - colored "$TEST_FILE: \n" $COLOR_CYAN - colored "$(extract_comment spec)\n\n" $COLOR_GRAY + colored "$TEST_FILE: " $COLOR_CYAN + printf "\n" + colored "$(extract_comment spec)" $COLOR_GRAY + printf "\n\n" } print_passed() { context="$1" - printf "$context: " + printf "%s: " "$context" colored "passed" $COLOR_GREEN echo } print_failed() { context="$1" - printf "$context: " + printf "%s: " "$context" colored "failed" $COLOR_RED echo } print_skiped() { context="$1" - printf "$context: " + printf "%s: " "$context" colored "not set" $COLOR_GRAY echo } @@ -98,7 +105,7 @@ expect_output() { expected_file="$2" actual_file="$3" - if [ "$(cat $expected_file | wc -l)" = "0" ]; then + if [ "$(wc -l < "$expected_file")" = "0" ]; then print_skiped "$context" return fi @@ -107,7 +114,7 @@ expect_output() { print_passed "$context" else if [ "$(cat "$expected_file")" = "%empty%" ]; then - if [ "$(cat $actual_file | wc -c)" = "0" ]; then + if [ "$(wc -c < "$actual_file")" = "0" ]; then print_passed "$context" return fi @@ -119,19 +126,16 @@ expect_output() { fi } -cleanup() { - rm -f $PROGRAM* -} - +# Tests test_compiler() { - $COMPILER_PATH $TEST_FILE -o $PROGRAM > "$COMPILER_OUTPUT_FILE" 2>&1 + $COMPILER_PATH "$TEST_FILE" -o "$PROGRAM" > "$COMPILER_OUTPUT_FILE" 2>&1 EXIT=$? EXPECTED_EXIT=$(extract_comment compiler_exit) expect_exit_code compiler_exit "$EXPECTED_EXIT" "$EXIT" - extract_comment compiler_output > $COMPILER_EXPECT_OUTPUT_FILE + extract_comment compiler_output > "$COMPILER_EXPECT_OUTPUT_FILE" expect_output compiler_output "$COMPILER_EXPECT_OUTPUT_FILE" "$COMPILER_OUTPUT_FILE" if [ "$EXIT" = "1" ]; then @@ -142,25 +146,25 @@ test_compiler() { } test_ast() { - $COMPILER_PATH $TEST_FILE --dump-ast > "$AST_OUTPUT_FILE" 2>&1 + $COMPILER_PATH "$TEST_FILE" --dump-ast > "$AST_OUTPUT_FILE" 2>&1 - extract_comment ast > $AST_EXPECT_OUTPUT_FILE + extract_comment ast > "$AST_EXPECT_OUTPUT_FILE" expect_output "ast" "$AST_EXPECT_OUTPUT_FILE" "$AST_OUTPUT_FILE" } test_tokens() { - extract_comment tokens > $TOKENS_EXPECT_OUTPUT_FILE - TOKEN_LINES=$(cat $TOKENS_EXPECT_OUTPUT_FILE | wc -l) + extract_comment tokens > "$TOKENS_EXPECT_OUTPUT_FILE" + TOKEN_LINES=$(wc -l < "$TOKENS_EXPECT_OUTPUT_FILE") - $COMPILER_PATH $TEST_FILE --dump-tokens | head -n $TOKEN_LINES > "$TOKENS_OUTPUT_FILE" 2>&1 + $COMPILER_PATH "$TEST_FILE" --dump-tokens | head -n "$TOKEN_LINES" > "$TOKENS_OUTPUT_FILE" 2>&1 - expect_output tokens $TOKENS_OUTPUT_FILE $TOKENS_EXPECT_OUTPUT_FILE + expect_output tokens "$TOKENS_OUTPUT_FILE" "$TOKENS_EXPECT_OUTPUT_FILE" print_passed tokens } test_program() { - ./$PROGRAM + "./$PROGRAM" EXIT=$? EXPECTED_EXIT=$(extract_comment program_exit) -- 2.34.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH olang 3/4] tests: include integration tests for function parser errors 2024-05-12 14:30 [PATCH olang 0/4] comment based integration tests Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 1/4] tests: add comment based integration tests mechanism Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 2/4] tests: remove previous integration tests structure Carlos Maniero @ 2024-05-12 14:30 ` Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 4/4] tests: print integration tests TODOs Carlos Maniero ` (2 subsequent siblings) 5 siblings, 0 replies; 8+ messages in thread From: Carlos Maniero @ 2024-05-12 14:30 UTC (permalink / raw) To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero This new tests covers the parser errors we support. Signed-off-by: Carlos Maniero <carlos@maniero.me> --- tests/integration/tests/0002_fn_without_fn_keyword.ol | 8 ++++++++ tests/integration/tests/0003_fn_without_type.ol | 8 ++++++++ tests/integration/tests/0004_fn_without_type_colon.ol | 8 ++++++++ .../tests/0005_fn_without_open_curly_brackets.ol | 8 ++++++++ .../tests/0006_fn_without_close_curly_brackets.ol | 7 +++++++ tests/integration/tests/0007_fn_without_return.ol | 8 ++++++++ 6 files changed, 47 insertions(+) create mode 100644 tests/integration/tests/0002_fn_without_fn_keyword.ol create mode 100644 tests/integration/tests/0003_fn_without_type.ol create mode 100644 tests/integration/tests/0004_fn_without_type_colon.ol create mode 100644 tests/integration/tests/0005_fn_without_open_curly_brackets.ol create mode 100644 tests/integration/tests/0006_fn_without_close_curly_brackets.ol create mode 100644 tests/integration/tests/0007_fn_without_return.ol diff --git a/tests/integration/tests/0002_fn_without_fn_keyword.ol b/tests/integration/tests/0002_fn_without_fn_keyword.ol new file mode 100644 index 0000000..f1ad57e --- /dev/null +++ b/tests/integration/tests/0002_fn_without_fn_keyword.ol @@ -0,0 +1,8 @@ +# spec: An invalid program with no fn keyword +main(): u32 { + return 0 +} +# compiler_exit: 1 +# compiler_output: ./tests/0002_fn_without_fn_keyword.ol:2:1: error: got 'main' token but expect <fn> +# compiler_output: main(): u32 { +# compiler_output: ^ diff --git a/tests/integration/tests/0003_fn_without_type.ol b/tests/integration/tests/0003_fn_without_type.ol new file mode 100644 index 0000000..32716e1 --- /dev/null +++ b/tests/integration/tests/0003_fn_without_type.ol @@ -0,0 +1,8 @@ +# spec: An invalid program with no type +fn main(): { + return 0 +} +# compiler_exit: 1 +# compiler_output: ./tests/0003_fn_without_type.ol:2:12: error: got '{' token but expect <identifier> +# compiler_output: fn main(): { +# compiler_output: ^ diff --git a/tests/integration/tests/0004_fn_without_type_colon.ol b/tests/integration/tests/0004_fn_without_type_colon.ol new file mode 100644 index 0000000..bd04c37 --- /dev/null +++ b/tests/integration/tests/0004_fn_without_type_colon.ol @@ -0,0 +1,8 @@ +# spec: An invalid program with no type colon +fn main() u32 { + return 0 +} +# compiler_exit: 1 +# compiler_output: ./tests/0004_fn_without_type_colon.ol:2:11: error: got 'u32' token but expect <:> +# compiler_output: fn main() u32 { +# compiler_output: ^ diff --git a/tests/integration/tests/0005_fn_without_open_curly_brackets.ol b/tests/integration/tests/0005_fn_without_open_curly_brackets.ol new file mode 100644 index 0000000..8e13946 --- /dev/null +++ b/tests/integration/tests/0005_fn_without_open_curly_brackets.ol @@ -0,0 +1,8 @@ +# spec: An invalid program with no open curly brackets +fn main(): u32 + return 0 +} +# compiler_exit: 1 +# compiler_output: ./tests/0005_fn_without_open_curly_brackets.ol:3:3: error: got 'return' token but expect <{> +# compiler_output: return 0 +# compiler_output: ^ diff --git a/tests/integration/tests/0006_fn_without_close_curly_brackets.ol b/tests/integration/tests/0006_fn_without_close_curly_brackets.ol new file mode 100644 index 0000000..d43c9f9 --- /dev/null +++ b/tests/integration/tests/0006_fn_without_close_curly_brackets.ol @@ -0,0 +1,7 @@ +# spec: An invalid program with no close curly brackets +fn main(): u32 { + return 0 +# compiler_exit: 1 +# compiler_output: ./tests/0006_fn_without_close_curly_brackets.ol:8:1: error: got '' token but expect <}> +# compiler_output: +# compiler_output: ^ diff --git a/tests/integration/tests/0007_fn_without_return.ol b/tests/integration/tests/0007_fn_without_return.ol new file mode 100644 index 0000000..fe33d69 --- /dev/null +++ b/tests/integration/tests/0007_fn_without_return.ol @@ -0,0 +1,8 @@ +# spec: An invalid program without return statment +# TODO: Improve the compiler error message +fn main(): u32 { +} +# compiler_exit: 1 +# compiler_output: ./tests/0007_fn_without_return.ol:4:1: error: got '}' token but expect <return> +# compiler_output: } +# compiler_output: ^ -- 2.34.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH olang 4/4] tests: print integration tests TODOs 2024-05-12 14:30 [PATCH olang 0/4] comment based integration tests Carlos Maniero ` (2 preceding siblings ...) 2024-05-12 14:30 ` [PATCH olang 3/4] tests: include integration tests for function parser errors Carlos Maniero @ 2024-05-12 14:30 ` Carlos Maniero 2024-05-12 14:31 ` [olang/patches/.build.yml] build success builds.sr.ht 2024-08-09 19:42 ` [PATCH olang 0/4] comment based integration tests Johnny Richard 2024-08-21 15:58 ` Johnny Richard 5 siblings, 1 reply; 8+ messages in thread From: Carlos Maniero @ 2024-05-12 14:30 UTC (permalink / raw) To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero Sometimes people use to ask how to contribute with olang. This is a great way to help them, we can just create TODOS on the integration tests as we already have some parts of the language specified. Signed-off-by: Carlos Maniero <carlos@maniero.me> --- tests/integration/test.sh | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/integration/test.sh b/tests/integration/test.sh index 9fdabef..c8535a7 100755 --- a/tests/integration/test.sh +++ b/tests/integration/test.sh @@ -56,6 +56,14 @@ print_test_description() { printf "\n\n" } +print_todo() { + TODOS=$(extract_comment TODO) + + if [ -n "$TODOS" ]; then + colored "TODO: $(extract_comment TODO)\n\n" $COLOR_YELLOW + fi +} + print_passed() { context="$1" printf "%s: " "$context" @@ -174,6 +182,7 @@ test_program() { main() { print_test_description + print_todo test_compiler test_program -- 2.34.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [olang/patches/.build.yml] build success 2024-05-12 14:30 ` [PATCH olang 4/4] tests: print integration tests TODOs Carlos Maniero @ 2024-05-12 14:31 ` builds.sr.ht 0 siblings, 0 replies; 8+ messages in thread From: builds.sr.ht @ 2024-05-12 14:31 UTC (permalink / raw) To: Carlos Maniero; +Cc: ~johnnyrichard/olang-devel olang/patches/.build.yml: SUCCESS in 51s [comment based integration tests][0] from [Carlos Maniero][1] [0]: https://lists.sr.ht/~johnnyrichard/olang-devel/patches/51804 [1]: mailto:carlos@maniero.me ✓ #1219016 SUCCESS olang/patches/.build.yml https://builds.sr.ht/~johnnyrichard/job/1219016 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH olang 0/4] comment based integration tests 2024-05-12 14:30 [PATCH olang 0/4] comment based integration tests Carlos Maniero ` (3 preceding siblings ...) 2024-05-12 14:30 ` [PATCH olang 4/4] tests: print integration tests TODOs Carlos Maniero @ 2024-08-09 19:42 ` Johnny Richard 2024-08-21 15:58 ` Johnny Richard 5 siblings, 0 replies; 8+ messages in thread From: Johnny Richard @ 2024-08-09 19:42 UTC (permalink / raw) To: Carlos Maniero; +Cc: ~johnnyrichard/olang-devel Thanks a lot for this contribution, I think it will improve the development experience and definitely will help us to move forward faster. I have only a small adjustment to suggest, actually a nitpick. On Sun, May 12, 2024 at 11:30:29AM GMT, Carlos Maniero wrote: > The old munit based integration tests was replaced by a new mechanism > that makes testing more tasteful and easy to perform. > > To create an integration tests you just need to create a olang file at > *tests/integration/tests* directory. The assertions are performed > thought a couple of comments. The folder *tests/integration/tests* feels a little bit redundant. What do you think about changing the folder structure to a smaller one like *test/integration*? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH olang 0/4] comment based integration tests 2024-05-12 14:30 [PATCH olang 0/4] comment based integration tests Carlos Maniero ` (4 preceding siblings ...) 2024-08-09 19:42 ` [PATCH olang 0/4] comment based integration tests Johnny Richard @ 2024-08-21 15:58 ` Johnny Richard 5 siblings, 0 replies; 8+ messages in thread From: Johnny Richard @ 2024-08-21 15:58 UTC (permalink / raw) To: Carlos Maniero; +Cc: ~johnnyrichard/olang-devel This patch has been rejected. Actually there is another patch which has superseded this one. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-08-21 15:58 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-05-12 14:30 [PATCH olang 0/4] comment based integration tests Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 1/4] tests: add comment based integration tests mechanism Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 2/4] tests: remove previous integration tests structure Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 3/4] tests: include integration tests for function parser errors Carlos Maniero 2024-05-12 14:30 ` [PATCH olang 4/4] tests: print integration tests TODOs Carlos Maniero 2024-05-12 14:31 ` [olang/patches/.build.yml] build success builds.sr.ht 2024-08-09 19:42 ` [PATCH olang 0/4] comment based integration tests Johnny Richard 2024-08-21 15:58 ` Johnny Richard
Code repositories for project(s) associated with this public inbox https://git.johnnyrichard.com/olang.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox