* [PATCH olang v1 1/2] tests: build: enhance compiler functional tests
2024-09-21 21:02 [PATCH olang v1 0/2] tests: build: improve makefile for tests Johnny Richard
@ 2024-09-21 21:02 ` Johnny Richard
2024-09-21 22:11 ` [PATCH olang v1 1/2] tests: build: enhance compiler functional test Carlos Maniero
2024-09-21 21:02 ` [PATCH olang v1 2/2] tests: build: add parallelization support for unit tests Johnny Richard
2024-09-22 15:23 ` [PATCH olang v1 0/2] tests: build: improve makefile for tests Johnny Richard
2 siblings, 1 reply; 7+ messages in thread
From: Johnny Richard @ 2024-09-21 21:02 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
There was two problem with the current state of our integration tests:
1) The folder structure is very annoying and long to navigate, the last
tests folder mess with the shell autocomplete because it has almost
the same name as the tests.sh script.
2) When we run the make all for integration tests, we are not using the
make how it was intended to be used. We have inline shell script
that loop over all tests and we cannot run tests in parallel.
This change will enable the parallelization and reduce the folder
structure so we can have a better development experience.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
Makefile | 8 +++----
.../tests => execute}/0001_main_exit.ol | 24 +++++++++----------
.../0002_binary_operator_addition.ol | 0
.../0003_binary_operator_multiplication.ol | 0
.../0004_binary_operator_division.ol | 0
.../0005_binary_operator_reminder.ol | 0
.../0006_binary_operator_subtraction.ol | 0
.../0007_binary_operator_eq.ol | 0
.../0008_binary_operator_lt.ol | 0
.../0009_binary_operator_gt.ol | 0
.../0010_binary_operator_neq.ol | 0
.../0011_binary_operator_leq.ol | 0
.../0012_binary_operator_geq.ol | 0
.../0013_binary_operator_lshift.ol | 0
.../0014_binary_operator_rshift.ol | 0
.../0015_binary_operator_xor.ol | 0
.../0016_binary_operator_and.ol | 0
.../0017_binary_operator_or.ol | 0
.../0018_binary_operator_logical_and.ol | 0
.../0019_binary_operator_logical_or.ol | 0
.../tests => execute}/0020_if_statement.ol | 2 +-
.../0021_if_statement_failed.ol | 0
.../0022_if_statement_literal.ol | 0
.../tests => execute}/0023_else_statement.ol | 2 +-
.../tests => execute}/0024_var_definition.ol | 2 +-
.../0025_var_definition_nested.ol | 0
tests/execute/Makefile | 15 ++++++++++++
tests/{integration/test.sh => execute/run.sh} | 0
tests/integration/Makefile | 9 -------
29 files changed, 34 insertions(+), 28 deletions(-)
rename tests/{integration/tests => execute}/0001_main_exit.ol (66%)
rename tests/{integration/tests => execute}/0002_binary_operator_addition.ol (100%)
rename tests/{integration/tests => execute}/0003_binary_operator_multiplication.ol (100%)
rename tests/{integration/tests => execute}/0004_binary_operator_division.ol (100%)
rename tests/{integration/tests => execute}/0005_binary_operator_reminder.ol (100%)
rename tests/{integration/tests => execute}/0006_binary_operator_subtraction.ol (100%)
rename tests/{integration/tests => execute}/0007_binary_operator_eq.ol (100%)
rename tests/{integration/tests => execute}/0008_binary_operator_lt.ol (100%)
rename tests/{integration/tests => execute}/0009_binary_operator_gt.ol (100%)
rename tests/{integration/tests => execute}/0010_binary_operator_neq.ol (100%)
rename tests/{integration/tests => execute}/0011_binary_operator_leq.ol (100%)
rename tests/{integration/tests => execute}/0012_binary_operator_geq.ol (100%)
rename tests/{integration/tests => execute}/0013_binary_operator_lshift.ol (100%)
rename tests/{integration/tests => execute}/0014_binary_operator_rshift.ol (100%)
rename tests/{integration/tests => execute}/0015_binary_operator_xor.ol (100%)
rename tests/{integration/tests => execute}/0016_binary_operator_and.ol (100%)
rename tests/{integration/tests => execute}/0017_binary_operator_or.ol (100%)
rename tests/{integration/tests => execute}/0018_binary_operator_logical_and.ol (100%)
rename tests/{integration/tests => execute}/0019_binary_operator_logical_or.ol (100%)
rename tests/{integration/tests => execute}/0020_if_statement.ol (96%)
rename tests/{integration/tests => execute}/0021_if_statement_failed.ol (100%)
rename tests/{integration/tests => execute}/0022_if_statement_literal.ol (100%)
rename tests/{integration/tests => execute}/0023_else_statement.ol (96%)
rename tests/{integration/tests => execute}/0024_var_definition.ol (96%)
rename tests/{integration/tests => execute}/0025_var_definition_nested.ol (100%)
create mode 100644 tests/execute/Makefile
rename tests/{integration/test.sh => execute/run.sh} (100%)
delete mode 100644 tests/integration/Makefile
diff --git a/Makefile b/Makefile
index 12b09a4..2b65f96 100644
--- a/Makefile
+++ b/Makefile
@@ -89,10 +89,10 @@ format-fix: $(SRCS) $(HEADERS)
clang-format -i $?
$(MAKE) -C tests/unit/ format-fix
-.PHONY: integration-test
-integration-test:
+.PHONY: check-execute
+check-execute:
$(MAKE)
- $(MAKE) -C tests/integration/
+ $(MAKE) -C tests/execute/
.PHONY: unit-test
unit-test:
@@ -108,7 +108,7 @@ clean:
.PHONY: check
check:
$(MAKE)
- $(MAKE) -C tests/integration/
+ $(MAKE) -C tests/execute/
$(MAKE) -C tests/unit/
.PHONY: docs
diff --git a/tests/integration/tests/0001_main_exit.ol b/tests/execute/0001_main_exit.ol
similarity index 66%
rename from tests/integration/tests/0001_main_exit.ol
rename to tests/execute/0001_main_exit.ol
index f446eb9..69e5ed1 100644
--- a/tests/integration/tests/0001_main_exit.ol
+++ b/tests/execute/0001_main_exit.ol
@@ -31,16 +31,16 @@ fn main(): u32 {
# END
#
# TEST test_contains_tokens WITH
-# ./tests/0001_main_exit.ol:17:1: <fn>
-# ./tests/0001_main_exit.ol:17:4: <identifier>
-# ./tests/0001_main_exit.ol:17:8: <(>
-# ./tests/0001_main_exit.ol:17:9: <)>
-# ./tests/0001_main_exit.ol:17:10: <:>
-# ./tests/0001_main_exit.ol:17:12: <identifier>
-# ./tests/0001_main_exit.ol:17:16: <{>
-# ./tests/0001_main_exit.ol:17:17: <line_feed>
-# ./tests/0001_main_exit.ol:18:3: <return>
-# ./tests/0001_main_exit.ol:18:10: <number>
-# ./tests/0001_main_exit.ol:18:11: <line_feed>
-# ./tests/0001_main_exit.ol:19:1: <}>
+# ./0001_main_exit.ol:17:1: <fn>
+# ./0001_main_exit.ol:17:4: <identifier>
+# ./0001_main_exit.ol:17:8: <(>
+# ./0001_main_exit.ol:17:9: <)>
+# ./0001_main_exit.ol:17:10: <:>
+# ./0001_main_exit.ol:17:12: <identifier>
+# ./0001_main_exit.ol:17:16: <{>
+# ./0001_main_exit.ol:17:17: <line_feed>
+# ./0001_main_exit.ol:18:3: <return>
+# ./0001_main_exit.ol:18:10: <number>
+# ./0001_main_exit.ol:18:11: <line_feed>
+# ./0001_main_exit.ol:19:1: <}>
# END
diff --git a/tests/integration/tests/0002_binary_operator_addition.ol b/tests/execute/0002_binary_operator_addition.ol
similarity index 100%
rename from tests/integration/tests/0002_binary_operator_addition.ol
rename to tests/execute/0002_binary_operator_addition.ol
diff --git a/tests/integration/tests/0003_binary_operator_multiplication.ol b/tests/execute/0003_binary_operator_multiplication.ol
similarity index 100%
rename from tests/integration/tests/0003_binary_operator_multiplication.ol
rename to tests/execute/0003_binary_operator_multiplication.ol
diff --git a/tests/integration/tests/0004_binary_operator_division.ol b/tests/execute/0004_binary_operator_division.ol
similarity index 100%
rename from tests/integration/tests/0004_binary_operator_division.ol
rename to tests/execute/0004_binary_operator_division.ol
diff --git a/tests/integration/tests/0005_binary_operator_reminder.ol b/tests/execute/0005_binary_operator_reminder.ol
similarity index 100%
rename from tests/integration/tests/0005_binary_operator_reminder.ol
rename to tests/execute/0005_binary_operator_reminder.ol
diff --git a/tests/integration/tests/0006_binary_operator_subtraction.ol b/tests/execute/0006_binary_operator_subtraction.ol
similarity index 100%
rename from tests/integration/tests/0006_binary_operator_subtraction.ol
rename to tests/execute/0006_binary_operator_subtraction.ol
diff --git a/tests/integration/tests/0007_binary_operator_eq.ol b/tests/execute/0007_binary_operator_eq.ol
similarity index 100%
rename from tests/integration/tests/0007_binary_operator_eq.ol
rename to tests/execute/0007_binary_operator_eq.ol
diff --git a/tests/integration/tests/0008_binary_operator_lt.ol b/tests/execute/0008_binary_operator_lt.ol
similarity index 100%
rename from tests/integration/tests/0008_binary_operator_lt.ol
rename to tests/execute/0008_binary_operator_lt.ol
diff --git a/tests/integration/tests/0009_binary_operator_gt.ol b/tests/execute/0009_binary_operator_gt.ol
similarity index 100%
rename from tests/integration/tests/0009_binary_operator_gt.ol
rename to tests/execute/0009_binary_operator_gt.ol
diff --git a/tests/integration/tests/0010_binary_operator_neq.ol b/tests/execute/0010_binary_operator_neq.ol
similarity index 100%
rename from tests/integration/tests/0010_binary_operator_neq.ol
rename to tests/execute/0010_binary_operator_neq.ol
diff --git a/tests/integration/tests/0011_binary_operator_leq.ol b/tests/execute/0011_binary_operator_leq.ol
similarity index 100%
rename from tests/integration/tests/0011_binary_operator_leq.ol
rename to tests/execute/0011_binary_operator_leq.ol
diff --git a/tests/integration/tests/0012_binary_operator_geq.ol b/tests/execute/0012_binary_operator_geq.ol
similarity index 100%
rename from tests/integration/tests/0012_binary_operator_geq.ol
rename to tests/execute/0012_binary_operator_geq.ol
diff --git a/tests/integration/tests/0013_binary_operator_lshift.ol b/tests/execute/0013_binary_operator_lshift.ol
similarity index 100%
rename from tests/integration/tests/0013_binary_operator_lshift.ol
rename to tests/execute/0013_binary_operator_lshift.ol
diff --git a/tests/integration/tests/0014_binary_operator_rshift.ol b/tests/execute/0014_binary_operator_rshift.ol
similarity index 100%
rename from tests/integration/tests/0014_binary_operator_rshift.ol
rename to tests/execute/0014_binary_operator_rshift.ol
diff --git a/tests/integration/tests/0015_binary_operator_xor.ol b/tests/execute/0015_binary_operator_xor.ol
similarity index 100%
rename from tests/integration/tests/0015_binary_operator_xor.ol
rename to tests/execute/0015_binary_operator_xor.ol
diff --git a/tests/integration/tests/0016_binary_operator_and.ol b/tests/execute/0016_binary_operator_and.ol
similarity index 100%
rename from tests/integration/tests/0016_binary_operator_and.ol
rename to tests/execute/0016_binary_operator_and.ol
diff --git a/tests/integration/tests/0017_binary_operator_or.ol b/tests/execute/0017_binary_operator_or.ol
similarity index 100%
rename from tests/integration/tests/0017_binary_operator_or.ol
rename to tests/execute/0017_binary_operator_or.ol
diff --git a/tests/integration/tests/0018_binary_operator_logical_and.ol b/tests/execute/0018_binary_operator_logical_and.ol
similarity index 100%
rename from tests/integration/tests/0018_binary_operator_logical_and.ol
rename to tests/execute/0018_binary_operator_logical_and.ol
diff --git a/tests/integration/tests/0019_binary_operator_logical_or.ol b/tests/execute/0019_binary_operator_logical_or.ol
similarity index 100%
rename from tests/integration/tests/0019_binary_operator_logical_or.ol
rename to tests/execute/0019_binary_operator_logical_or.ol
diff --git a/tests/integration/tests/0020_if_statement.ol b/tests/execute/0020_if_statement.ol
similarity index 96%
rename from tests/integration/tests/0020_if_statement.ol
rename to tests/execute/0020_if_statement.ol
index d48122f..ece9450 100644
--- a/tests/integration/tests/0020_if_statement.ol
+++ b/tests/execute/0020_if_statement.ol
@@ -24,7 +24,7 @@ fn main(): u32 {
# TEST test_run_binary(exit_code=0)
#
# TEST test_contains_tokens WITH
-# ./tests/0020_if_statement.ol:17:3: <if>
+# ./0020_if_statement.ol:17:3: <if>
# END
# TEST test_ast WITH
diff --git a/tests/integration/tests/0021_if_statement_failed.ol b/tests/execute/0021_if_statement_failed.ol
similarity index 100%
rename from tests/integration/tests/0021_if_statement_failed.ol
rename to tests/execute/0021_if_statement_failed.ol
diff --git a/tests/integration/tests/0022_if_statement_literal.ol b/tests/execute/0022_if_statement_literal.ol
similarity index 100%
rename from tests/integration/tests/0022_if_statement_literal.ol
rename to tests/execute/0022_if_statement_literal.ol
diff --git a/tests/integration/tests/0023_else_statement.ol b/tests/execute/0023_else_statement.ol
similarity index 96%
rename from tests/integration/tests/0023_else_statement.ol
rename to tests/execute/0023_else_statement.ol
index fafb22e..539ff95 100644
--- a/tests/integration/tests/0023_else_statement.ol
+++ b/tests/execute/0023_else_statement.ol
@@ -26,7 +26,7 @@ fn main(): u32 {
# TEST test_run_binary(exit_code=0)
# TEST test_contains_tokens WITH
-# ./tests/0023_else_statement.ol:19:5: <else>
+# ./0023_else_statement.ol:19:5: <else>
# END
# TEST test_ast WITH
diff --git a/tests/integration/tests/0024_var_definition.ol b/tests/execute/0024_var_definition.ol
similarity index 96%
rename from tests/integration/tests/0024_var_definition.ol
rename to tests/execute/0024_var_definition.ol
index 5c23449..3f09fa5 100644
--- a/tests/integration/tests/0024_var_definition.ol
+++ b/tests/execute/0024_var_definition.ol
@@ -23,7 +23,7 @@ fn main(): u32 {
# TEST test_run_binary(exit_code=0)
# TEST test_contains_tokens WITH
-# ./tests/0024_var_definition.ol:17:3: <var>
+# ./0024_var_definition.ol:17:3: <var>
# END
# TEST test_ast WITH
diff --git a/tests/integration/tests/0025_var_definition_nested.ol b/tests/execute/0025_var_definition_nested.ol
similarity index 100%
rename from tests/integration/tests/0025_var_definition_nested.ol
rename to tests/execute/0025_var_definition_nested.ol
diff --git a/tests/execute/Makefile b/tests/execute/Makefile
new file mode 100644
index 0000000..c736a78
--- /dev/null
+++ b/tests/execute/Makefile
@@ -0,0 +1,15 @@
+.POSIX:
+
+RUNNER := ./run.sh
+SRCS := $(wildcard *.ol)
+TESTS := $(patsubst %.ol, %.run, $(SRCS))
+
+.SUFFIXES:
+.SUFFIXES: .ol .run
+
+.PHONY: all
+all: $(TESTS)
+
+.ol.run:
+ @$(RUNNER) ./$<
+
diff --git a/tests/integration/test.sh b/tests/execute/run.sh
similarity index 100%
rename from tests/integration/test.sh
rename to tests/execute/run.sh
diff --git a/tests/integration/Makefile b/tests/integration/Makefile
deleted file mode 100644
index bda51af..0000000
--- a/tests/integration/Makefile
+++ /dev/null
@@ -1,9 +0,0 @@
-TESTER_SRC := ./test.sh
-TESTS := $(wildcard ./tests/*.ol)
-
-.PHONY: all
-all:
- @set -e; \
- for file in $(TESTS); do \
- $(TESTER_SRC) "$$file"; \
- done
--
2.46.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH olang v1 2/2] tests: build: add parallelization support for unit tests
2024-09-21 21:02 [PATCH olang v1 0/2] tests: build: improve makefile for tests Johnny Richard
2024-09-21 21:02 ` [PATCH olang v1 1/2] tests: build: enhance compiler functional tests Johnny Richard
@ 2024-09-21 21:02 ` Johnny Richard
2024-09-21 21:05 ` [olang/patches/.build.yml] build success builds.sr.ht
2024-09-21 22:30 ` [PATCH olang v1 2/2] tests: build: add parallelization support for unit tests Carlos Maniero
2024-09-22 15:23 ` [PATCH olang v1 0/2] tests: build: improve makefile for tests Johnny Richard
2 siblings, 2 replies; 7+ messages in thread
From: Johnny Richard @ 2024-09-21 21:02 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
Removes inline shell script and enable test execution by target on make.
Now we should be able to execute tests with more jobs (make -j<n-of-jobs>)
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
Makefile | 4 ++--
tests/unit/Makefile | 40 ++++++++++++++++++----------------------
2 files changed, 20 insertions(+), 24 deletions(-)
diff --git a/Makefile b/Makefile
index 2b65f96..83c88a2 100644
--- a/Makefile
+++ b/Makefile
@@ -94,8 +94,8 @@ check-execute:
$(MAKE)
$(MAKE) -C tests/execute/
-.PHONY: unit-test
-unit-test:
+.PHONY: check-unit
+check-unit:
$(MAKE)
$(MAKE) -C tests/unit/
diff --git a/tests/unit/Makefile b/tests/unit/Makefile
index 783225c..686938f 100644
--- a/tests/unit/Makefile
+++ b/tests/unit/Makefile
@@ -1,33 +1,29 @@
-SRCS := $(wildcard *_test.c)
-OBJS := $(patsubst %_test.c, %_test.o, $(SRCS))
-SUBJECT_OBJS := $(filter-out ../../build/main.o, $(wildcard ../../build/*.o))
-CFLAGS := -I../../src -I../shared
-TESTS := $(patsubst %_test.c, %_test, $(SRCS))
-EXEC_TESTS := $(patsubst %_test, ./%_test, $(TESTS))
-MUNIT_SRC := ../shared/munit.c
-MUNIT := ./munit.o
+SRCS := $(wildcard *.c)
+DEP_OBJS := $(filter-out ../../build/main.o, $(wildcard ../../build/*.o))
+CFLAGS := -I../../src -I../shared
+TESTS := $(patsubst %.c, %.bin, $(SRCS))
+RUN_TESTS := $(patsubst %.bin, %.run, $(TESTS))
+MUNIT_SRC := ../shared/munit.c
+MUNIT := ./munit.o
-.PHONY: all
-all: $(MUNIT) $(TESTS)
- @for file in $(EXEC_TESTS); do \
- ./"$$file"; \
- done
+.PHONY: all clean format format-fix
+all: $(RUN_TESTS)
+
+%.bin: %.c $(MUNIT)
+ @$(CC) $(CFLAGS) $(MUNIT) $(DEP_OBJS) $< -o $@
+
+%.run: %.bin
+ @./$<
-.PHONY: clean
clean:
- $(RM) *.o *_test
- $(RM) -rfv lib
+ @$(RM) *.o *.bin
+ @$(RM) -rfv lib
-.PHONY: format
format: $(SRCS)
clang-format --dry-run --Werror $?
-.PHONY: format-fix
format-fix: $(SRCS)
clang-format -i $?
-%_test: $(MUNIT) $(SUBJECT_OBJS) %_test.c
- $(CC) $? $(CFLAGS) -o $@
-
$(MUNIT):
- $(CC) -c $(MUNIT_SRC) $(CFLAGS) -o $(MUNIT)
+ @$(CC) -c $(MUNIT_SRC) $(CFLAGS) -o $(MUNIT)
--
2.46.0
^ permalink raw reply [flat|nested] 7+ messages in thread