* [PATCH olang] fix: codegen: prevent stack overwrite
@ 2024-10-15 12:14 Carlos Maniero
2024-10-15 12:14 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-10-16 22:33 ` [PATCH olang] fix: codegen: prevent stack overwrite Johnny Richard
0 siblings, 2 replies; 35+ messages in thread
From: Carlos Maniero @ 2024-10-15 12:14 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
There was an issue in the stack allocation algorithm. Consider this
function:
fn a(): u32 {
var a: u32 = 0xAAAA
var b: u64 = 0xBBBBBBBB
ret
}
There are three information the stack is required to store:
- 8 bytes: rip (from call instruction)
- 4 bytes: a
- 8 bytes: b
The 0x7FFFFF07 memory address was used to represent the RIP value at
call instant.
Our codegen was assuming the stack works that way:
0 -8 -C
^-------^---^-------
7FFFFF07AAAABBBBBBBB
^-------^---^-------
rip a b
So the code gen was:
- Adding the value at the stack;
- Increasing the offset.
But actually the stack was behaving as following:
8 0 -8 -C
^-------^-------^---^
7FFFFF070000BBBBBBBB.
^---------------^---^
rip a b
Once the instruction *mov %rax, -0xC(%rbp)* writes from -0xC(%rbp)
(exclusive) to -0x4(%rbp) (inclusive).
So after this change, this is the actual stack template:
0 -4 -C
--------^---^-------^
7FFFFF07AAAABBBBBBBB.
--------^---^-------^
rip a b
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
src/codegen_linux_x86_64.c | 13 ++++++-------
tests/olc/0036_variable_overflow.ol | 30 +++++++++++++++++++++++++++++
2 files changed, 36 insertions(+), 7 deletions(-)
create mode 100644 tests/olc/0036_variable_overflow.ol
diff --git a/src/codegen_linux_x86_64.c b/src/codegen_linux_x86_64.c
index fc8fcc4..83d1d2c 100644
--- a/src/codegen_linux_x86_64.c
+++ b/src/codegen_linux_x86_64.c
@@ -28,7 +28,6 @@
// The call instruction pushes EIP into stack so the first 8 bytes from stack
// must be preserved else the ret instruction will jump to nowere.
-#define X86_CALL_EIP_STACK_OFFSET (8)
#define X86_CALL_ARG_SIZE 6
#define bytes_max(a, b) ((a) > (b) ? (a) : (b))
@@ -795,6 +794,9 @@ codegen_linux_x86_64_emit_block(codegen_x86_64_t *codegen, ast_block_t *block)
symbol_t *symbol = scope_lookup(scope, var_def.id);
assert(symbol);
+ size_t type_size = type_to_bytes(symbol->type);
+ codegen->base_offset += type_size;
+
codegen_linux_x86_64_put_stack_offset(
codegen, symbol, codegen->base_offset);
@@ -803,13 +805,10 @@ codegen_linux_x86_64_emit_block(codegen_x86_64_t *codegen, ast_block_t *block)
var_def.value);
}
- size_t type_size = type_to_bytes(symbol->type);
-
fprintf(codegen->out,
" mov %s, -%ld(%%rbp)\n",
get_reg_for(REG_ACCUMULATOR, type_size),
codegen->base_offset);
- codegen->base_offset += type_size;
break;
}
@@ -957,7 +956,7 @@ static void
codegen_linux_x86_64_emit_function(codegen_x86_64_t *codegen,
ast_fn_definition_t *fn_def)
{
- codegen->base_offset = X86_CALL_EIP_STACK_OFFSET;
+ codegen->base_offset = 0;
ast_node_t *block_node = fn_def->block;
fprintf(codegen->out, "" SV_FMT ":\n", SV_ARG(fn_def->id));
@@ -975,6 +974,8 @@ codegen_linux_x86_64_emit_function(codegen_x86_64_t *codegen,
symbol_t *symbol = scope_lookup(fn_def->scope, param->id);
assert(symbol);
+ // FIXME: add offset according to the param size
+ codegen->base_offset += 8;
size_t offset = codegen->base_offset;
codegen_linux_x86_64_put_stack_offset(
@@ -986,8 +987,6 @@ codegen_linux_x86_64_emit_function(codegen_x86_64_t *codegen,
get_reg_for(x86_call_args[i], symbol->type->as_primitive.size),
offset);
- // FIXME: add offset according to the param size
- codegen->base_offset += 8;
++i;
}
diff --git a/tests/olc/0036_variable_overflow.ol b/tests/olc/0036_variable_overflow.ol
new file mode 100644
index 0000000..edb3c7e
--- /dev/null
+++ b/tests/olc/0036_variable_overflow.ol
@@ -0,0 +1,30 @@
+# Copyright (C) 2024 olang mantainers
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+fn main(): u32 {
+ var a: u32 = 0
+ var b: u64 = 0
+ var c: u32 = 0
+
+ # This operation will fill all bits in b location.
+ # If there is an overflow, both a or c would be impacted
+ b = ~b
+
+ return a + c
+}
+
+# TEST test_compile(exit_code=0)
+
+# TEST test_run_binary(exit_code=0)
base-commit: cf5e4abf07a38f0ddf3ac6979b01b942ab99a691
--
2.46.1
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v1 1/1] codegen: x64: deref returns pointer value
@ 2024-10-17 2:48 Carlos Maniero
2024-10-17 2:49 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-10-17 2:48 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
Deref is context dependent, when doing an assignment:
*a = 1
It is expected for the deref codegen to return the pointer location, so
than the assignment binop would be able to assign a new value at that
location.
On another hand, when performing:
return *a
It is expected for deref to actually returns the pointer location value.
The codegen of both behaviors were defined in new functions to avoid
indiscriminately increase the codegen_x86_64_emit_expression
switch/case.
Register choice:
================
Since the operation *mov (%rax), %rax* is not possible, I arbitrarily
choose R10 as it is a caller-saved register.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
src/codegen_x86_64.c | 61 +++++++++++++++++++++++++++++++++++++-------
1 file changed, 52 insertions(+), 9 deletions(-)
diff --git a/src/codegen_x86_64.c b/src/codegen_x86_64.c
index deb7e24..db1b77a 100644
--- a/src/codegen_x86_64.c
+++ b/src/codegen_x86_64.c
@@ -52,6 +52,8 @@ typedef enum x86_64_register_type
REG_R15
} x86_64_register_type_t;
+typedef size_t size_in_bytes_t;
+
/**
* Arch/ABI arg1 arg2 arg3 arg4 arg5 arg6 arg7 Notes
* ──────────────────────────────────────────────────────────────
@@ -76,6 +78,14 @@ codegen_x86_64_put_stack_offset(codegen_x86_64_t *codegen,
static size_t
codegen_x86_64_get_stack_offset(codegen_x86_64_t *codegen, symbol_t *symbol);
+static size_in_bytes_t
+codegen_x86_64_emit_unary_deref_address(codegen_x86_64_t *codegen,
+ ast_unary_op_t *unary_op);
+
+static size_in_bytes_t
+codegen_x86_64_emit_unary_deref_value(codegen_x86_64_t *codegen,
+ ast_unary_op_t *unary_op);
+
static size_t
type_to_bytes(type_t *type);
@@ -126,8 +136,6 @@ codegen_x86_64_get_next_label(codegen_x86_64_t *codegen)
return ++codegen->label_index;
}
-typedef size_t size_in_bytes_t;
-
static size_in_bytes_t
codegen_x86_64_emit_expression(codegen_x86_64_t *codegen, ast_node_t *expr_node)
{
@@ -619,7 +627,8 @@ codegen_x86_64_emit_expression(codegen_x86_64_t *codegen, ast_node_t *expr_node)
AST_UNARY_DEREFERENCE &&
"unsupported assignment lhs");
- codegen_x86_64_emit_expression(codegen, bin_op.lhs);
+ codegen_x86_64_emit_unary_deref_address(
+ codegen, &bin_op.lhs->as_unary_op);
fprintf(codegen->out, " push %%rax\n");
@@ -679,12 +688,8 @@ codegen_x86_64_emit_expression(codegen_x86_64_t *codegen, ast_node_t *expr_node)
return 8;
}
case AST_UNARY_DEREFERENCE: {
- // FIXME: support dereference of dereference (**)
- assert(unary_op.expr->kind == AST_NODE_REF &&
- "unsupported unary expression for dereference (*)");
-
- return codegen_x86_64_emit_expression(codegen,
- unary_op.expr);
+ return codegen_x86_64_emit_unary_deref_value(
+ codegen, &unary_op);
}
default: {
assert(0 && "unsupported unary operation");
@@ -829,6 +834,44 @@ codegen_x86_64_emit_if(codegen_x86_64_t *codegen, ast_if_stmt_t if_stmt)
fprintf(codegen->out, ".L%ld:\n", end_else_label);
}
+static size_in_bytes_t
+codegen_x86_64_emit_unary_deref_address(codegen_x86_64_t *codegen,
+ ast_unary_op_t *unary_op)
+{
+ assert(unary_op->kind == AST_UNARY_DEREFERENCE);
+ // FIXME: support dereference of dereference (**)
+ assert(unary_op->expr->kind == AST_NODE_REF &&
+ "unsupported unary expression for dereference (*)");
+
+ return codegen_x86_64_emit_expression(codegen, unary_op->expr);
+}
+
+static size_in_bytes_t
+codegen_x86_64_emit_unary_deref_value(codegen_x86_64_t *codegen,
+ ast_unary_op_t *unary_op)
+{
+ codegen_x86_64_emit_unary_deref_address(codegen, unary_op);
+
+ ast_ref_t ref = unary_op->expr->as_ref;
+
+ symbol_t *symbol = scope_lookup(ref.scope, ref.id);
+
+ assert(symbol->type->kind == TYPE_PTR);
+
+ size_in_bytes_t deref_size = type_to_bytes(symbol->type->as_ptr.type);
+
+ fprintf(codegen->out,
+ " mov (%%rax), %s\n",
+ get_reg_for(REG_R10, deref_size));
+
+ fprintf(codegen->out,
+ " mov %s, %s\n",
+ get_reg_for(REG_R10, deref_size),
+ get_reg_for(REG_ACCUMULATOR, deref_size));
+
+ return deref_size;
+}
+
static size_t
type_to_bytes(type_t *type)
{
--
2.46.1
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v1] fix: build: add missing dependencies for check-spec
@ 2024-10-11 3:42 Johnny Richard
2024-10-11 1:43 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-10-11 3:42 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
.build.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.build.yml b/.build.yml
index 9c8eaa5..f09153c 100644
--- a/.build.yml
+++ b/.build.yml
@@ -6,6 +6,8 @@ packages:
- hut
- clang
- texinfo
+ - clojure
+ - rlwrap
environment:
site: o-lang.org
sources:
base-commit: 9d94b76123df435b52365fbd82babbb66ec20839
--
2.46.0
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH olang] parser: conform block line feeds with the spec
@ 2024-10-08 16:33 Carlos Maniero
2024-10-08 16:33 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-10-08 16:33 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
The line feeds has been handled by the statement parser functions when
the olang spec describe this grammar rule as a block responsibility.
There was also a FIXME related to allowing line feeds in between the
if/else statement and the block start that was also addressed.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
src/parser.c | 47 ++++++++++++++-----------
| 30 ++++++++++++++++
2 files changed, 57 insertions(+), 20 deletions(-)
create mode 100644 tests/olc/0031_else_extra_line_feeds.ol
diff --git a/src/parser.c b/src/parser.c
index d16b79d..2096129 100644
--- a/src/parser.c
+++ b/src/parser.c
@@ -70,6 +70,9 @@ parser_parse_factor(parser_t *parser);
static void
skip_line_feeds(lexer_t *lexer);
+static void
+peek_next_non_lf_token(lexer_t *lexer, token_t *token);
+
void
parser_init(parser_t *parser, lexer_t *lexer, arena_t *arena)
{
@@ -493,6 +496,12 @@ StartLoop:
return NULL;
}
+ if (!skip_expected_token(parser, TOKEN_LF)) {
+ return NULL;
+ }
+
+ skip_line_feeds(parser->lexer);
+
list_append(node_block->as_block.nodes, node);
goto StartLoop;
@@ -522,11 +531,6 @@ parser_parse_return_stmt(parser_t *parser)
ast_node_t *node_return_stmt = ast_new_node_return_stmt(parser->arena, token_ret.loc, expr);
assert(node_return_stmt);
- if (!skip_expected_token(parser, TOKEN_LF)) {
- return NULL;
- }
- skip_line_feeds(parser->lexer);
-
return node_return_stmt;
}
@@ -544,6 +548,8 @@ parser_parse_if_stmt(parser_t *parser)
return NULL;
}
+ skip_line_feeds(parser->lexer);
+
ast_node_t *then = parser_parse_block(parser);
if (then == NULL) {
@@ -553,29 +559,25 @@ parser_parse_if_stmt(parser_t *parser)
ast_node_t *_else = NULL;
token_t next_token;
- lexer_next_token(parser->lexer, &next_token);
-
- // FIXME: We are not allowing line feed right after if block statement when
- // the else branch is present. We also noticed that if has the same
- // problem which will be addressed later.
+ peek_next_non_lf_token(parser->lexer, &next_token);
if (next_token.kind == TOKEN_ELSE) {
+ skip_line_feeds(parser->lexer);
+ lexer_next_token(parser->lexer, &next_token);
+ skip_line_feeds(parser->lexer);
+
_else = parser_parse_block(parser);
if (_else == NULL) {
return NULL;
}
- } else if (!expected_token(&next_token, TOKEN_LF)) {
- return NULL;
}
ast_node_t *node_if_stmt = ast_new_node_if_stmt(parser->arena, token_if.loc, cond, then, _else);
assert(node_if_stmt);
- skip_line_feeds(parser->lexer);
-
return node_if_stmt;
}
@@ -608,8 +610,6 @@ parser_parse_var_def(parser_t *parser)
ast_node_t *var_node = ast_new_node_var_def(parser->arena, token_id.loc, token_id.value, var_type, expr);
- skip_line_feeds(parser->lexer);
-
return var_node;
}
@@ -631,10 +631,6 @@ parser_parse_var_assign_stmt(parser_t *parser)
ast_node_t *ref = ast_new_node_ref(parser->arena, token_id.loc, token_id.value);
ast_node_t *expr = parser_parse_expr(parser);
- // FIXME: The expected line feed should be parsed from parent call
- // according to the grammar rules
- skip_line_feeds(parser->lexer);
-
return ast_new_node_var_assign_stmt(parser->arena, token_eq.loc, ref, expr);
}
@@ -683,3 +679,14 @@ skip_line_feeds(lexer_t *lexer)
lexer_peek_next(lexer, &token);
}
}
+
+static void
+peek_next_non_lf_token(lexer_t *lexer, token_t *token)
+{
+ lexer_cursor_t cur = lexer->cur;
+
+ skip_line_feeds(lexer);
+ lexer_peek_next(lexer, token);
+
+ lexer->cur = cur;
+}
--git a/tests/olc/0031_else_extra_line_feeds.ol b/tests/olc/0031_else_extra_line_feeds.ol
new file mode 100644
index 0000000..bc44ccd
--- /dev/null
+++ b/tests/olc/0031_else_extra_line_feeds.ol
@@ -0,0 +1,30 @@
+# Copyright (C) 2024 olang mantainers
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+fn main(): u32
+{
+ if 0 != 0
+ {
+ return 1
+ }
+ else
+ {
+ return 0
+ }
+}
+
+# TEST test_compile(exit_code=0)
+
+# TEST test_run_binary(exit_code=0)
base-commit: 3c8975ba27c87d084187eefe622cbd783e289c99
--
2.46.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v1 2/2] ast: inline ast_node_data_t union definition
@ 2024-08-13 18:16 Johnny Richard
2024-08-13 17:27 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-08-13 18:16 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
src/ast.c | 20 ++++++++++----------
src/ast.h | 20 +++++++++-----------
src/codegen_linux_aarch64.c | 10 +++++-----
src/codegen_linux_x86_64.c | 12 ++++++------
src/parser.c | 4 ++--
src/pretty_print_ast.c | 12 ++++++------
tests/unit/parser_test.c | 14 +++++++-------
7 files changed, 45 insertions(+), 47 deletions(-)
diff --git a/src/ast.c b/src/ast.c
index 1f7df9c..aa9e6db 100644
--- a/src/ast.c
+++ b/src/ast.c
@@ -29,7 +29,7 @@ ast_new_program(arena_t *arena, ast_node_t *fn_def)
assert(node);
node->kind = AST_NODE_PROGRAM;
- ast_program_t *program = &node->data.as_program;
+ ast_program_t *program = &node->as_program;
program->fn = fn_def;
@@ -43,7 +43,7 @@ ast_new_node_fn_def(arena_t *arena, string_view_t identifier, type_t return_type
assert(node_fn_def);
node_fn_def->kind = AST_NODE_FN_DEF;
- ast_fn_definition_t *fn_def = &node_fn_def->data.as_fn_def;
+ ast_fn_definition_t *fn_def = &node_fn_def->as_fn_def;
fn_def->identifier = identifier;
fn_def->return_type = return_type;
@@ -59,9 +59,9 @@ ast_new_node_bin_op(arena_t *arena, ast_binary_op_kind_t kind, ast_node_t *lhs,
assert(node_bin_op);
node_bin_op->kind = AST_NODE_BINARY_OP;
- node_bin_op->data.as_bin_op.kind = kind;
- node_bin_op->data.as_bin_op.lhs = lhs;
- node_bin_op->data.as_bin_op.rhs = rhs;
+ node_bin_op->as_bin_op.kind = kind;
+ node_bin_op->as_bin_op.lhs = lhs;
+ node_bin_op->as_bin_op.rhs = rhs;
return node_bin_op;
}
@@ -73,8 +73,8 @@ ast_new_node_literal_u32(arena_t *arena, uint32_t value)
assert(node_literal);
node_literal->kind = AST_NODE_LITERAL;
- node_literal->data.as_literal.kind = AST_LITERAL_U32;
- node_literal->data.as_literal.as_u32 = value;
+ node_literal->as_literal.kind = AST_LITERAL_U32;
+ node_literal->as_literal.as_u32 = value;
return node_literal;
}
@@ -98,10 +98,10 @@ ast_new_node_block(arena_t *arena)
node_block->kind = AST_NODE_BLOCK;
- node_block->data.as_block.nodes = (list_t *)arena_alloc(arena, sizeof(list_t));
- assert(node_block->data.as_block.nodes);
+ node_block->as_block.nodes = (list_t *)arena_alloc(arena, sizeof(list_t));
+ assert(node_block->as_block.nodes);
- list_init(node_block->data.as_block.nodes, arena);
+ list_init(node_block->as_block.nodes, arena);
return node_block;
}
diff --git a/src/ast.h b/src/ast.h
index a58a492..024f2cc 100644
--- a/src/ast.h
+++ b/src/ast.h
@@ -106,20 +106,18 @@ typedef struct ast_return_stmt
ast_node_t *data;
} ast_return_stmt_t;
-typedef union
-{
- ast_program_t as_program;
- ast_fn_definition_t as_fn_def;
- ast_binary_op_t as_bin_op;
- ast_literal_t as_literal;
- ast_block_t as_block;
- ast_return_stmt_t as_return_stmt;
-} ast_node_data_t;
-
typedef struct ast_node
{
ast_node_kind_t kind;
- ast_node_data_t data;
+ union
+ {
+ ast_program_t as_program;
+ ast_fn_definition_t as_fn_def;
+ ast_binary_op_t as_bin_op;
+ ast_literal_t as_literal;
+ ast_block_t as_block;
+ ast_return_stmt_t as_return_stmt;
+ };
} ast_node_t;
ast_node_t *
diff --git a/src/codegen_linux_aarch64.c b/src/codegen_linux_aarch64.c
index 73f4aab..dafdcc4 100644
--- a/src/codegen_linux_aarch64.c
+++ b/src/codegen_linux_aarch64.c
@@ -47,9 +47,9 @@ codegen_linux_aarch64_emit_program(FILE *out, ast_node_t *node)
codegen_linux_aarch64_emit_start_entrypoint(out);
assert(node->kind == AST_NODE_PROGRAM);
- ast_program_t program = node->data.as_program;
+ ast_program_t program = node->as_program;
- ast_fn_definition_t fn = program.fn->data.as_fn_def;
+ ast_fn_definition_t fn = program.fn->as_fn_def;
assert(string_view_eq_to_cstr(fn.identifier, "main"));
codegen_linux_aarch64_emit_function(out, &fn);
@@ -72,18 +72,18 @@ codegen_linux_aarch64_emit_function(FILE *out, ast_fn_definition_t *fn)
{
ast_node_t *block_node = fn->block;
assert(block_node->kind == AST_NODE_BLOCK);
- ast_block_t block = block_node->data.as_block;
+ ast_block_t block = block_node->as_block;
assert(list_size(block.nodes) == 1);
list_item_t *nodes_item = list_get(block.nodes, 0);
ast_node_t *return_node = nodes_item->value;
assert(return_node->kind == AST_NODE_RETURN_STMT);
- ast_return_stmt_t return_stmt = return_node->data.as_return_stmt;
+ ast_return_stmt_t return_stmt = return_node->as_return_stmt;
ast_node_t *literal_node = return_stmt.data;
assert(literal_node->kind == AST_NODE_LITERAL);
- ast_literal_t literal_u32 = literal_node->data.as_literal;
+ ast_literal_t literal_u32 = literal_node->as_literal;
assert(literal_u32.kind == AST_LITERAL_U32);
uint32_t exit_code = literal_u32.as_u32;
diff --git a/src/codegen_linux_x86_64.c b/src/codegen_linux_x86_64.c
index 28e7f8e..64ec0e0 100644
--- a/src/codegen_linux_x86_64.c
+++ b/src/codegen_linux_x86_64.c
@@ -38,9 +38,9 @@ codegen_linux_x86_64_emit_program(FILE *out, ast_node_t *node)
codegen_linux_x86_64_emit_start_entrypoint(out);
assert(node->kind == AST_NODE_PROGRAM);
- ast_program_t program = node->data.as_program;
+ ast_program_t program = node->as_program;
- ast_fn_definition_t fn = program.fn->data.as_fn_def;
+ ast_fn_definition_t fn = program.fn->as_fn_def;
assert(string_view_eq_to_cstr(fn.identifier, "main"));
codegen_linux_x86_64_emit_function(out, &fn);
@@ -70,7 +70,7 @@ codegen_linux_x86_64_emit_expression(FILE *out, ast_node_t *expr_node)
{
switch (expr_node->kind) {
case AST_NODE_LITERAL: {
- ast_literal_t literal_u32 = expr_node->data.as_literal;
+ ast_literal_t literal_u32 = expr_node->as_literal;
assert(literal_u32.kind == AST_LITERAL_U32);
uint32_t n = literal_u32.as_u32;
@@ -78,7 +78,7 @@ codegen_linux_x86_64_emit_expression(FILE *out, ast_node_t *expr_node)
return;
}
case AST_NODE_BINARY_OP: {
- ast_binary_op_t bin_op = expr_node->data.as_bin_op;
+ ast_binary_op_t bin_op = expr_node->as_bin_op;
switch (bin_op.kind) {
case AST_BINOP_ADDITION: {
codegen_linux_x86_64_emit_expression(out, bin_op.rhs);
@@ -305,14 +305,14 @@ codegen_linux_x86_64_emit_function(FILE *out, ast_fn_definition_t *fn)
{
ast_node_t *block_node = fn->block;
assert(block_node->kind == AST_NODE_BLOCK);
- ast_block_t block = block_node->data.as_block;
+ ast_block_t block = block_node->as_block;
assert(list_size(block.nodes) == 1);
list_item_t *nodes_item = list_get(block.nodes, 0);
ast_node_t *return_node = nodes_item->value;
assert(return_node->kind == AST_NODE_RETURN_STMT);
- ast_return_stmt_t return_stmt = return_node->data.as_return_stmt;
+ ast_return_stmt_t return_stmt = return_node->as_return_stmt;
ast_node_t *expr = return_stmt.data;
diff --git a/src/parser.c b/src/parser.c
index 5dd4ef1..24094b3 100644
--- a/src/parser.c
+++ b/src/parser.c
@@ -336,9 +336,9 @@ parser_parse_block(parser_t *parser)
return NULL;
}
- node_return_stmt->data.as_return_stmt.data = expr;
+ node_return_stmt->as_return_stmt.data = expr;
- list_append(node_block->data.as_block.nodes, node_return_stmt);
+ list_append(node_block->as_block.nodes, node_return_stmt);
if (!skip_expected_token(parser, TOKEN_LF)) {
return NULL;
}
diff --git a/src/pretty_print_ast.c b/src/pretty_print_ast.c
index 19ccafe..6ca172f 100644
--- a/src/pretty_print_ast.c
+++ b/src/pretty_print_ast.c
@@ -116,13 +116,13 @@ ast_node_to_pretty_print_node(ast_node_t *ast, arena_t *arena)
pretty_print_node_t *node = pretty_print_node_new(arena);
node->name = "Translation_Unit";
- pretty_print_node_t *fn_node = ast_node_to_pretty_print_node(ast->data.as_program.fn, arena);
+ pretty_print_node_t *fn_node = ast_node_to_pretty_print_node(ast->as_program.fn, arena);
list_append(node->children, fn_node);
return node;
}
case AST_NODE_FN_DEF: {
pretty_print_node_t *node = pretty_print_node_new(arena);
- ast_fn_definition_t fn_def = ast->data.as_fn_def;
+ ast_fn_definition_t fn_def = ast->as_fn_def;
char name[256];
sprintf(name,
@@ -138,7 +138,7 @@ ast_node_to_pretty_print_node(ast_node_t *ast, arena_t *arena)
}
case AST_NODE_BLOCK: {
pretty_print_node_t *node = pretty_print_node_new(arena);
- ast_block_t block = ast->data.as_block;
+ ast_block_t block = ast->as_block;
node->name = "Block";
@@ -152,7 +152,7 @@ ast_node_to_pretty_print_node(ast_node_t *ast, arena_t *arena)
}
case AST_NODE_RETURN_STMT: {
pretty_print_node_t *node = pretty_print_node_new(arena);
- ast_return_stmt_t return_stmt = ast->data.as_return_stmt;
+ ast_return_stmt_t return_stmt = ast->as_return_stmt;
node->name = "Return_Statement";
@@ -163,7 +163,7 @@ ast_node_to_pretty_print_node(ast_node_t *ast, arena_t *arena)
}
case AST_NODE_LITERAL: {
pretty_print_node_t *node = pretty_print_node_new(arena);
- ast_literal_t literal = ast->data.as_literal;
+ ast_literal_t literal = ast->as_literal;
char name[256];
switch (literal.kind) {
@@ -181,7 +181,7 @@ ast_node_to_pretty_print_node(ast_node_t *ast, arena_t *arena)
}
case AST_NODE_BINARY_OP: {
pretty_print_node_t *node = pretty_print_node_new(arena);
- ast_binary_op_t binop = ast->data.as_bin_op;
+ ast_binary_op_t binop = ast->as_bin_op;
switch (binop.kind) {
case AST_BINOP_ADDITION: {
diff --git a/tests/unit/parser_test.c b/tests/unit/parser_test.c
index 1925a95..3cdac41 100644
--- a/tests/unit/parser_test.c
+++ b/tests/unit/parser_test.c
@@ -45,11 +45,11 @@ parse_program_test(const MunitParameter params[], void *user_data_or_fixture)
assert_not_null(program_node);
assert_uint(program_node->kind, ==, AST_NODE_PROGRAM);
- ast_program_t program = program_node->data.as_program;
+ ast_program_t program = program_node->as_program;
assert_not_null(program.fn);
assert_uint(program.fn->kind, ==, AST_NODE_FN_DEF);
- ast_fn_definition_t fn = program.fn->data.as_fn_def;
+ ast_fn_definition_t fn = program.fn->as_fn_def;
assert_memory_equal(fn.identifier.size, fn.identifier.chars, "main");
assert_uint(fn.return_type, ==, TYPE_U32);
@@ -57,8 +57,8 @@ parse_program_test(const MunitParameter params[], void *user_data_or_fixture)
assert_not_null(block);
assert_uint(block->kind, ==, AST_NODE_BLOCK);
- assert_uint(list_size(block->data.as_block.nodes), ==, 1);
- list_item_t *block_item = list_get(block->data.as_block.nodes, 0);
+ assert_uint(list_size(block->as_block.nodes), ==, 1);
+ list_item_t *block_item = list_get(block->as_block.nodes, 0);
assert_not_null(block_item);
assert_not_null(block_item->value);
@@ -66,11 +66,11 @@ parse_program_test(const MunitParameter params[], void *user_data_or_fixture)
assert_not_null(node);
assert_uint(node->kind, ==, AST_NODE_RETURN_STMT);
- ast_node_t *number_node = node->data.as_return_stmt.data;
+ ast_node_t *number_node = node->as_return_stmt.data;
assert_not_null(number_node);
assert_uint(number_node->kind, ==, AST_NODE_LITERAL);
- assert_uint(number_node->data.as_literal.kind, ==, AST_LITERAL_U32);
- assert_uint(number_node->data.as_literal.as_u32, ==, 69);
+ assert_uint(number_node->as_literal.kind, ==, AST_LITERAL_U32);
+ assert_uint(number_node->as_literal.as_u32, ==, 69);
arena_free(&arena);
--
2.46.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v1] build: rename linter to format to avoid confusion
@ 2024-04-20 13:54 Johnny Richard
2024-04-20 12:57 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-04-20 13:54 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
We have been using linter as the target to check our code formatting.
Today we are using `clang-format` to verify the formatting. This tool
does only formatting, and miss other features that a linter have.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
.build.yml | 4 ++--
Makefile | 16 ++++++++--------
docs/pages/contribute.md | 16 ++++++++--------
tests/integration/Makefile | 8 ++++----
tests/unit/Makefile | 8 ++++----
5 files changed, 26 insertions(+), 26 deletions(-)
diff --git a/.build.yml b/.build.yml
index 72f0254..01f4aab 100644
--- a/.build.yml
+++ b/.build.yml
@@ -11,9 +11,9 @@ environment:
sources:
- https://git.sr.ht/~johnnyrichard/olang
tasks:
- - lint: |
+ - format: |
cd olang
- make linter
+ make format
- build: |
cd olang
make
diff --git a/Makefile b/Makefile
index cdfc8e1..27337d4 100644
--- a/Makefile
+++ b/Makefile
@@ -16,17 +16,17 @@ $(TARGET): $(BUILD_DIR) $(OBJS)
$(BUILD_DIR):
@mkdir -p $@
-.PHONY: linter
-linter: $(SRCS) $(HEADERS)
+.PHONY: format
+format: $(SRCS) $(HEADERS)
clang-format --dry-run --Werror $?
- $(MAKE) -C tests/integration/ linter
- $(MAKE) -C tests/unit/ linter
+ $(MAKE) -C tests/integration/ format
+ $(MAKE) -C tests/unit/ format
-.PHONY: linter-fix
-linter-fix: $(SRCS) $(HEADERS)
+.PHONY: format-fix
+format-fix: $(SRCS) $(HEADERS)
clang-format -i $?
- $(MAKE) -C tests/integration/ linter-fix
- $(MAKE) -C tests/unit/ linter-fix
+ $(MAKE) -C tests/integration/ format-fix
+ $(MAKE) -C tests/unit/ format-fix
.PHONY: integration-test
integration-test:
diff --git a/docs/pages/contribute.md b/docs/pages/contribute.md
index 884c4b4..a6dfd04 100644
--- a/docs/pages/contribute.md
+++ b/docs/pages/contribute.md
@@ -25,21 +25,21 @@ Code style
Instead of delineating every element of our coding style, we have
adopted the use of **clang-format** to enforce the olang code style.
-Please refer to the linter section below for guidance on its
+Please refer to the **Format** section below for guidance on its
application.
-### Linter
+### Format
-Checking for linter issues:
+Checking for format issues:
``` {.sh}
-make linter
+make format
```
Most of the common code style mistakes are fixed by:
``` {.sh}
-make linter-fix
+make format-fix
```
### .editorconfig
@@ -92,9 +92,9 @@ the email-driven workflow here, but you can check it out at
1. Write single-purpose commits.
2. Write a meaningful commit message.
3. Every commit must be production ready.
- - If the tests or the linter fail, you should not create a fix commit.
- Instead, you should amend the commit that caused the issue and then
- resend the patchset.
+ - If the tests or the format check fail, you should not create a fix
+ commit. Instead, you should amend the commit that caused the issue and
+ then resend the patchset.
### Step 2: Create your patch
diff --git a/tests/integration/Makefile b/tests/integration/Makefile
index db2b7d9..4625707 100644
--- a/tests/integration/Makefile
+++ b/tests/integration/Makefile
@@ -16,12 +16,12 @@ all: $(MUNIT) proc_exec.o cli_runner.o $(TESTS)
clean:
$(RM) *.o *_test
-.PHONY: linter
-linter: $(SRCS)
+.PHONY: format
+format: $(SRCS)
clang-format --dry-run --Werror $?
-.PHONY: linter-fix
-linter-fix: $(SRCS)
+.PHONY: format-fix
+format-fix: $(SRCS)
clang-format -i $?
cli_test: $(MUNIT) proc_exec.o cli_runner.o cli_test.o
diff --git a/tests/unit/Makefile b/tests/unit/Makefile
index 498bf98..783225c 100644
--- a/tests/unit/Makefile
+++ b/tests/unit/Makefile
@@ -18,12 +18,12 @@ clean:
$(RM) *.o *_test
$(RM) -rfv lib
-.PHONY: linter
-linter: $(SRCS)
+.PHONY: format
+format: $(SRCS)
clang-format --dry-run --Werror $?
-.PHONY: linter-fix
-linter-fix: $(SRCS)
+.PHONY: format-fix
+format-fix: $(SRCS)
clang-format -i $?
%_test: $(MUNIT) $(SUBJECT_OBJS) %_test.c
base-commit: 36b028f712ff2402761ea307467860c346d3c0a0
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v1] fe: lexer: add single line comments support
@ 2024-03-28 15:58 Johnny Richard
2024-03-28 14:59 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-28 15:58 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
examples/main_exit.ol | 3 +++
src/lexer.c | 7 +++++++
src/parser.c | 1 +
tests/integration/cli_test.c | 31 +++++++++++++++++--------------
4 files changed, 28 insertions(+), 14 deletions(-)
diff --git a/examples/main_exit.ol b/examples/main_exit.ol
index c86fc68..8952017 100644
--- a/examples/main_exit.ol
+++ b/examples/main_exit.ol
@@ -1,3 +1,6 @@
+# Expected:
+# - output: ""
+
fn main(): u32 {
return 0
}
diff --git a/src/lexer.c b/src/lexer.c
index 801e4d0..684cad1 100644
--- a/src/lexer.c
+++ b/src/lexer.c
@@ -76,6 +76,13 @@ lexer_next_token(lexer_t *lexer, token_t *token)
}
while (lexer_is_not_eof(lexer)) {
+ if (current_char == '#') {
+ while (current_char != '\n' && lexer_is_not_eof(lexer)) {
+ lexer_skip_char(lexer);
+ current_char = lexer_current_char(lexer);
+ }
+ }
+
if (isalpha(current_char)) {
size_t start_offset = lexer->offset;
while (isalnum(current_char) && lexer_is_not_eof(lexer)) {
diff --git a/src/parser.c b/src/parser.c
index 76ef91a..b800870 100644
--- a/src/parser.c
+++ b/src/parser.c
@@ -62,6 +62,7 @@ parser_init(parser_t *parser, lexer_t *lexer, arena_t *arena, char *file_path)
ast_node_t *
parser_parse_program(parser_t *parser)
{
+ skip_line_feeds(parser->lexer);
ast_node_t *fn = parser_parse_fn_definition(parser);
if (fn == NULL) {
return NULL;
diff --git a/tests/integration/cli_test.c b/tests/integration/cli_test.c
index d46471b..e7ae059 100644
--- a/tests/integration/cli_test.c
+++ b/tests/integration/cli_test.c
@@ -25,20 +25,23 @@ test_cli_dump_tokens_example_main_exit(const MunitParameter params[], void *user
cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/main_exit.ol");
munit_assert_int(compilation_result.exec.exit_code, ==, 0);
munit_assert_string_equal(compilation_result.exec.stdout_buf,
- "../../examples/main_exit.ol:1:1: <fn>\n"
- "../../examples/main_exit.ol:1:4: <identifier>\n"
- "../../examples/main_exit.ol:1:8: <(>\n"
- "../../examples/main_exit.ol:1:9: <)>\n"
- "../../examples/main_exit.ol:1:10: <:>\n"
- "../../examples/main_exit.ol:1:12: <identifier>\n"
- "../../examples/main_exit.ol:1:16: <{>\n"
- "../../examples/main_exit.ol:1:17: <line_feed>\n"
- "../../examples/main_exit.ol:2:3: <return>\n"
- "../../examples/main_exit.ol:2:10: <number>\n"
- "../../examples/main_exit.ol:2:11: <line_feed>\n"
- "../../examples/main_exit.ol:3:1: <}>\n"
- "../../examples/main_exit.ol:3:2: <line_feed>\n"
- "../../examples/main_exit.ol:4:1: <EOF>\n");
+ "../../examples/main_exit.ol:1:12: <line_feed>\n"
+ "../../examples/main_exit.ol:2:16: <line_feed>\n"
+ "../../examples/main_exit.ol:3:1: <line_feed>\n"
+ "../../examples/main_exit.ol:4:1: <fn>\n"
+ "../../examples/main_exit.ol:4:4: <identifier>\n"
+ "../../examples/main_exit.ol:4:8: <(>\n"
+ "../../examples/main_exit.ol:4:9: <)>\n"
+ "../../examples/main_exit.ol:4:10: <:>\n"
+ "../../examples/main_exit.ol:4:12: <identifier>\n"
+ "../../examples/main_exit.ol:4:16: <{>\n"
+ "../../examples/main_exit.ol:4:17: <line_feed>\n"
+ "../../examples/main_exit.ol:5:3: <return>\n"
+ "../../examples/main_exit.ol:5:10: <number>\n"
+ "../../examples/main_exit.ol:5:11: <line_feed>\n"
+ "../../examples/main_exit.ol:6:1: <}>\n"
+ "../../examples/main_exit.ol:6:2: <line_feed>\n"
+ "../../examples/main_exit.ol:7:1: <EOF>\n");
return MUNIT_OK;
}
base-commit: 117a06874c48c64e8ad4befbab244670f4f9ca9c
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [olang/patches/.build.yml] build failed
2024-03-28 15:58 [PATCH olang v1] fe: lexer: add single line comments support Johnny Richard
@ 2024-03-28 14:59 ` builds.sr.ht
2024-03-28 16:46 ` Johnny Richard
0 siblings, 1 reply; 35+ messages in thread
From: builds.sr.ht @ 2024-03-28 14:59 UTC (permalink / raw)
To: Johnny Richard; +Cc: ~johnnyrichard/olang-devel
olang/patches/.build.yml: FAILED in 36s
[fe: lexer: add single line comments support][0] from [Johnny Richard][1]
[0]: https://lists.sr.ht/~johnnyrichard/olang-devel/patches/50503
[1]: mailto:johnny@johnnyrichard.com
✗ #1181030 FAILED olang/patches/.build.yml https://builds.sr.ht/~johnnyrichard/job/1181030
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [olang/patches/.build.yml] build failed
2024-03-28 14:59 ` [olang/patches/.build.yml] build failed builds.sr.ht
@ 2024-03-28 16:46 ` Johnny Richard
0 siblings, 0 replies; 35+ messages in thread
From: Johnny Richard @ 2024-03-28 16:46 UTC (permalink / raw)
To: builds.sr.ht; +Cc: ~johnnyrichard/olang-devel
On Thu, Mar 28, 2024 at 02:59:02PM +0000, builds.sr.ht wrote:
> olang/patches/.build.yml: FAILED in 36s
>
> [fe: lexer: add single line comments support][0] from [Johnny Richard][1]
>
> [0]: https://lists.sr.ht/~johnnyrichard/olang-devel/patches/50503
> [1]: mailto:johnny@johnnyrichard.com
>
> ✗ #1181030 FAILED olang/patches/.build.yml https://builds.sr.ht/~johnnyrichard/job/1181030
This build is failing due to a clang bumped version on Arch Linux
repositories. It was a major bump from 16 to 17 (it has breaking
changes).
We can solve this problem by fixing the version or by bumping our clang
locally. I will upgrade my clang meanwhile.
Please, consider the patch to fix it.
---->8----
Subject: [PATCH olang] fixup! fe: lexer: add single line comments support
diff --git a/src/pretty_print_ast.c b/src/pretty_print_ast.c
index e950796..129f090 100644
--- a/src/pretty_print_ast.c
+++ b/src/pretty_print_ast.c
@@ -26,7 +26,7 @@
#define ANSI_COLOR_MAGENTA "\x1b[35m"
#define ANSI_COLOR_RESET "\x1b[0m"
-#define PP_IS_BIT_SET(data, index) ((data)&1 << index)
+#define PP_IS_BIT_SET(data, index) ((data) & 1 << index)
typedef struct pretty_print_node
{
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH olang v1 2/2] docs: spec: add variables and constants specification
@ 2024-03-27 3:21 Carlos Maniero
2024-03-27 3:22 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-03-27 3:21 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
This commit introduces the specification for variables and constants. A
valid program under this specification is as follows:
var x: u32 = 1
const y: u32 = 2
x = 2
fn main(): u32 {
var x: u32 = 1; const y: u32 = 2
return x
}
Variables and Linkage:
----------------------
Variables can be defined globally or within a function. Variables
defined globally are added to the *.data* section. On the other hand,
variables defined within a function utilize the stack. Currently, the
specification does not provide a keyword for external linkage yet, hence
variables cannot be accessed outside their translation unit.
Constants and Linkage:
----------------------
Constants behave similarly to variables. Constants defined at the file
level are added to the *.rodata* section. Constants within functions are
subject to semantic checks. Attempting to bypass these checks to
reassign a global constant will result in a segmentation fault
(SEGFAULT) and will be accepted at the function level.
Example of bypassing semantic checks in C:
int a = 1;
const int b = 2;
int *c = &a + 1;
*c = 3;
assert(b == 3); // its true
Static variables in function level
----------------------------------
In C, static variables can be defined within the scope of a function.
These variables are added to the `*.data*` segment and are only
accessible within the function where they are defined. This is a unique
behavior of C that some might find unusual.
However, olang does not support this feature. In olang, if you need a
variable that retains its value across function calls (like a static
variable in C), you must define it at the file level.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
docs/pages/language-specification.md | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/docs/pages/language-specification.md b/docs/pages/language-specification.md
index 4541ba8..708b679 100644
--- a/docs/pages/language-specification.md
+++ b/docs/pages/language-specification.md
@@ -24,7 +24,15 @@ language.
(* Entry Point *)
<translation-unit> ::= <ows> (<global-statements> <end-of-statement>)*
-<global-statements> ::= <function-definition>
+<global-statements> ::= <function-definition> | <variable-definition> | <variable-reassign> | <const-definition>
+
+(* Variables *)
+<variable-definition> ::= 'var' <ws> <variable-name> <ows> ':' <ows> <type> <ows> <variable-assign>?
+<const-definition> ::= 'const' <ws> <variable-name> <ows> ':' <ows> <type> <ows> <variable-assign>
+
+<variable-name> ::= <identifier>
+<variable-assign> ::= '=' <ows> <expression>
+<variable-reassign> ::= <variable-name> <ows> <variable-assign> <end-of-statement>
(* Functions *)
<function-definition> ::= 'fn' <ws> <function-name> <ows>
@@ -38,11 +46,11 @@ language.
<block> ::= '{' <ows> <statement> <ows> (<end-of-statement>
<ows> <statement> <ows>)* <end-of-statement>? <ows> '}'
<end-of-statement> ::= ';' | <line-break> | <end-of-file>
-<statement> ::= <return-statement>
+<statement> ::= <return-statement> | <variable-definition> | <variable-reassign> | <const-definition>
<return-statement> ::= 'return' <ws> <expression>
(* Expressions *)
-<expression> ::= <integer>
+<expression> ::= <integer> | <identifier>
(* Identifiers *)
<type> ::= 'u32'
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v1 2/2] cli: remove code duplication
@ 2024-03-25 22:36 Johnny Richard
2024-03-25 21:37 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-25 22:36 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
Ops... LOL
src/main.c | 5 -----
1 file changed, 5 deletions(-)
diff --git a/src/main.c b/src/main.c
index 70e7d3f..e16695b 100644
--- a/src/main.c
+++ b/src/main.c
@@ -68,11 +68,6 @@ main(int argc, char **argv)
return EXIT_SUCCESS;
}
- if (opts.options & CLI_OPT_DUMP_TOKENS) {
- handle_dump_tokens(&opts);
- return EXIT_SUCCESS;
- }
-
return EXIT_FAILURE;
}
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v3] refactor: rename zero programming language to olang
@ 2024-03-13 12:32 Fabio Maciel
2024-03-13 12:33 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Fabio Maciel @ 2024-03-13 12:32 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard, Maria Ruy, Fabio Maciel
From: Johnny Richard <johnny@johnnyrichard.com>
After a long discussion we've decided to rename the project to olang.
This patch removes anything related to zero programming language and
also renames the compiler and extension.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
Co-authored-by: Maria Ruy <maria.dev456@gmail.com>
Co-authored-by: Fabio Maciel <fabio@fabiomaciel.com>
---
V3 fix hacking.md
.gitignore | 2 +-
Makefile | 4 ++--
docs/Makefile | 2 +-
docs/index.md | 2 +-
docs/manpages/{0c.md => olang.md} | 8 +++----
docs/pages/hacking.md | 2 +-
docs/template.html | 2 +-
examples/{main_exit.0 => main_exit.ol} | 0
tests/integration/cli_runner.c | 6 ++---
tests/integration/cli_test.c | 32 +++++++++++++-------------
10 files changed, 30 insertions(+), 30 deletions(-)
rename docs/manpages/{0c.md => olang.md} (74%)
rename examples/{main_exit.0 => main_exit.ol} (100%)
diff --git a/.gitignore b/.gitignore
index fc7d161..a565aae 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,4 @@
-0c
+olang
build
*.o
docs/site.tar.gz
diff --git a/Makefile b/Makefile
index 662d039..cdfc8e1 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,4 @@
-TARGET := 0c
+TARGET := olang
SRC_DIR := src
BUILD_DIR := build
CFLAGS := -Werror -Wall -Wextra -Wmissing-declarations -pedantic -std=c11 -ggdb
@@ -42,7 +42,7 @@ unit-test:
clean:
$(MAKE) -C tests/integration/ clean
$(MAKE) -C tests/unit/ clean
- @rm -rf build/ 0c
+ @rm -rf build/ $(TARGET)
.PHONY: check
check:
diff --git a/docs/Makefile b/docs/Makefile
index 54561a1..d34f9f5 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -21,7 +21,7 @@ clean:
dist: $(DIST_FILE)
.PHONY: manpages
-manpages: $(BUILD_DIR) $(MANPAGES)/0c.1
+manpages: $(BUILD_DIR) $(MANPAGES)/oc.1
$(MANPAGES)/%.1: manpages/%.md
$(PANDOC) -s -t man $< > $@
diff --git a/docs/index.md b/docs/index.md
index b6d5c1a..1a28069 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,6 +1,6 @@
% Welcome to olang documentation
-The zero programming language.
+The O Programming Language.
## olang manifest
diff --git a/docs/manpages/0c.md b/docs/manpages/olang.md
similarity index 74%
rename from docs/manpages/0c.md
rename to docs/manpages/olang.md
index e3d3cfc..c54c597 100644
--- a/docs/manpages/0c.md
+++ b/docs/manpages/olang.md
@@ -1,21 +1,21 @@
-% 0C(1)
+% OLANG(1)
% olang mantainers
% Feb 2024
# NAME
-0c - zero language compiler
+olang - O Programming Language compiler
# SYNOPSIS
-**0c**
+**olang**
source_file
[**----dump-tokens**]
[**--o** output_file [**----save-temps**]]
# DESCRIPTION
-**0c** is the offical compiler for zero language, it is also a tool that
+**olang** is the offical O programming language compiler, it is also a tool that
contains utilities to help the language development.
# GENERAL OPTIONS
diff --git a/docs/pages/hacking.md b/docs/pages/hacking.md
index ef88791..fe8f705 100644
--- a/docs/pages/hacking.md
+++ b/docs/pages/hacking.md
@@ -53,7 +53,7 @@ Testing
-------
There are two layers of tests **integration** and **unit**. The
-integration test is going to execute the **0c** compiler and check if
+integration test is going to execute the **olang** compiler and check if
the generated binary acts as expected. Unit tests will test C functions.
For both unit and integration we use **munit** framework:
diff --git a/docs/template.html b/docs/template.html
index ecc92a2..4e066d1 100644
--- a/docs/template.html
+++ b/docs/template.html
@@ -52,7 +52,7 @@
</head>
<body>
<header>
- <h1>∅lang | The zero programming language</h1>
+ <h1>olang | The O Programming Language</h1>
<nav>
<a href="/">Index</a> |
<a href="/pages/getting-started.html">Getting started (WIP)</a> |
diff --git a/examples/main_exit.0 b/examples/main_exit.ol
similarity index 100%
rename from examples/main_exit.0
rename to examples/main_exit.ol
diff --git a/tests/integration/cli_runner.c b/tests/integration/cli_runner.c
index fed12ab..636abfc 100644
--- a/tests/integration/cli_runner.c
+++ b/tests/integration/cli_runner.c
@@ -24,7 +24,7 @@
#include <sys/wait.h>
#include <unistd.h>
-#define OLANG_COMPILER_PATH "../../0c"
+#define OLANG_COMPILER_PATH "../../olang"
static int compiler_exists_already_checked = 0;
@@ -83,7 +83,7 @@ cli_runner_compiler_dump_tokens(char *src)
{
cli_result_t result = { 0 };
- char *program_args[] = { "0c", "--dump-tokens", src, NULL };
+ char *program_args[] = { "olang", "--dump-tokens", src, NULL };
cli_runner_compiler(&result, program_args);
return result;
}
@@ -94,7 +94,7 @@ cli_runner_compiler_compile(char *src)
cli_result_t result = { 0 };
create_tmp_file_name(result.binary_path);
- char *program_args[] = { "0c", src, "-o", result.binary_path, NULL };
+ char *program_args[] = { "olang", src, "-o", result.binary_path, NULL };
cli_runner_compiler(&result, program_args);
return result;
}
diff --git a/tests/integration/cli_test.c b/tests/integration/cli_test.c
index c5896df..8cc22f9 100644
--- a/tests/integration/cli_test.c
+++ b/tests/integration/cli_test.c
@@ -22,30 +22,30 @@
static MunitResult
test_cli_dump_tokens(const MunitParameter params[], void *user_data_or_fixture)
{
- cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/main_exit.0");
+ cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/main_exit.ol");
munit_assert_int(compilation_result.exec.exit_code, ==, 0);
munit_assert_string_equal(compilation_result.exec.stdout_buf,
- "../../examples/main_exit.0:1:1: <fn>\n"
- "../../examples/main_exit.0:1:4: <identifier>\n"
- "../../examples/main_exit.0:1:8: <(>\n"
- "../../examples/main_exit.0:1:9: <)>\n"
- "../../examples/main_exit.0:1:10: <:>\n"
- "../../examples/main_exit.0:1:12: <identifier>\n"
- "../../examples/main_exit.0:1:16: <{>\n"
- "../../examples/main_exit.0:1:17: <line_feed>\n"
- "../../examples/main_exit.0:2:3: <return>\n"
- "../../examples/main_exit.0:2:10: <number>\n"
- "../../examples/main_exit.0:2:11: <line_feed>\n"
- "../../examples/main_exit.0:3:1: <}>\n"
- "../../examples/main_exit.0:3:2: <line_feed>\n"
- "../../examples/main_exit.0:4:1: <EOF>\n");
+ "../../examples/main_exit.ol:1:1: <fn>\n"
+ "../../examples/main_exit.ol:1:4: <identifier>\n"
+ "../../examples/main_exit.ol:1:8: <(>\n"
+ "../../examples/main_exit.ol:1:9: <)>\n"
+ "../../examples/main_exit.ol:1:10: <:>\n"
+ "../../examples/main_exit.ol:1:12: <identifier>\n"
+ "../../examples/main_exit.ol:1:16: <{>\n"
+ "../../examples/main_exit.ol:1:17: <line_feed>\n"
+ "../../examples/main_exit.ol:2:3: <return>\n"
+ "../../examples/main_exit.ol:2:10: <number>\n"
+ "../../examples/main_exit.ol:2:11: <line_feed>\n"
+ "../../examples/main_exit.ol:3:1: <}>\n"
+ "../../examples/main_exit.ol:3:2: <line_feed>\n"
+ "../../examples/main_exit.ol:4:1: <EOF>\n");
return MUNIT_OK;
}
static MunitResult
test_cli_compile_minimal_program(const MunitParameter params[], void *user_data_or_fixture)
{
- cli_result_t compilation_result = cli_runner_compiler_compile("../../examples/main_exit.0");
+ cli_result_t compilation_result = cli_runner_compiler_compile("../../examples/main_exit.ol");
munit_assert_int(compilation_result.exec.exit_code, ==, 0);
char *command_args[] = { compilation_result.binary_path, NULL };
--
2.39.3 (Apple Git-145)
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang] parser: abort when parser identifies a syntax error
@ 2024-03-08 20:52 Johnny Richard
2024-03-08 19:54 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-08 20:52 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
The current implementation fails with segfault when the parser finds a
syntax error. We are not prepared to recover from errors at this
moment.
This patch aborts when found an expected token and it also improves the
error output by showing the wrong token string value.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
Let's postpone syntax error recovery. I want to keep things simple
right now.
src/parser.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/parser.c b/src/parser.c
index a9699be..cfea9ef 100644
--- a/src/parser.c
+++ b/src/parser.c
@@ -173,18 +173,18 @@ expected_token(parser_t *parser, token_t *token, token_kind_t expected_kind)
if (token->kind != expected_kind) {
fprintf(stderr,
- "%s:%lu:%lu: error: got <%s> token but expect <%s>\n",
+ "%s:%lu:%lu: error: got '"SV_FMT"' token but expect <%s>\n",
parser->file_path,
token->location.row + 1,
(token->location.offset - token->location.bol) + 1,
- token_kind_to_cstr(token->kind),
+ SV_ARG(token->value),
token_kind_to_cstr(expected_kind));
string_view_t line = lexer_get_token_line(parser->lexer, token);
fprintf(stderr, "" SV_FMT "\n", SV_ARG(line));
fprintf(stderr, "%*s\n", (int)(token->location.offset - token->location.bol + 1), "^");
- return false;
+ exit(EXIT_FAILURE);
}
return true;
}
base-commit: 35f594370443a2b9f73d2d2ebe573b4cab472be6
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v2 3/3] cli: add compilation -o option with --save-temps
@ 2024-03-05 8:44 Johnny Richard
2024-03-05 7:51 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-05 8:44 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
In order to compile the program, we are introduction the option -o to
the compiler.
This implementation is very similar to the gcc one. Since the program
is made of a single file, no need to over complicate the compilation
with multiple files.
The option --save-temps is used to keep files used to create the binary
file (.a and .o). By default this option is disabled.
WARNING
-------
This implementation relies on "as" (GNU Assembler) and the "ld" (GNU
linker) command. Make sure you have it install before trying to compile
any program.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
v1 -> v2: Change file_path type from string_view_t to cstr.
Use first bit for enum cli_opt_t.
Change cli_opts_t#prog -> cli_opts_t#compiler_path.
Fix assert on args.
Refactor creation of cstr asm_file.
docs/manpages/0c.md | 13 +++-
src/main.c | 164 ++++++++++++++++++++++++++++++++++++--------
2 files changed, 146 insertions(+), 31 deletions(-)
diff --git a/docs/manpages/0c.md b/docs/manpages/0c.md
index 87a56df..e3d3cfc 100644
--- a/docs/manpages/0c.md
+++ b/docs/manpages/0c.md
@@ -4,11 +4,14 @@
# NAME
-0c - zero langague compiler
+0c - zero language compiler
# SYNOPSIS
-**0c** **----dump-tokens** source.0
+**0c**
+ source_file
+ [**----dump-tokens**]
+ [**--o** output_file [**----save-temps**]]
# DESCRIPTION
@@ -19,3 +22,9 @@ contains utilities to help the language development.
**----dump-tokens**
: Display lexical tokens given a soruce.0 code.
+
+**--o <file>**
+: Compile program into a binary file
+
+**----save-temps**
+: Keep temp files used to compile program
diff --git a/src/main.c b/src/main.c
index 2b2f12a..e4be5f7 100644
--- a/src/main.c
+++ b/src/main.c
@@ -22,7 +22,9 @@
#include <string.h>
#include "arena.h"
+#include "codegen_linux_x86_64.h"
#include "lexer.h"
+#include "parser.h"
#include "string_view.h"
// TODO: find a better solution to define the arena capacity
@@ -37,14 +39,29 @@ typedef struct cli_args
char *
cli_args_shift(cli_args_t *args);
+typedef enum
+{
+ CLI_OPT_DUMP_TOKENS = 1 << 0,
+ CLI_OPT_OUTPUT = 1 << 1,
+ CLI_OPT_SAVE_TEMPS = 1 << 2
+} cli_opt_t;
+
typedef struct cli_opts
{
- bool dump_tokens;
+ uint32_t options;
+ char *compiler_path;
char *file_path;
+ string_view_t output_bin;
} cli_opts_t;
void
-print_usage(FILE *stream, char *prog);
+print_usage(FILE *stream, char *compiler_path);
+
+void
+handle_dump_tokens(cli_opts_t *opts);
+
+void
+handle_codegen_linux_x86_64(cli_opts_t *opts);
static void
print_token(char *file_path, token_t *token);
@@ -52,34 +69,64 @@ print_token(char *file_path, token_t *token);
string_view_t
read_entire_file(char *file_path, arena_t *arena);
+void
+cli_opts_parse_output(cli_opts_t *opts, cli_args_t *args);
+
int
main(int argc, char **argv)
{
cli_args_t args = { .argc = argc, .argv = argv };
cli_opts_t opts = { 0 };
- char *prog = cli_args_shift(&args);
+ opts.compiler_path = cli_args_shift(&args);
- if (argc != 3) {
- print_usage(stderr, prog);
- return EXIT_FAILURE;
- }
-
- for (char *arg = cli_args_shift(&args); arg != NULL; arg = cli_args_shift(&args)) {
+ char *arg = cli_args_shift(&args);
+ while (arg != NULL) {
if (strcmp(arg, "--dump-tokens") == 0) {
- opts.dump_tokens = true;
+ opts.options |= CLI_OPT_DUMP_TOKENS;
+ } else if (strcmp(arg, "--save-temps") == 0) {
+ opts.options |= CLI_OPT_SAVE_TEMPS;
+ } else if (strcmp(arg, "-o") == 0) {
+ cli_opts_parse_output(&opts, &args);
} else {
opts.file_path = arg;
}
+ arg = cli_args_shift(&args);
+ }
+
+ if (opts.options & CLI_OPT_OUTPUT) {
+ handle_codegen_linux_x86_64(&opts);
+ return EXIT_SUCCESS;
}
- if (!opts.dump_tokens) {
- print_usage(stderr, prog);
- return EXIT_FAILURE;
+ if (opts.options & CLI_OPT_DUMP_TOKENS) {
+ handle_dump_tokens(&opts);
+ return EXIT_SUCCESS;
+ }
+
+ print_usage(stderr, opts.compiler_path);
+ return EXIT_FAILURE;
+}
+
+char *
+cli_args_shift(cli_args_t *args)
+{
+ if (args->argc == 0)
+ return NULL;
+ --(args->argc);
+ return *(args->argv)++;
+}
+
+void
+handle_dump_tokens(cli_opts_t *opts)
+{
+ if (opts->file_path == NULL) {
+ print_usage(stderr, opts->compiler_path);
+ exit(EXIT_FAILURE);
}
arena_t arena = arena_new(ARENA_CAPACITY);
- string_view_t file_content = read_entire_file(opts.file_path, &arena);
+ string_view_t file_content = read_entire_file(opts->file_path, &arena);
lexer_t lexer = { 0 };
lexer_init(&lexer, file_content);
@@ -87,29 +134,70 @@ main(int argc, char **argv)
token_t token = { 0 };
lexer_next_token(&lexer, &token);
while (token.kind != TOKEN_EOF) {
- print_token(opts.file_path, &token);
+ print_token(opts->file_path, &token);
lexer_next_token(&lexer, &token);
}
- print_token(opts.file_path, &token);
+ print_token(opts->file_path, &token);
- free(file_content.chars);
-
- return EXIT_SUCCESS;
+ arena_free(&arena);
}
-char *
-cli_args_shift(cli_args_t *args)
+void
+handle_codegen_linux_x86_64(cli_opts_t *opts)
{
- if (args->argc == 0)
- return NULL;
- --(args->argc);
- return *(args->argv)++;
+ if (opts->file_path == NULL) {
+ print_usage(stderr, opts->compiler_path);
+ exit(EXIT_FAILURE);
+ }
+
+ arena_t arena = arena_new(ARENA_CAPACITY);
+ lexer_t lexer = { 0 };
+ parser_t parser = { 0 };
+
+ string_view_t file_content = read_entire_file(opts->file_path, &arena);
+ lexer_init(&lexer, file_content);
+ parser_init(&parser, &lexer, &arena, opts->file_path);
+
+ ast_node_t *ast = parser_parse_fn_definition(&parser);
+
+ char asm_file[opts->output_bin.size + 3];
+ sprintf(asm_file, "" SV_FMT ".s", SV_ARG(opts->output_bin));
+
+ FILE *out = fopen(asm_file, "w");
+ assert(out);
+ codegen_linux_x86_64_emit_program(out, ast);
+ fclose(out);
+
+ char command[512];
+ sprintf(command, "as %s -o " SV_FMT ".o", asm_file, SV_ARG(opts->output_bin));
+ system(command);
+
+ sprintf(command, "ld " SV_FMT ".o -o " SV_FMT "", SV_ARG(opts->output_bin), SV_ARG(opts->output_bin));
+ system(command);
+
+ if (!(opts->options & CLI_OPT_SAVE_TEMPS)) {
+ char output_file[256];
+
+ sprintf(output_file, "" SV_FMT ".s", SV_ARG(opts->output_bin));
+ remove(output_file);
+
+ sprintf(output_file, "" SV_FMT ".o", SV_ARG(opts->output_bin));
+ remove(output_file);
+ }
+
+ arena_free(&arena);
}
void
-print_usage(FILE *stream, char *prog)
+print_usage(FILE *stream, char *compiler_path)
{
- fprintf(stream, "usage: %s <source.0> --dump-tokens\n", prog);
+ fprintf(stream,
+ "Usage: %s [options] file...\n"
+ "Options:\n"
+ " --dump-tokens\t\tDisplay lexer token stream\n"
+ " -o <file>\t\tCompile program into a binary file\n"
+ " --save-temps\t\tKeep temp files used to compile program\n",
+ compiler_path);
}
string_view_t
@@ -118,7 +206,7 @@ read_entire_file(char *file_path, arena_t *arena)
FILE *stream = fopen(file_path, "rb");
if (stream == NULL) {
- fprintf(stderr, "Could not open file %s: %s\n", file_path, strerror(errno));
+ fprintf(stderr, "error: could not open file %s: %s\n", file_path, strerror(errno));
exit(EXIT_FAILURE);
}
@@ -133,7 +221,7 @@ read_entire_file(char *file_path, arena_t *arena)
file_content.chars = (char *)arena_alloc(arena, (size_t)file_content.size);
if (file_content.chars == NULL) {
- fprintf(stderr, "Could not read file %s: %s\n", file_path, strerror(errno));
+ fprintf(stderr, "error: could not read file %s: %s\n", file_path, strerror(errno));
exit(EXIT_FAILURE);
}
@@ -143,6 +231,24 @@ read_entire_file(char *file_path, arena_t *arena)
return file_content;
}
+void
+cli_opts_parse_output(cli_opts_t *opts, cli_args_t *args)
+{
+ assert(opts && "opts is required");
+ assert(args && "args is required");
+
+ char *output_bin = cli_args_shift(args);
+
+ if (output_bin == NULL) {
+ fprintf(stderr, "error: missing filename after '-o'\n");
+ print_usage(stderr, opts->compiler_path);
+ exit(EXIT_FAILURE);
+ }
+
+ opts->options |= CLI_OPT_OUTPUT;
+ opts->output_bin = string_view_from_cstr(output_bin);
+}
+
static void
print_token(char *file_path, token_t *token)
{
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v1 3/3] cli: add compilation -o option with --save-temps
@ 2024-03-04 19:23 Johnny Richard
2024-03-04 18:33 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-04 19:23 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
In order to compile the program, we are introduction the option -o to
the compiler.
This implementation is very similar to the gcc one. Since the program
is made of a single file, no need to over complicate the compilation
with multiple files.
The option --save-temps is used to keep files used to create the binary
file (.a and .o). By default this option is disabled.
WARNING
-------
This implementation relies on "as" (GNU Assembler) and the "ld" (GNU
linker) command. Make sure you have it install before trying to compile
any program.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
docs/manpages/0c.md | 13 +++-
src/main.c | 180 +++++++++++++++++++++++++++++++++++---------
2 files changed, 157 insertions(+), 36 deletions(-)
diff --git a/docs/manpages/0c.md b/docs/manpages/0c.md
index 87a56df..e3d3cfc 100644
--- a/docs/manpages/0c.md
+++ b/docs/manpages/0c.md
@@ -4,11 +4,14 @@
# NAME
-0c - zero langague compiler
+0c - zero language compiler
# SYNOPSIS
-**0c** **----dump-tokens** source.0
+**0c**
+ source_file
+ [**----dump-tokens**]
+ [**--o** output_file [**----save-temps**]]
# DESCRIPTION
@@ -19,3 +22,9 @@ contains utilities to help the language development.
**----dump-tokens**
: Display lexical tokens given a soruce.0 code.
+
+**--o <file>**
+: Compile program into a binary file
+
+**----save-temps**
+: Keep temp files used to compile program
diff --git a/src/main.c b/src/main.c
index 2b2f12a..b27715a 100644
--- a/src/main.c
+++ b/src/main.c
@@ -22,7 +22,9 @@
#include <string.h>
#include "arena.h"
+#include "codegen_linux_x86_64.h"
#include "lexer.h"
+#include "parser.h"
#include "string_view.h"
// TODO: find a better solution to define the arena capacity
@@ -37,20 +39,38 @@ typedef struct cli_args
char *
cli_args_shift(cli_args_t *args);
+typedef enum
+{
+ CLI_OPT_DUMP_TOKENS = 1 << 1,
+ CLI_OPT_OUTPUT = 1 << 2,
+ CLI_OPT_SAVE_TEMPS = 1 << 3
+} cli_opt;
+
typedef struct cli_opts
{
- bool dump_tokens;
- char *file_path;
+ uint32_t options;
+ string_view_t prog;
+ string_view_t file_path;
+ string_view_t output_bin;
} cli_opts_t;
void
-print_usage(FILE *stream, char *prog);
+print_usage(FILE *stream, string_view_t prog);
+
+void
+handle_dump_tokens(cli_opts_t *opts);
+
+void
+handle_codegen_linux_x86_64(cli_opts_t *opts);
static void
print_token(char *file_path, token_t *token);
string_view_t
-read_entire_file(char *file_path, arena_t *arena);
+read_entire_file(string_view_t file_path, arena_t *arena);
+
+void
+cli_opts_parse_output(cli_opts_t *opts, cli_args_t *args);
int
main(int argc, char **argv)
@@ -58,28 +78,58 @@ main(int argc, char **argv)
cli_args_t args = { .argc = argc, .argv = argv };
cli_opts_t opts = { 0 };
- char *prog = cli_args_shift(&args);
+ opts.prog = string_view_from_cstr(cli_args_shift(&args));
- if (argc != 3) {
- print_usage(stderr, prog);
- return EXIT_FAILURE;
- }
-
- for (char *arg = cli_args_shift(&args); arg != NULL; arg = cli_args_shift(&args)) {
+ char *arg = cli_args_shift(&args);
+ while (arg != NULL) {
if (strcmp(arg, "--dump-tokens") == 0) {
- opts.dump_tokens = true;
+ opts.options |= CLI_OPT_DUMP_TOKENS;
+ arg = cli_args_shift(&args);
+ } else if (strcmp(arg, "--save-temps") == 0) {
+ opts.options |= CLI_OPT_SAVE_TEMPS;
+ arg = cli_args_shift(&args);
+ } else if (strcmp(arg, "-o") == 0) {
+ cli_opts_parse_output(&opts, &args);
+ arg = cli_args_shift(&args);
} else {
- opts.file_path = arg;
+ opts.file_path = string_view_from_cstr(arg);
+ arg = cli_args_shift(&args);
}
}
- if (!opts.dump_tokens) {
- print_usage(stderr, prog);
- return EXIT_FAILURE;
+ if (opts.options & CLI_OPT_OUTPUT) {
+ handle_codegen_linux_x86_64(&opts);
+ return EXIT_SUCCESS;
+ }
+
+ if (opts.options & CLI_OPT_DUMP_TOKENS) {
+ handle_dump_tokens(&opts);
+ return EXIT_SUCCESS;
+ }
+
+ print_usage(stderr, opts.prog);
+ return EXIT_FAILURE;
+}
+
+char *
+cli_args_shift(cli_args_t *args)
+{
+ if (args->argc == 0)
+ return NULL;
+ --(args->argc);
+ return *(args->argv)++;
+}
+
+void
+handle_dump_tokens(cli_opts_t *opts)
+{
+ if (opts->file_path.chars == NULL) {
+ print_usage(stderr, opts->prog);
+ exit(EXIT_FAILURE);
}
arena_t arena = arena_new(ARENA_CAPACITY);
- string_view_t file_content = read_entire_file(opts.file_path, &arena);
+ string_view_t file_content = read_entire_file(opts->file_path, &arena);
lexer_t lexer = { 0 };
lexer_init(&lexer, file_content);
@@ -87,38 +137,82 @@ main(int argc, char **argv)
token_t token = { 0 };
lexer_next_token(&lexer, &token);
while (token.kind != TOKEN_EOF) {
- print_token(opts.file_path, &token);
+ print_token(opts->file_path.chars, &token);
lexer_next_token(&lexer, &token);
}
- print_token(opts.file_path, &token);
+ print_token(opts->file_path.chars, &token);
- free(file_content.chars);
-
- return EXIT_SUCCESS;
+ arena_free(&arena);
}
-char *
-cli_args_shift(cli_args_t *args)
+void
+handle_codegen_linux_x86_64(cli_opts_t *opts)
{
- if (args->argc == 0)
- return NULL;
- --(args->argc);
- return *(args->argv)++;
+ if (opts->file_path.chars == NULL) {
+ print_usage(stderr, opts->prog);
+ exit(EXIT_FAILURE);
+ }
+
+ arena_t arena = arena_new(ARENA_CAPACITY);
+ lexer_t lexer = { 0 };
+ parser_t parser = { 0 };
+
+ string_view_t file_content = read_entire_file(opts->file_path, &arena);
+ lexer_init(&lexer, file_content);
+ parser_init(&parser, &lexer, &arena, opts->file_path.chars);
+
+ ast_node_t *ast = parser_parse_fn_definition(&parser);
+
+ string_view_t asm_ext = string_view_from_cstr(".s");
+ char asm_file[opts->output_bin.size + asm_ext.size + 1];
+ memcpy(asm_file, opts->output_bin.chars, opts->output_bin.size);
+ memcpy(asm_file + opts->output_bin.size, asm_ext.chars, asm_ext.size);
+ asm_file[opts->output_bin.size + asm_ext.size] = 0;
+
+ FILE *out = fopen(asm_file, "w");
+ assert(out);
+ codegen_linux_x86_64_emit_program(out, ast);
+ fclose(out);
+
+ char command[512];
+ sprintf(command, "as %s -o " SV_FMT ".o", asm_file, SV_ARG(opts->output_bin));
+ system(command);
+
+ sprintf(command, "ld " SV_FMT ".o -o " SV_FMT "", SV_ARG(opts->output_bin), SV_ARG(opts->output_bin));
+ system(command);
+
+ if (!(opts->options & CLI_OPT_SAVE_TEMPS)) {
+ char output_file[256];
+
+ sprintf(output_file, "" SV_FMT ".s", SV_ARG(opts->output_bin));
+ remove(output_file);
+
+ sprintf(output_file, "" SV_FMT ".o", SV_ARG(opts->output_bin));
+ remove(output_file);
+ }
+
+ arena_free(&arena);
}
void
-print_usage(FILE *stream, char *prog)
+print_usage(FILE *stream, string_view_t prog)
{
- fprintf(stream, "usage: %s <source.0> --dump-tokens\n", prog);
+ fprintf(stream,
+ "Usage: " SV_FMT " [options] file...\n"
+ "Options:\n"
+ " --dump-tokens\t\tDisplay lexer token stream\n"
+ " -o <file>\t\tCompile program into a binary file\n"
+ " --save-temps\t\tKeep temp files used to compile program\n",
+ SV_ARG(prog));
}
string_view_t
-read_entire_file(char *file_path, arena_t *arena)
+read_entire_file(string_view_t file_path, arena_t *arena)
{
- FILE *stream = fopen(file_path, "rb");
+ FILE *stream = fopen(file_path.chars, "rb");
if (stream == NULL) {
- fprintf(stderr, "Could not open file %s: %s\n", file_path, strerror(errno));
+ fprintf(stderr, "error: could not open file " SV_FMT ": %s\n", SV_ARG(file_path), strerror(errno));
exit(EXIT_FAILURE);
}
@@ -133,7 +227,7 @@ read_entire_file(char *file_path, arena_t *arena)
file_content.chars = (char *)arena_alloc(arena, (size_t)file_content.size);
if (file_content.chars == NULL) {
- fprintf(stderr, "Could not read file %s: %s\n", file_path, strerror(errno));
+ fprintf(stderr, "Could not read file " SV_FMT ": %s\n", SV_ARG(file_path), strerror(errno));
exit(EXIT_FAILURE);
}
@@ -143,6 +237,24 @@ read_entire_file(char *file_path, arena_t *arena)
return file_content;
}
+void
+cli_opts_parse_output(cli_opts_t *opts, cli_args_t *args)
+{
+ assert(opts && "opts is required");
+ assert(opts && "args is required");
+
+ char *output_bin = cli_args_shift(args);
+
+ if (output_bin == NULL) {
+ fprintf(stderr, "error: missing filename after '-o'\n");
+ print_usage(stderr, opts->prog);
+ exit(EXIT_FAILURE);
+ }
+
+ opts->options |= CLI_OPT_OUTPUT;
+ opts->output_bin = string_view_from_cstr(output_bin);
+}
+
static void
print_token(char *file_path, token_t *token)
{
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [olang/patches/.build.yml] build failed
2024-03-04 19:23 [PATCH olang v1 3/3] cli: add compilation -o option with --save-temps Johnny Richard
@ 2024-03-04 18:33 ` builds.sr.ht
2024-03-04 19:39 ` Johnny Richard
0 siblings, 1 reply; 35+ messages in thread
From: builds.sr.ht @ 2024-03-04 18:33 UTC (permalink / raw)
To: Johnny Richard; +Cc: ~johnnyrichard/olang-devel
olang/patches/.build.yml: FAILED in 12s
[implement assembly linux x86_64 compiler][0] from [Johnny Richard][1]
[0]: https://lists.sr.ht/~johnnyrichard/olang-devel/patches/49981
[1]: mailto:johnny@johnnyrichard.com
✗ #1161742 FAILED olang/patches/.build.yml https://builds.sr.ht/~johnnyrichard/job/1161742
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [olang/patches/.build.yml] build failed
2024-03-04 18:33 ` [olang/patches/.build.yml] build failed builds.sr.ht
@ 2024-03-04 19:39 ` Johnny Richard
2024-03-05 2:05 ` Carlos Maniero
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-04 19:39 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel
On Mon, Mar 04, 2024 at 06:33:31PM +0000, builds.sr.ht wrote:
> olang/patches/.build.yml: FAILED in 12s
>
> [implement assembly linux x86_64 compiler][0] from [Johnny Richard][1]
>
> [0]: https://lists.sr.ht/~johnnyrichard/olang-devel/patches/49981
> [1]: mailto:johnny@johnnyrichard.com
>
> ✗ #1161742 FAILED olang/patches/.build.yml https://builds.sr.ht/~johnnyrichard/job/1161742
I not sure what is happening, the build works find on my machine. Looks
like the CI machines are failing to setup the environment, nothing to do
with my changes I believe:
[#1161745] 2024/03/04 18:36:41 Booting image archlinux (default) on port 22563
[#1161745] 2024/03/04 18:36:42 Waiting for guest to settle
[#1161745] 2024/03/04 18:36:49 Sending tasks
[#1161745] 2024/03/04 18:36:52 Sending build environment
[#1161745] 2024/03/04 18:36:53 Installing packages
Warning: Permanently added '[localhost]:22563' (ED25519) to the list of known hosts.
:: Synchronizing package databases...
core downloading...
extra downloading...
multilib downloading...
warning: archlinux-keyring-20240208-1 is up to date -- skipping
there is nothing to do
Warning: Permanently added '[localhost]:22563' (ED25519) to the list of known hosts.
error: missing dependency 'initramfs' for package 'linux'
linux: ignoring package upgrade (6.7.6.arch1-1 => 6.7.8.arch1-1)
mkinitcpio: ignoring package upgrade (37.3-1 => 38-3)
Resolving dependencies...
Checking package conflicts...
:: uninstalling package 'mkinitcpio-37.3-1' due to conflict with 'cryptsetup-2.7.0-3'
[#1161745] 2024/03/04 18:36:54 Processing post-failed triggers...
[#1161745] 2024/03/04 18:36:54 Sending webhook...
[#1161745] 2024/03/04 18:36:54 Webhook response: 200
[#1161745] 2024/03/04 18:36:54 Thanks!
[#1161745] 2024/03/04 18:36:54 Build failed.
[#1161745] 2024/03/04 18:36:54 The build environment will be kept alive for 10 minutes.
[#1161745] 2024/03/04 18:36:54 To log in with SSH and examine it, use the following command:
[#1161745] 2024/03/04 18:36:54
[#1161745] 2024/03/04 18:36:54 ssh -t builds@fra02.builds.sr.ht connect 1161745
[#1161745] 2024/03/04 18:36:54
[#1161745] 2024/03/04 18:36:54 After logging in, the deadline is increased to your remaining build time.
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH olang] string_view: fix stack buffer overflow on to_u32
@ 2024-03-02 20:01 Johnny Richard
2024-03-02 19:02 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-02 20:01 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
src/string_view.c | 2 +-
tests/unit/string_view_test.c | 4 ++++
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/string_view.c b/src/string_view.c
index 084f417..646dabd 100644
--- a/src/string_view.c
+++ b/src/string_view.c
@@ -40,7 +40,7 @@ uint32_t
string_view_to_u32(string_view_t str)
{
char ret[str.size + 1];
- ret[str.size + 1] = 0;
+ ret[str.size] = 0;
memcpy(ret, str.chars, str.size);
return atoi(ret);
}
diff --git a/tests/unit/string_view_test.c b/tests/unit/string_view_test.c
index 1d8627f..fe3dacb 100644
--- a/tests/unit/string_view_test.c
+++ b/tests/unit/string_view_test.c
@@ -48,6 +48,10 @@ string_view_to_u32_test(const MunitParameter params[], void *user_data_or_fixtur
assert_uint32(string_view_to_u32(str), ==, 69);
+ str = (string_view_t) { .chars = "39;", .size = 2 };
+
+ assert_uint32(string_view_to_u32(str), ==, 39);
+
return MUNIT_OK;
}
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang] string_view: add n + 1 test to string_view_to_u32 function
@ 2024-03-02 19:02 Johnny Richard
2024-03-02 18:03 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-03-02 19:02 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
tests/unit/string_view_test.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tests/unit/string_view_test.c b/tests/unit/string_view_test.c
index 1d8627f..fe3dacb 100644
--- a/tests/unit/string_view_test.c
+++ b/tests/unit/string_view_test.c
@@ -48,6 +48,10 @@ string_view_to_u32_test(const MunitParameter params[], void *user_data_or_fixtur
assert_uint32(string_view_to_u32(str), ==, 69);
+ str = (string_view_t) { .chars = "39;", .size = 2 };
+
+ assert_uint32(string_view_to_u32(str), ==, 39);
+
return MUNIT_OK;
}
--
2.44.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v2] utils: add arena
@ 2024-02-20 16:39 Carlos Maniero
2024-02-20 16:45 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-02-20 16:39 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
This is a simple implementation of arena. At this point arena is not
expandable. Since the size of arena is predicable and basically used
used to store pointers, but we can make it more robust in the future.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
v2:
- fix lint
- improve commit message
.gitignore | 2 +-
Makefile | 8 ++++--
src/arena.c | 53 +++++++++++++++++++++++++++++++++++
src/arena.h | 41 +++++++++++++++++++++++++++
tests/unit/Makefile | 19 ++++++++-----
tests/unit/arena_test.c | 62 +++++++++++++++++++++++++++++++++++++++++
6 files changed, 175 insertions(+), 10 deletions(-)
create mode 100644 src/arena.c
create mode 100644 src/arena.h
create mode 100644 tests/unit/arena_test.c
diff --git a/.gitignore b/.gitignore
index 92496d7..fc7d161 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,4 +2,4 @@
build
*.o
docs/site.tar.gz
-tests/integration/*_test
+tests/*/*_test
diff --git a/Makefile b/Makefile
index 555029f..ea6ba53 100644
--- a/Makefile
+++ b/Makefile
@@ -20,11 +20,13 @@ $(BUILD_DIR):
linter: $(SRCS) $(HEADERS)
clang-format --dry-run --Werror $?
$(MAKE) -C tests/integration/ linter
+ $(MAKE) -C tests/unit/ linter
.PHONY: linter-fix
linter-fix: $(SRCS) $(HEADERS)
clang-format -i $?
$(MAKE) -C tests/integration/ linter-fix
+ $(MAKE) -C tests/unit/ linter-fix
.PHONY: integration-test
integration-test:
@@ -33,12 +35,14 @@ integration-test:
.PHONY: unit-test
unit-test:
+ $(MAKE)
$(MAKE) -C tests/unit/
.PHONY: check
check:
- $(MAKE) integration-test
- $(MAKE) unit-test
+ $(MAKE)
+ $(MAKE) -C tests/integration/
+ $(MAKE) -C tests/unit/
.PHONY: docs
docs:
diff --git a/src/arena.c b/src/arena.c
new file mode 100644
index 0000000..3d084ab
--- /dev/null
+++ b/src/arena.c
@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#include "arena.h"
+#include <stdlib.h>
+#include <stdio.h>
+
+arena_t
+arena_new(size_t size)
+{
+ arena_t arena;
+ arena.offset = 0;
+ arena.region = malloc(sizeof(uint8_t) * size);
+ arena.size = size;
+ return arena;
+}
+
+void *
+arena_alloc(arena_t *arena, size_t size)
+{
+ if ((arena->offset + size) > arena->size) {
+ return NULL;
+ }
+ void *pointer = arena->region + arena->offset;
+ arena->offset += size;
+ return pointer;
+}
+
+void
+arena_release(arena_t *arena)
+{
+ arena->offset = 0;
+}
+
+void
+arena_free(arena_t *arena)
+{
+ arena->size = 0;
+ free(arena->region);
+}
diff --git a/src/arena.h b/src/arena.h
new file mode 100644
index 0000000..a3e97aa
--- /dev/null
+++ b/src/arena.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#ifndef ARENA_H
+#define ARENA_H
+#include <stdlib.h>
+#include <stdint.h>
+
+typedef struct arena
+{
+ size_t offset;
+ size_t size;
+ uint8_t *region;
+} arena_t;
+
+arena_t
+arena_new(size_t size);
+
+void *
+arena_alloc(arena_t *arena, size_t size);
+
+void
+arena_release(arena_t *arena);
+
+void
+arena_free(arena_t *arena);
+
+#endif
diff --git a/tests/unit/Makefile b/tests/unit/Makefile
index ab250cf..7c6a8b3 100644
--- a/tests/unit/Makefile
+++ b/tests/unit/Makefile
@@ -1,10 +1,11 @@
-SRCS := $(wildcard *_test.c)
-OBJS := $(patsubst %_test.c, %_test.o, $(SRCS))
-CFLAGS := -I../../src -I../shared
-TESTS := $(patsubst %_test.c, %_test, $(SRCS))
-EXEC_TESTS := $(patsubst %_test, ./%_test, $(TESTS))
-MUNIT_SRC := ../shared/munit.c
-MUNIT := ./munit.o
+SRCS := $(wildcard *_test.c)
+OBJS := $(patsubst %_test.c, %_test.o, $(SRCS))
+SUBJECT_OBJS := $(filter-out ../../build/0c.o, $(wildcard ../../build/*.o))
+CFLAGS := -I../../src -I../shared
+TESTS := $(patsubst %_test.c, %_test, $(SRCS))
+EXEC_TESTS := $(patsubst %_test, ./%_test, $(TESTS))
+MUNIT_SRC := ../shared/munit.c
+MUNIT := ./munit.o
.PHONY: all
all: $(MUNIT) $(TESTS)
@@ -15,6 +16,7 @@ all: $(MUNIT) $(TESTS)
.PHONY: clean
clean:
$(RM) *.o *_test
+ $(RM) -rfv lib
.PHONY: linter
linter: $(SRCS)
@@ -24,5 +26,8 @@ linter: $(SRCS)
linter-fix: $(SRCS)
clang-format -i $?
+%_test: $(MUNIT) $(SUBJECT_OBJS) %_test.c
+ $(CC) $? $(CFLAGS) -o $@
+
$(MUNIT):
$(CC) -c $(MUNIT_SRC) $(CFLAGS) -o $(MUNIT)
diff --git a/tests/unit/arena_test.c b/tests/unit/arena_test.c
new file mode 100644
index 0000000..6310795
--- /dev/null
+++ b/tests/unit/arena_test.c
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#define MUNIT_ENABLE_ASSERT_ALIASES
+#include "arena.h"
+#include "munit.h"
+
+static MunitResult
+arena_test(const MunitParameter params[], void *user_data_or_fixture)
+{
+ arena_t arena = arena_new(sizeof(int) * 2);
+
+ int *a = arena_alloc(&arena, sizeof(int));
+ *a = 1;
+
+ int *b = arena_alloc(&arena, sizeof(int));
+ *b = 2;
+
+ munit_assert_int(*a, ==, 1);
+ munit_assert_int(*b, ==, 2);
+
+ arena_release(&arena);
+
+ int *c = arena_alloc(&arena, sizeof(int));
+ *c = 3;
+
+ munit_assert_int(*c, ==, 3);
+ munit_assert_int(*a, ==, 3);
+ munit_assert_int(*b, ==, 2);
+
+ munit_assert_ptr_not_null(arena_alloc(&arena, sizeof(int)));
+ munit_assert_ptr_null(arena_alloc(&arena, 1));
+
+ arena_free(&arena);
+
+ return MUNIT_OK;
+}
+
+static MunitTest tests[] = { { "/arena_test", arena_test, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL },
+ { NULL, NULL, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL } };
+
+static const MunitSuite suite = { "/cli_test", tests, NULL, 1, MUNIT_SUITE_OPTION_NONE };
+
+int
+main(int argc, char *argv[])
+{
+ return munit_suite_main(&suite, NULL, argc, argv);
+ return EXIT_SUCCESS;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang] utils: add arena
@ 2024-02-20 16:10 Carlos Maniero
2024-02-20 16:15 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-02-20 16:10 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
This is a simple implementation of arena. At this point arena is not
expandable. We can add support to it if neded after we implement linked
lists.
Since arena was the first code unit-tested, I've completed the unit
tests setup at this commit.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
.gitignore | 2 +-
Makefile | 8 ++++--
src/arena.c | 53 +++++++++++++++++++++++++++++++++++
src/arena.h | 41 +++++++++++++++++++++++++++
tests/unit/Makefile | 19 ++++++++-----
tests/unit/arena_test.c | 62 +++++++++++++++++++++++++++++++++++++++++
6 files changed, 175 insertions(+), 10 deletions(-)
create mode 100644 src/arena.c
create mode 100644 src/arena.h
create mode 100644 tests/unit/arena_test.c
diff --git a/.gitignore b/.gitignore
index 92496d7..fc7d161 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,4 +2,4 @@
build
*.o
docs/site.tar.gz
-tests/integration/*_test
+tests/*/*_test
diff --git a/Makefile b/Makefile
index 555029f..ea6ba53 100644
--- a/Makefile
+++ b/Makefile
@@ -20,11 +20,13 @@ $(BUILD_DIR):
linter: $(SRCS) $(HEADERS)
clang-format --dry-run --Werror $?
$(MAKE) -C tests/integration/ linter
+ $(MAKE) -C tests/unit/ linter
.PHONY: linter-fix
linter-fix: $(SRCS) $(HEADERS)
clang-format -i $?
$(MAKE) -C tests/integration/ linter-fix
+ $(MAKE) -C tests/unit/ linter-fix
.PHONY: integration-test
integration-test:
@@ -33,12 +35,14 @@ integration-test:
.PHONY: unit-test
unit-test:
+ $(MAKE)
$(MAKE) -C tests/unit/
.PHONY: check
check:
- $(MAKE) integration-test
- $(MAKE) unit-test
+ $(MAKE)
+ $(MAKE) -C tests/integration/
+ $(MAKE) -C tests/unit/
.PHONY: docs
docs:
diff --git a/src/arena.c b/src/arena.c
new file mode 100644
index 0000000..3d084ab
--- /dev/null
+++ b/src/arena.c
@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#include "arena.h"
+#include <stdlib.h>
+#include <stdio.h>
+
+arena_t
+arena_new(size_t size)
+{
+ arena_t arena;
+ arena.offset = 0;
+ arena.region = malloc(sizeof(uint8_t) * size);
+ arena.size = size;
+ return arena;
+}
+
+void *
+arena_alloc(arena_t *arena, size_t size)
+{
+ if ((arena->offset + size) > arena->size) {
+ return NULL;
+ }
+ void *pointer = arena->region + arena->offset;
+ arena->offset += size;
+ return pointer;
+}
+
+void
+arena_release(arena_t *arena)
+{
+ arena->offset = 0;
+}
+
+void
+arena_free(arena_t *arena)
+{
+ arena->size = 0;
+ free(arena->region);
+}
diff --git a/src/arena.h b/src/arena.h
new file mode 100644
index 0000000..a3e97aa
--- /dev/null
+++ b/src/arena.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#ifndef ARENA_H
+#define ARENA_H
+#include <stdlib.h>
+#include <stdint.h>
+
+typedef struct arena
+{
+ size_t offset;
+ size_t size;
+ uint8_t *region;
+} arena_t;
+
+arena_t
+arena_new(size_t size);
+
+void *
+arena_alloc(arena_t *arena, size_t size);
+
+void
+arena_release(arena_t *arena);
+
+void
+arena_free(arena_t *arena);
+
+#endif
diff --git a/tests/unit/Makefile b/tests/unit/Makefile
index ab250cf..7c6a8b3 100644
--- a/tests/unit/Makefile
+++ b/tests/unit/Makefile
@@ -1,10 +1,11 @@
-SRCS := $(wildcard *_test.c)
-OBJS := $(patsubst %_test.c, %_test.o, $(SRCS))
-CFLAGS := -I../../src -I../shared
-TESTS := $(patsubst %_test.c, %_test, $(SRCS))
-EXEC_TESTS := $(patsubst %_test, ./%_test, $(TESTS))
-MUNIT_SRC := ../shared/munit.c
-MUNIT := ./munit.o
+SRCS := $(wildcard *_test.c)
+OBJS := $(patsubst %_test.c, %_test.o, $(SRCS))
+SUBJECT_OBJS := $(filter-out ../../build/0c.o, $(wildcard ../../build/*.o))
+CFLAGS := -I../../src -I../shared
+TESTS := $(patsubst %_test.c, %_test, $(SRCS))
+EXEC_TESTS := $(patsubst %_test, ./%_test, $(TESTS))
+MUNIT_SRC := ../shared/munit.c
+MUNIT := ./munit.o
.PHONY: all
all: $(MUNIT) $(TESTS)
@@ -15,6 +16,7 @@ all: $(MUNIT) $(TESTS)
.PHONY: clean
clean:
$(RM) *.o *_test
+ $(RM) -rfv lib
.PHONY: linter
linter: $(SRCS)
@@ -24,5 +26,8 @@ linter: $(SRCS)
linter-fix: $(SRCS)
clang-format -i $?
+%_test: $(MUNIT) $(SUBJECT_OBJS) %_test.c
+ $(CC) $? $(CFLAGS) -o $@
+
$(MUNIT):
$(CC) -c $(MUNIT_SRC) $(CFLAGS) -o $(MUNIT)
diff --git a/tests/unit/arena_test.c b/tests/unit/arena_test.c
new file mode 100644
index 0000000..6310795
--- /dev/null
+++ b/tests/unit/arena_test.c
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#define MUNIT_ENABLE_ASSERT_ALIASES
+#include "arena.h"
+#include "munit.h"
+
+static MunitResult
+arena_test(const MunitParameter params[], void *user_data_or_fixture)
+{
+ arena_t arena = arena_new(sizeof(int) * 2);
+
+ int *a = arena_alloc(&arena, sizeof(int));
+ *a = 1;
+
+ int *b = arena_alloc(&arena, sizeof(int));
+ *b = 2;
+
+ munit_assert_int(*a, ==, 1);
+ munit_assert_int(*b, ==, 2);
+
+ arena_release(&arena);
+
+ int *c = arena_alloc(&arena, sizeof(int));
+ *c = 3;
+
+ munit_assert_int(*c, ==, 3);
+ munit_assert_int(*a, ==, 3);
+ munit_assert_int(*b, ==, 2);
+
+ munit_assert_ptr_not_null(arena_alloc(&arena, sizeof(int)));
+ munit_assert_ptr_null(arena_alloc(&arena, 1));
+
+ arena_free(&arena);
+
+ return MUNIT_OK;
+}
+
+static MunitTest tests[] = { { "/arena_test", arena_test, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL },
+ { NULL, NULL, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL } };
+
+static const MunitSuite suite = { "/cli_test", tests, NULL, 1, MUNIT_SUITE_OPTION_NONE };
+
+int
+main(int argc, char *argv[])
+{
+ return munit_suite_main(&suite, NULL, argc, argv);
+ return EXIT_SUCCESS;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v4 4/4] lexer: test: add integration tests for --dump-tokens
@ 2024-02-19 21:04 Johnny Richard
2024-02-19 20:07 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-02-19 21:04 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero, Johnny Richard
From: Carlos Maniero <carlos@maniero.me>
We want to test the 0c compiler in a black box way. This test explores
`pipe` in order to handle input/output data, and it tests the
--dump-tokens against the examples/main_exit.0.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
src/0c.c | 1 -
tests/integration/cli_runner.c | 47 ++++++++++++++++++++++++++++++----
tests/integration/cli_runner.h | 1 +
tests/integration/cli_test.c | 14 ++++++++++
4 files changed, 57 insertions(+), 6 deletions(-)
diff --git a/src/0c.c b/src/0c.c
index e84559d..978b770 100644
--- a/src/0c.c
+++ b/src/0c.c
@@ -75,7 +75,6 @@ main(int argc, char **argv)
string_view_t file_content = read_entire_file(opts.file_path);
- // TODO: missing integration test for lexer tokenizing
lexer_t lexer = { 0 };
lexer_init(&lexer, file_content);
diff --git a/tests/integration/cli_runner.c b/tests/integration/cli_runner.c
index 0531bcc..7e4fe9a 100644
--- a/tests/integration/cli_runner.c
+++ b/tests/integration/cli_runner.c
@@ -20,6 +20,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
+#include <sys/wait.h>
#include <unistd.h>
#define OLANG_COMPILER_PATH "../../0c"
@@ -62,16 +63,52 @@ create_tmp_file_name(char *file_name)
}
cli_result_t
-cli_runner_compiler_dump_tokens(char *src)
+cli_runner_compiler(char *src, char *args[])
{
assert_compiler_exists();
- cli_result_t result;
+ cli_result_t result = { 0 };
create_tmp_file_name(result.program_path);
- char command[1024];
- sprintf(command, "%s %s --dump-tokens", OLANG_COMPILER_PATH, src);
+ int fd_link[2];
+
+ if (pipe(fd_link) == -1) {
+ perror("pipe error.");
+ exit(1);
+ }
+
+ pid_t pid = fork();
+
+ if (pid == -1) {
+ perror("fork error.");
+ exit(1);
+ }
+
+ if (pid == 0) {
+ dup2(fd_link[1], STDOUT_FILENO);
+ close(fd_link[0]);
+ close(fd_link[1]);
+
+ execv(OLANG_COMPILER_PATH, args);
+ perror("execl error.");
+ exit(127);
+ } else {
+ close(fd_link[1]);
+ if (read(fd_link[0], result.compiler_output, sizeof(result.compiler_output)) == -1) {
+ perror("read error.");
+ exit(1);
+ }
+ int status;
+ waitpid(pid, &status, 0);
+ result.exit_code = WEXITSTATUS(status);
+ }
- result.exit_code = system(command);
return result;
}
+
+cli_result_t
+cli_runner_compiler_dump_tokens(char *src)
+{
+ char *program_args[] = { "0c", "--dump-tokens", src, NULL };
+ return cli_runner_compiler(src, program_args);
+}
diff --git a/tests/integration/cli_runner.h b/tests/integration/cli_runner.h
index 8f4d69a..7ce4e7b 100644
--- a/tests/integration/cli_runner.h
+++ b/tests/integration/cli_runner.h
@@ -20,6 +20,7 @@ typedef struct cli_result_t
{
int exit_code;
char program_path[255];
+ char compiler_output[1024];
} cli_result_t;
cli_result_t
diff --git a/tests/integration/cli_test.c b/tests/integration/cli_test.c
index ce2ed91..1fd70c7 100644
--- a/tests/integration/cli_test.c
+++ b/tests/integration/cli_test.c
@@ -23,6 +23,20 @@ test_cli_hello_file(const MunitParameter params[], void *user_data_or_fixture)
{
cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/main_exit.0");
munit_assert_int(compilation_result.exit_code, ==, 0);
+ munit_assert_string_equal(compilation_result.compiler_output,
+ "../../examples/main_exit.0:1:1: <fn>\n"
+ "../../examples/main_exit.0:1:4: <identifier>\n"
+ "../../examples/main_exit.0:1:8: <(>\n"
+ "../../examples/main_exit.0:1:9: <)>\n"
+ "../../examples/main_exit.0:1:10: <:>\n"
+ "../../examples/main_exit.0:1:12: <identifier>\n"
+ "../../examples/main_exit.0:1:16: <{>\n"
+ "../../examples/main_exit.0:1:17: <line_feed>\n"
+ "../../examples/main_exit.0:2:3: <return>\n"
+ "../../examples/main_exit.0:2:10: <number>\n"
+ "../../examples/main_exit.0:2:11: <line_feed>\n"
+ "../../examples/main_exit.0:3:1: <}>\n"
+ "../../examples/main_exit.0:3:2: <line_feed>\n");
return MUNIT_OK;
}
--
2.43.2
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v2 2/2] lexer: create --dump-tokens cli command
@ 2024-02-19 1:23 Johnny Richard
2024-02-19 0:27 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-02-19 1:23 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
This patch introduces the dump tokens interface and create the initial
setup for lexical analysis.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
Changes:
- v2 make linter-fix due to linter build failure
examples/main_exit.0 | 3 +
src/0c.c | 121 ++++++++++++++++++++++-
src/lexer.c | 224 +++++++++++++++++++++++++++++++++++++++++++
src/lexer.h | 74 ++++++++++++++
4 files changed, 420 insertions(+), 2 deletions(-)
create mode 100644 examples/main_exit.0
create mode 100644 src/lexer.c
create mode 100644 src/lexer.h
diff --git a/examples/main_exit.0 b/examples/main_exit.0
new file mode 100644
index 0000000..c86fc68
--- /dev/null
+++ b/examples/main_exit.0
@@ -0,0 +1,3 @@
+fn main(): u32 {
+ return 0
+}
diff --git a/src/0c.c b/src/0c.c
index 33ac945..e5199a7 100644
--- a/src/0c.c
+++ b/src/0c.c
@@ -14,8 +14,125 @@
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
+#include <errno.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "lexer.h"
+#include "string_view.h"
+
+typedef struct cli_args
+{
+ int argc;
+ char **argv;
+} cli_args_t;
+
+char *
+cli_args_shift(cli_args_t *args);
+
+typedef struct cli_opts
+{
+ // TODO: create man page instruction for --dump-tokens option
+ bool dump_tokens;
+ char *file_path;
+} cli_opts_t;
+
+void
+print_usage(FILE *stream, char *prog);
+
+string_view_t
+read_entire_file(char *file_path);
+
int
-main(void)
+main(int argc, char **argv)
+{
+ cli_args_t args = { .argc = argc, .argv = argv };
+ cli_opts_t opts = { 0 };
+
+ char *prog = cli_args_shift(&args);
+
+ if (argc != 3) {
+ print_usage(stderr, prog);
+ return EXIT_FAILURE;
+ }
+
+ for (char *arg = cli_args_shift(&args); arg != NULL; arg = cli_args_shift(&args)) {
+ if (strcmp(arg, "--dump-tokens") == 0) {
+ opts.dump_tokens = true;
+ } else {
+ opts.file_path = arg;
+ }
+ }
+
+ if (!opts.dump_tokens) {
+ print_usage(stderr, prog);
+ return EXIT_FAILURE;
+ }
+
+ string_view_t file_content = read_entire_file(opts.file_path);
+
+ // TODO: missing integration test for lexer tokenizing
+ lexer_t lexer = { 0 };
+ lexer_init(&lexer, file_content);
+
+ token_t token = { 0 };
+ lexer_next_token(&lexer, &token);
+ while (token.kind != TOKEN_EOF) {
+ printf("%s:%lu:%lu: <%s>\n",
+ opts.file_path,
+ token.location.row + 1,
+ (token.location.offset - token.location.bol) + 1,
+ token_kind_to_cstr(token.kind));
+ lexer_next_token(&lexer, &token);
+ }
+
+ free(file_content.chars);
+
+ return EXIT_SUCCESS;
+}
+
+char *
+cli_args_shift(cli_args_t *args)
+{
+ if (args->argc == 0)
+ return NULL;
+ --(args->argc);
+ return *(args->argv)++;
+}
+
+void
+print_usage(FILE *stream, char *prog)
+{
+ fprintf(stream, "usage: %s <source.0> --dump-tokens\n", prog);
+}
+
+string_view_t
+read_entire_file(char *file_path)
{
- return 0;
+ FILE *stream = fopen(file_path, "rb");
+
+ if (stream == NULL) {
+ fprintf(stderr, "Could not open file %s: %s\n", file_path, strerror(errno));
+ exit(EXIT_FAILURE);
+ }
+
+ string_view_t file_content = { 0 };
+
+ fseek(stream, 0, SEEK_END);
+ file_content.size = ftell(stream);
+ fseek(stream, 0, SEEK_SET);
+
+ file_content.chars = (char *)malloc(file_content.size);
+
+ if (file_content.chars == NULL) {
+ fprintf(stderr, "Could not read file %s: %s\n", file_path, strerror(errno));
+ exit(EXIT_FAILURE);
+ }
+
+ fread(file_content.chars, 1, file_content.size, stream);
+ fclose(stream);
+
+ return file_content;
}
diff --git a/src/lexer.c b/src/lexer.c
new file mode 100644
index 0000000..544a54d
--- /dev/null
+++ b/src/lexer.c
@@ -0,0 +1,224 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#include "lexer.h"
+
+#include <assert.h>
+#include <ctype.h>
+#include <stdbool.h>
+
+void
+lexer_init(lexer_t *lexer, string_view_t source)
+{
+ assert(lexer);
+ lexer->source = source;
+ lexer->offset = 0;
+ lexer->row = 0;
+ lexer->bol = 0;
+}
+
+static char
+lexer_next_char(lexer_t *lexer);
+
+static void
+lexer_skip_char(lexer_t *lexer);
+
+static bool
+lexer_is_eof(lexer_t *lexer);
+
+static bool
+lexer_is_not_eof(lexer_t *lexer);
+
+static bool
+_isspace(char c);
+
+static void
+lexer_init_char_token(lexer_t *lexer, token_t *token, token_kind_t kind);
+
+static void
+lexer_init_str_token(lexer_t *lexer, token_t *token, token_kind_t kind, size_t start_offset);
+
+static token_kind_t
+lexer_str_to_token_kind(string_view_t text);
+
+void
+lexer_next_token(lexer_t *lexer, token_t *token)
+{
+ if (lexer_is_eof(lexer)) {
+ *token = (token_t){ .kind = TOKEN_EOF };
+ return;
+ }
+
+ char current_char = lexer_next_char(lexer);
+
+ if (_isspace(current_char)) {
+ while (_isspace(current_char) && lexer_is_not_eof(lexer)) {
+ lexer_skip_char(lexer);
+ current_char = lexer_next_char(lexer);
+ }
+ }
+
+ while (lexer_is_not_eof(lexer)) {
+ if (isalpha(current_char)) {
+ size_t start_offset = lexer->offset;
+ while (isalnum(current_char) && lexer_is_not_eof(lexer)) {
+ lexer_skip_char(lexer);
+ current_char = lexer_next_char(lexer);
+ }
+
+ string_view_t text = { .chars = lexer->source.chars + start_offset, .size = lexer->offset - start_offset };
+
+ lexer_init_str_token(lexer, token, lexer_str_to_token_kind(text), start_offset);
+ return;
+ }
+
+ if (isdigit(current_char)) {
+ size_t start_offset = lexer->offset;
+ while (isdigit(current_char) && lexer_is_not_eof(lexer)) {
+ lexer_skip_char(lexer);
+ current_char = lexer_next_char(lexer);
+ }
+
+ lexer_init_str_token(lexer, token, TOKEN_NUMBER, start_offset);
+ return;
+ }
+
+ switch (current_char) {
+ case '(': {
+ lexer_init_char_token(lexer, token, TOKEN_OPAREN);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case ')': {
+ lexer_init_char_token(lexer, token, TOKEN_CPAREN);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case ':': {
+ lexer_init_char_token(lexer, token, TOKEN_COLON);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case '{': {
+ lexer_init_char_token(lexer, token, TOKEN_OCURLY);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case '}': {
+ lexer_init_char_token(lexer, token, TOKEN_CCURLY);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case '\n': {
+ lexer_init_char_token(lexer, token, TOKEN_LF);
+ lexer_skip_char(lexer);
+ return;
+ }
+ default: {
+ lexer_init_char_token(lexer, token, TOKEN_UNKNOWN);
+ lexer_skip_char(lexer);
+ return;
+ }
+ }
+ }
+
+ if (lexer_is_eof(lexer)) {
+ *token = (token_t){ .kind = TOKEN_EOF };
+ return;
+ }
+}
+
+static char *token_kind_str_table[] = {
+ [TOKEN_UNKNOWN] = "unknown", [TOKEN_IDENTIFIER] = "identifier",
+ [TOKEN_NUMBER] = "number", [TOKEN_FN] = "fn",
+ [TOKEN_RETURN] = "return", [TOKEN_LF] = "line_feed",
+ [TOKEN_OPAREN] = "(", [TOKEN_CPAREN] = ")",
+ [TOKEN_COLON] = ":", [TOKEN_OCURLY] = "{",
+ [TOKEN_CCURLY] = "}", [TOKEN_EOF] = "EOF",
+};
+
+char *
+token_kind_to_cstr(token_kind_t kind)
+{
+ assert(kind < sizeof(token_kind_str_table));
+ return token_kind_str_table[kind];
+}
+
+static char
+lexer_next_char(lexer_t *lexer)
+{
+ return lexer->source.chars[lexer->offset];
+}
+
+static void
+lexer_skip_char(lexer_t *lexer)
+{
+ assert(lexer->offset < lexer->source.size);
+ if (lexer->source.chars[lexer->offset] == '\n') {
+ lexer->row++;
+ lexer->bol = ++lexer->offset;
+ } else {
+ lexer->offset++;
+ }
+}
+
+static bool
+lexer_is_eof(lexer_t *lexer)
+{
+ return lexer->offset >= lexer->source.size;
+}
+
+static bool
+lexer_is_not_eof(lexer_t *lexer)
+{
+ return !lexer_is_eof(lexer);
+}
+
+static bool
+_isspace(char c)
+{
+ return c == ' ' || c == '\f' || c == '\r' || c == '\t' || c == '\v';
+}
+
+static void
+lexer_init_char_token(lexer_t *lexer, token_t *token, token_kind_t kind)
+{
+ string_view_t str = { .chars = lexer->source.chars + lexer->offset, .size = 1 };
+ token_loc_t location = { .offset = lexer->offset, .row = lexer->row, .bol = lexer->bol };
+ *token = (token_t){ .kind = kind, .value = str, .location = location };
+}
+
+static void
+lexer_init_str_token(lexer_t *lexer, token_t *token, token_kind_t kind, size_t start_offset)
+{
+ string_view_t str = { .chars = lexer->source.chars + start_offset, .size = lexer->offset - start_offset };
+ token_loc_t location = { .offset = start_offset, .row = lexer->row, .bol = lexer->bol };
+ *token = (token_t){ .kind = kind, .value = str, .location = location };
+}
+
+static token_kind_t
+lexer_str_to_token_kind(string_view_t text)
+{
+ if (string_view_eq_to_cstr(text, "return")) {
+ return TOKEN_RETURN;
+ }
+
+ if (string_view_eq_to_cstr(text, "fn")) {
+ return TOKEN_FN;
+ }
+
+ return TOKEN_IDENTIFIER;
+}
diff --git a/src/lexer.h b/src/lexer.h
new file mode 100644
index 0000000..8c09e02
--- /dev/null
+++ b/src/lexer.h
@@ -0,0 +1,74 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#ifndef LEXER_H
+#define LEXER_H
+
+#include "string_view.h"
+#include <stdint.h>
+
+typedef struct lexer
+{
+ string_view_t source;
+ size_t offset;
+ size_t row;
+ size_t bol;
+} lexer_t;
+
+typedef enum token_kind
+{
+ TOKEN_UNKNOWN,
+ TOKEN_IDENTIFIER,
+ TOKEN_NUMBER,
+
+ // Keywords
+ TOKEN_FN,
+ TOKEN_RETURN,
+
+ // Single char
+ TOKEN_LF,
+ TOKEN_OPAREN,
+ TOKEN_CPAREN,
+ TOKEN_COLON,
+ TOKEN_OCURLY,
+ TOKEN_CCURLY,
+ TOKEN_EOF
+} token_kind_t;
+
+typedef struct token_loc
+{
+ size_t offset;
+ size_t row;
+ size_t bol;
+} token_loc_t;
+
+typedef struct token
+{
+ token_kind_t kind;
+ string_view_t value;
+ token_loc_t location;
+} token_t;
+
+void
+lexer_init(lexer_t *lexer, string_view_t source);
+
+void
+lexer_next_token(lexer_t *lexer, token_t *token);
+
+char *
+token_kind_to_cstr(token_kind_t kind);
+
+#endif /* LEXER_H */
--
2.43.2
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang 2/2] lexer: create --dump-tokens cli command
@ 2024-02-19 1:15 Johnny Richard
2024-02-19 0:20 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-02-19 1:15 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
This patch introduces the dump tokens interface and create the initial
setup for lexical analysis.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
examples/main_exit.0 | 3 +
src/0c.c | 121 ++++++++++++++++++++++-
src/lexer.c | 224 +++++++++++++++++++++++++++++++++++++++++++
src/lexer.h | 74 ++++++++++++++
4 files changed, 420 insertions(+), 2 deletions(-)
create mode 100644 examples/main_exit.0
create mode 100644 src/lexer.c
create mode 100644 src/lexer.h
diff --git a/examples/main_exit.0 b/examples/main_exit.0
new file mode 100644
index 0000000..c86fc68
--- /dev/null
+++ b/examples/main_exit.0
@@ -0,0 +1,3 @@
+fn main(): u32 {
+ return 0
+}
diff --git a/src/0c.c b/src/0c.c
index 33ac945..e5199a7 100644
--- a/src/0c.c
+++ b/src/0c.c
@@ -14,8 +14,125 @@
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
+#include <errno.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "lexer.h"
+#include "string_view.h"
+
+typedef struct cli_args
+{
+ int argc;
+ char **argv;
+} cli_args_t;
+
+char *
+cli_args_shift(cli_args_t *args);
+
+typedef struct cli_opts
+{
+ // TODO: create man page instruction for --dump-tokens option
+ bool dump_tokens;
+ char *file_path;
+} cli_opts_t;
+
+void
+print_usage(FILE *stream, char *prog);
+
+string_view_t
+read_entire_file(char *file_path);
+
int
-main(void)
+main(int argc, char **argv)
+{
+ cli_args_t args = { .argc = argc, .argv = argv };
+ cli_opts_t opts = { 0 };
+
+ char *prog = cli_args_shift(&args);
+
+ if (argc != 3) {
+ print_usage(stderr, prog);
+ return EXIT_FAILURE;
+ }
+
+ for (char *arg = cli_args_shift(&args); arg != NULL; arg = cli_args_shift(&args)) {
+ if (strcmp(arg, "--dump-tokens") == 0) {
+ opts.dump_tokens = true;
+ } else {
+ opts.file_path = arg;
+ }
+ }
+
+ if (!opts.dump_tokens) {
+ print_usage(stderr, prog);
+ return EXIT_FAILURE;
+ }
+
+ string_view_t file_content = read_entire_file(opts.file_path);
+
+ // TODO: missing integration test for lexer tokenizing
+ lexer_t lexer = { 0 };
+ lexer_init(&lexer, file_content);
+
+ token_t token = { 0 };
+ lexer_next_token(&lexer, &token);
+ while (token.kind != TOKEN_EOF) {
+ printf("%s:%lu:%lu: <%s>\n",
+ opts.file_path,
+ token.location.row + 1,
+ (token.location.offset - token.location.bol) + 1,
+ token_kind_to_cstr(token.kind));
+ lexer_next_token(&lexer, &token);
+ }
+
+ free(file_content.chars);
+
+ return EXIT_SUCCESS;
+}
+
+char *
+cli_args_shift(cli_args_t *args)
+{
+ if (args->argc == 0)
+ return NULL;
+ --(args->argc);
+ return *(args->argv)++;
+}
+
+void
+print_usage(FILE *stream, char *prog)
+{
+ fprintf(stream, "usage: %s <source.0> --dump-tokens\n", prog);
+}
+
+string_view_t
+read_entire_file(char *file_path)
{
- return 0;
+ FILE *stream = fopen(file_path, "rb");
+
+ if (stream == NULL) {
+ fprintf(stderr, "Could not open file %s: %s\n", file_path, strerror(errno));
+ exit(EXIT_FAILURE);
+ }
+
+ string_view_t file_content = { 0 };
+
+ fseek(stream, 0, SEEK_END);
+ file_content.size = ftell(stream);
+ fseek(stream, 0, SEEK_SET);
+
+ file_content.chars = (char *)malloc(file_content.size);
+
+ if (file_content.chars == NULL) {
+ fprintf(stderr, "Could not read file %s: %s\n", file_path, strerror(errno));
+ exit(EXIT_FAILURE);
+ }
+
+ fread(file_content.chars, 1, file_content.size, stream);
+ fclose(stream);
+
+ return file_content;
}
diff --git a/src/lexer.c b/src/lexer.c
new file mode 100644
index 0000000..7866a9a
--- /dev/null
+++ b/src/lexer.c
@@ -0,0 +1,224 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#include "lexer.h"
+
+#include <assert.h>
+#include <ctype.h>
+#include <stdbool.h>
+
+void
+lexer_init(lexer_t *lexer, string_view_t source)
+{
+ assert(lexer);
+ lexer->source = source;
+ lexer->offset = 0;
+ lexer->row = 0;
+ lexer->bol = 0;
+}
+
+static char
+lexer_next_char(lexer_t *lexer);
+
+static void
+lexer_skip_char(lexer_t *lexer);
+
+static bool
+lexer_is_eof(lexer_t *lexer);
+
+static bool
+lexer_is_not_eof(lexer_t *lexer);
+
+static bool
+_isspace(char c);
+
+static void
+lexer_init_char_token(lexer_t *lexer, token_t *token, token_kind_t kind);
+
+static void
+lexer_init_str_token(lexer_t *lexer, token_t *token, token_kind_t kind, size_t start_offset);
+
+static token_kind_t
+lexer_str_to_token_kind(string_view_t text);
+
+void
+lexer_next_token(lexer_t *lexer, token_t *token)
+{
+ if (lexer_is_eof(lexer)) {
+ *token = (token_t){ .kind = TOKEN_EOF };
+ return;
+ }
+
+ char current_char = lexer_next_char(lexer);
+
+ if (_isspace(current_char)) {
+ while (_isspace(current_char) && lexer_is_not_eof(lexer)) {
+ lexer_skip_char(lexer);
+ current_char = lexer_next_char(lexer);
+ }
+ }
+
+ while (lexer_is_not_eof(lexer)) {
+ if (isalpha(current_char)) {
+ size_t start_offset = lexer->offset;
+ while (isalnum(current_char) && lexer_is_not_eof(lexer)) {
+ lexer_skip_char(lexer);
+ current_char = lexer_next_char(lexer);
+ }
+
+ string_view_t text = { .chars = lexer->source.chars + start_offset, .size = lexer->offset - start_offset };
+
+ lexer_init_str_token(lexer, token, lexer_str_to_token_kind(text), start_offset);
+ return;
+ }
+
+ if (isdigit(current_char)) {
+ size_t start_offset = lexer->offset;
+ while (isdigit(current_char) && lexer_is_not_eof(lexer)) {
+ lexer_skip_char(lexer);
+ current_char = lexer_next_char(lexer);
+ }
+
+ lexer_init_str_token(lexer, token, TOKEN_NUMBER, start_offset);
+ return;
+ }
+
+ switch (current_char) {
+ case '(': {
+ lexer_init_char_token(lexer, token, TOKEN_OPAREN);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case ')': {
+ lexer_init_char_token(lexer, token, TOKEN_CPAREN);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case ':': {
+ lexer_init_char_token(lexer, token, TOKEN_COLON);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case '{': {
+ lexer_init_char_token(lexer, token, TOKEN_OCURLY);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case '}': {
+ lexer_init_char_token(lexer, token, TOKEN_CCURLY);
+ lexer_skip_char(lexer);
+ return;
+ }
+ case '\n': {
+ lexer_init_char_token(lexer, token, TOKEN_LF);
+ lexer_skip_char(lexer);
+ return;
+ }
+ default: {
+ lexer_init_char_token(lexer, token, TOKEN_UNKNOWN);
+ lexer_skip_char(lexer);
+ return;
+ }
+ }
+ }
+
+ if (lexer_is_eof(lexer)) {
+ *token = (token_t){ .kind = TOKEN_EOF };
+ return;
+ }
+}
+
+static char *token_kind_str_table[] = {
+ [TOKEN_UNKNOWN] = "unknown", [TOKEN_IDENTIFIER] = "identifier",
+ [TOKEN_NUMBER] = "number", [TOKEN_FN] = "fn",
+ [TOKEN_RETURN] = "return", [TOKEN_LF] = "line_feed",
+ [TOKEN_OPAREN] = "(", [TOKEN_CPAREN] = ")",
+ [TOKEN_COLON] = ":", [TOKEN_OCURLY] = "{",
+ [TOKEN_CCURLY] = "}", [TOKEN_EOF] = "EOF",
+};
+
+char *
+token_kind_to_cstr(token_kind_t kind)
+{
+ assert(kind < sizeof(token_kind_str_table));
+ return token_kind_str_table[kind];
+}
+
+static char
+lexer_next_char(lexer_t *lexer)
+{
+ return lexer->source.chars[lexer->offset];
+}
+
+static void
+lexer_skip_char(lexer_t *lexer)
+{
+ assert(lexer->offset < lexer->source.size);
+ if (lexer->source.chars[lexer->offset] == '\n') {
+ lexer->row++;
+ lexer->bol = ++lexer->offset;
+ } else {
+ lexer->offset++;
+ }
+}
+
+static bool
+lexer_is_eof(lexer_t *lexer)
+{
+ return lexer->offset >= lexer->source.size;
+}
+
+static bool
+lexer_is_not_eof(lexer_t *lexer)
+{
+ return !lexer_is_eof(lexer);
+}
+
+static bool
+_isspace(char c)
+{
+ return c == ' ' || c == '\f' || c == '\r' || c == '\t' || c == '\v';
+}
+
+static void
+lexer_init_char_token(lexer_t *lexer, token_t *token, token_kind_t kind)
+{
+ string_view_t str = { .chars = lexer->source.chars + lexer->offset, .size = 1 };
+ token_loc_t location = { .offset = lexer->offset, .row = lexer->row, .bol = lexer->bol };
+ *token = (token_t){ .kind = kind, .value = str, .location = location };
+}
+
+static void
+lexer_init_str_token(lexer_t *lexer, token_t *token, token_kind_t kind, size_t start_offset)
+{
+ string_view_t str = { .chars = lexer->source.chars + start_offset, .size = lexer->offset - start_offset };
+ token_loc_t location = { .offset = start_offset, .row = lexer->row, .bol = lexer->bol };
+ *token = (token_t){ .kind = kind, .value = str, .location = location };
+}
+
+static token_kind_t
+lexer_str_to_token_kind(string_view_t text)
+{
+ if (string_view_eq_to_cstr(text, "return")) {
+ return TOKEN_RETURN;
+ }
+
+ if (string_view_eq_to_cstr(text, "fn")) {
+ return TOKEN_FN;
+ }
+
+ return TOKEN_IDENTIFIER;
+}
diff --git a/src/lexer.h b/src/lexer.h
new file mode 100644
index 0000000..8c09e02
--- /dev/null
+++ b/src/lexer.h
@@ -0,0 +1,74 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#ifndef LEXER_H
+#define LEXER_H
+
+#include "string_view.h"
+#include <stdint.h>
+
+typedef struct lexer
+{
+ string_view_t source;
+ size_t offset;
+ size_t row;
+ size_t bol;
+} lexer_t;
+
+typedef enum token_kind
+{
+ TOKEN_UNKNOWN,
+ TOKEN_IDENTIFIER,
+ TOKEN_NUMBER,
+
+ // Keywords
+ TOKEN_FN,
+ TOKEN_RETURN,
+
+ // Single char
+ TOKEN_LF,
+ TOKEN_OPAREN,
+ TOKEN_CPAREN,
+ TOKEN_COLON,
+ TOKEN_OCURLY,
+ TOKEN_CCURLY,
+ TOKEN_EOF
+} token_kind_t;
+
+typedef struct token_loc
+{
+ size_t offset;
+ size_t row;
+ size_t bol;
+} token_loc_t;
+
+typedef struct token
+{
+ token_kind_t kind;
+ string_view_t value;
+ token_loc_t location;
+} token_t;
+
+void
+lexer_init(lexer_t *lexer, string_view_t source);
+
+void
+lexer_next_token(lexer_t *lexer, token_t *token);
+
+char *
+token_kind_to_cstr(token_kind_t kind);
+
+#endif /* LEXER_H */
--
2.43.2
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang 2/2] Revert "docs: add sphinx documentation support"
@ 2024-02-17 13:46 Carlos Maniero
2024-02-17 13:51 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-02-17 13:46 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
This reverts commit 0a48a94d9ade6ac92bbc42c3e7681f326d4f8d70.
---
.gitignore | 1 -
docs/Makefile | 20 --------------------
docs/conf.py | 28 ----------------------------
docs/index.rst | 13 -------------
4 files changed, 62 deletions(-)
delete mode 100644 docs/Makefile
delete mode 100644 docs/conf.py
delete mode 100644 docs/index.rst
diff --git a/.gitignore b/.gitignore
index 77f3000..b140d08 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,3 @@
0c
build
*.o
-docs/_build
diff --git a/docs/Makefile b/docs/Makefile
deleted file mode 100644
index d4bb2cb..0000000
--- a/docs/Makefile
+++ /dev/null
@@ -1,20 +0,0 @@
-# Minimal makefile for Sphinx documentation
-#
-
-# You can set these variables from the command line, and also
-# from the environment for the first two.
-SPHINXOPTS ?=
-SPHINXBUILD ?= sphinx-build
-SOURCEDIR = .
-BUILDDIR = _build
-
-# Put it first so that "make" without argument is like "make help".
-help:
- @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
-
-.PHONY: help Makefile
-
-# Catch-all target: route all unknown targets to Sphinx using the new
-# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
-%: Makefile
- @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/docs/conf.py b/docs/conf.py
deleted file mode 100644
index ee7421a..0000000
--- a/docs/conf.py
+++ /dev/null
@@ -1,28 +0,0 @@
-# Configuration file for the Sphinx documentation builder.
-#
-# For the full list of built-in configuration values, see the documentation:
-# https://www.sphinx-doc.org/en/master/usage/configuration.html
-
-# -- Project information -----------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
-
-project = 'olang'
-copyright = '2024, olang maintainers'
-author = 'olang maintainers'
-release = '0.0.1'
-
-# -- General configuration ---------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
-
-extensions = []
-
-templates_path = ['_templates']
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
-
-
-
-# -- Options for HTML output -------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
-
-html_theme = 'alabaster'
-html_static_path = ['_static']
diff --git a/docs/index.rst b/docs/index.rst
deleted file mode 100644
index 4ad0971..0000000
--- a/docs/index.rst
+++ /dev/null
@@ -1,13 +0,0 @@
-Welcome to olang's documentation!
-=================================
-
-.. toctree::
- :caption: Contents:
-
-
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`search`
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH olang] docs: add HACKING documentation
@ 2024-02-17 4:23 Carlos Maniero
2024-02-17 4:23 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-02-17 4:23 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
The purpose of this docs is to teach newcomers developers to start
contributing with the project.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
docs/index.rst | 8 ++-
docs/pages/hacking.rst | 158 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 164 insertions(+), 2 deletions(-)
create mode 100644 docs/pages/hacking.rst
diff --git a/docs/index.rst b/docs/index.rst
index 4ad0971..f09a97d 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,9 +1,13 @@
-Welcome to olang's documentation!
-=================================
+Welcome to olang's docs!
+========================
+The zero programming language.
+
.. toctree::
:caption: Contents:
+ pages/hacking
+
Indices and tables
diff --git a/docs/pages/hacking.rst b/docs/pages/hacking.rst
new file mode 100644
index 0000000..5fd7a39
--- /dev/null
+++ b/docs/pages/hacking.rst
@@ -0,0 +1,158 @@
+=======
+Hacking
+=======
+
+We’re thrilled to have you here! Your interest in making olang the most
+exceptional and straightforward language ever is greatly appreciated.
+
+In this document, we’ll outline how you can begin contributing to olang.
+
+First and foremost, clone the project if you haven’t done so already.
+
+.. code-block:: sh
+
+ git clone https://git.sr.ht/~johnnyrichard/olang
+
+Dependencies
+============
+
+The olang compiler is crafted in C. To build the project, you’ll require
+merely three dependencies: **make**, **gcc** (version 11 or higher), and
+**clang-format** (version 14 or higher).
+
+As an optional step, you can install **sphinx** to refresh this documentation.
+
+Code style
+==========
+
+Instead of delineating every element of our coding style, we have adopted the
+use of **clang-format** to enforce the olang code style. Please refer to the
+linter section below for guidance on its application.
+
+Linter
+------
+
+Checking for linter issues:
+
+.. code-block:: sh
+
+ make linter
+
+Most of the common code style mistakes are fixed by:
+
+.. code-block:: sh
+
+ make linter-fix
+
+.editorconfig
+-------------
+
+We also provide a **.editorconfig** file at project's root. Follow
+https://editorconfig.org/ to learn how to make it work with your favorite
+editor.
+
+Testing
+=======
+
+There are two layers of tests **integration** and **unit**. The integration
+test is going to execute the **0c** compiler and check if the generated binary
+acts as expected. Unit tests will test C functions.
+
+For both unit and integration we use **munit** framework: https://nemequ.github.io/munit/.
+
+To execute tests you can execute:
+
+.. code-block:: sh
+
+ make check
+
+
+Submitting a patch
+==================
+
+Before submit a patch, ensure your code follows our coding style and is
+well-tested. After that, you're good to follow the steps bellow.
+
+Step 1: Commit your changes
+---------------------------
+
+Begin by committing your modifications with a detailed and significant commit
+message. We take great pleasure in maintaining a record of our changes over
+time. Skeptical? Execute a **git log** command and admire the well-documented
+history we’ve created so far.
+
+But it isn't all about personal preference. We use a email-driven workflow
+to propose our changes, meaning the commit message you write is the email the
+olang maintainers will get. I won’t go into the perks of the email-driven
+workflow here, but you can check it out at
+https://drewdevault.com/2018/07/02/Email-driven-git.html.
+
+Best practices
+^^^^^^^^^^^^^^
+
+#. Write single-purpose commits.
+#. Write a meaningful commit message.
+#. Every commit must be production ready.
+ - If the tests or the linter fail, you should not create a fix commit.
+ Instead, you should amend the commit that caused the issue and then resend
+ the patchset.
+
+Step 2: Create your patch
+-------------------------
+
+You can create a patch using the command:
+
+.. code-block:: sh
+
+ git format-patch --cover-letter -M origin/main -o outgoing/
+
+Step 3: Write a cover letter:
+-----------------------------
+
+The command above generates a **outgoing/0000-cover-letter.patch** file.
+
+ The cover letter is like a pull request description on Github. Replace the
+ \*\*\* SUBJECT HERE \*\*\* with the patchset title and the
+ \*\*\* BLURB HERE \*\*\* with a synopsis of your changes.
+
+If you are sending a single-commit patch you can remove the **--cover-lette**
+argument from **git format-patch** and skip this step.
+
+Step 4: Submit your patch
+-------------------------
+
+Make sure you have configured your **git send-email**. You can learn how to
+configure it here:
+https://git-scm.com/docs/git-send-email#_examples
+
+Once you have everything set you just need to send the patch over our
+mailing list.
+
+.. code-block:: sh
+
+ git send-email outgoing/* --to=~johnnyrichard/olang-devel@lists.sr.ht
+
+The review process
+------------------
+
+Upon submission, you’ll receive an automated email from our pipeline. If the
+check is successful, the olang maintainers will review your patch.
+Subsequently, you’ll receive an email indicating whether your patch has been
+approved, requires changes, or has been rejected.
+
+Submitting changes in a patchset
+--------------------------------
+
+If your patchset requires any modifications, you’ll have to submit a new
+version of your patch. The submission process remains unchanged, except for the
+addition of the version argument to the **git format-patch** command.
+
+.. code-block:: sh
+
+ git format-patch --cover-letter -M origin/main -o outgoing/ -v2
+
+After send a new email with **git send-email**.
+
+----
+
+Thanks again and happy hacking!
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang v2 2/2] tests: add integration test setup
@ 2024-02-16 2:58 Carlos Maniero
2024-02-16 3:03 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-02-16 2:58 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
This is a basic setup for integration tests which includes:
- a *cli_runner* helper file to provide an interface to interact with
the CLI.
- a simple test that validates if the compiler is returning zero exit
code when a file is provided.
At this point the compiler still doing nothing making all this bootstrap
just a fancy way to check if the compiler was compiled properly.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
.build.yml | 4 ++
Makefile | 12 ++++++
tests/integration/Makefile | 27 ++++++++++++
tests/integration/cli_runner.c | 77 ++++++++++++++++++++++++++++++++++
tests/integration/cli_runner.h | 27 ++++++++++++
tests/integration/cli_test.c | 39 +++++++++++++++++
6 files changed, 186 insertions(+)
create mode 100644 tests/integration/Makefile
create mode 100644 tests/integration/cli_runner.c
create mode 100644 tests/integration/cli_runner.h
create mode 100644 tests/integration/cli_test.c
diff --git a/.build.yml b/.build.yml
index 3aebfcf..b38efb3 100644
--- a/.build.yml
+++ b/.build.yml
@@ -12,3 +12,7 @@ tasks:
- build: |
cd olang
make
+ - check: |
+ cd olang
+ make check
+
diff --git a/Makefile b/Makefile
index 2a23b59..b13b41b 100644
--- a/Makefile
+++ b/Makefile
@@ -19,10 +19,22 @@ $(BUILD_DIR):
.PHONY: linter
linter: $(SRCS) $(HEADERS)
clang-format --dry-run --Werror $?
+ $(MAKE) -C tests/integration/ linter
+
.PHONY: linter-fix
linter-fix: $(SRCS) $(HEADERS)
clang-format -i $?
+ $(MAKE) -C tests/integration/ linter-fix
+
+.PHONY: integration-test
+integration-test:
+ $(MAKE)
+ $(MAKE) -C tests/integration/
+
+.PHONY: check
+check:
+ $(MAKE) integration-test
$(BUILD_DIR)/%.o: $(SRC_DIR)/%.c
$(CC) $(CFLAGS) -c $< -o $@
diff --git a/tests/integration/Makefile b/tests/integration/Makefile
new file mode 100644
index 0000000..a42f787
--- /dev/null
+++ b/tests/integration/Makefile
@@ -0,0 +1,27 @@
+SRCS := $(wildcard *_test.c)
+TO_LINT := $(filter-out munit.c munit.h,$(wildcard *.c *.h))
+OBJS := $(patsubst %_test.c, %_test.o, $(SRCS))
+CFLAGS := -I../../src
+TESTS := $(patsubst %_test.c, %_test, $(SRCS))
+EXEC_TESTS := $(patsubst %_test, ./%_test, $(TESTS))
+
+.PHONY: all
+all: munit.o cli_runner.o $(TESTS)
+ @for file in $(EXEC_TESTS); do \
+ ./"$$file"; \
+ done
+
+.PHONY: clean
+clean:
+ $(RM) *.o *_test
+
+.PHONY: linter
+linter: $(TO_LINT)
+ clang-format --dry-run --Werror $?
+
+.PHONY: linter-fix
+linter-fix: $(TO_LINT)
+ clang-format -i $?
+
+cli_test: munit.o cli_runner.o cli_test.o
+ $(CC) $? $(CFLAGS) -o $@
diff --git a/tests/integration/cli_runner.c b/tests/integration/cli_runner.c
new file mode 100644
index 0000000..0ca0b37
--- /dev/null
+++ b/tests/integration/cli_runner.c
@@ -0,0 +1,77 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#include "cli_runner.h"
+#include <assert.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#define OLANG_COMPILER_PATH "../../0c"
+
+static int compiler_exists_already_checked = 0;
+
+void
+static assert_compiler_exists()
+{
+ {
+ if (compiler_exists_already_checked == 1) {
+ return;
+ }
+
+ compiler_exists_already_checked = 1;
+ }
+
+ FILE *file = fopen(OLANG_COMPILER_PATH, "r");
+
+ if (file != NULL) {
+ fclose(file);
+ return;
+ }
+
+ perror("Build the compiler before executing tests");
+ exit(1);
+}
+
+void
+create_tmp_file_name(char *file_name)
+{
+ sprintf(file_name, "%s/olang_programXXXXXX", P_tmpdir);
+ int fd = mkstemp(file_name);
+
+ if (fd == -1) {
+ perror("Could not create a tmp file. Check your P_tmpdir permission.");
+ exit(1);
+ }
+ close(fd);
+}
+
+cli_result_t
+cli_runner_compile_file(char *src)
+{
+ assert_compiler_exists();
+
+ cli_result_t result;
+ create_tmp_file_name(result.program_path);
+
+ char command[1024];
+ sprintf(command, "%s -o %s %s", OLANG_COMPILER_PATH, result.program_path, src);
+
+ result.exit_code = system(command);
+ return result;
+}
diff --git a/tests/integration/cli_runner.h b/tests/integration/cli_runner.h
new file mode 100644
index 0000000..5caa319
--- /dev/null
+++ b/tests/integration/cli_runner.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#ifndef CLI_RUNNER_H
+#define CLI_RUNNER_H
+typedef struct cli_result_t
+{
+ int exit_code;
+ char program_path[255];
+} cli_result_t;
+
+cli_result_t
+cli_runner_compile_file(char *src);
+#endif
diff --git a/tests/integration/cli_test.c b/tests/integration/cli_test.c
new file mode 100644
index 0000000..c7a9557
--- /dev/null
+++ b/tests/integration/cli_test.c
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2024 olang maintainers
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#define MUNIT_ENABLE_ASSERT_ALIASES
+#include "cli_runner.h"
+#include "munit.h"
+
+static MunitResult
+test_cli_hello_file(const MunitParameter params[], void *user_data_or_fixture)
+{
+ cli_result_t compilation_result = cli_runner_compile_file("../../examples/hello.olang");
+ munit_assert_int(compilation_result.exit_code, ==, 0);
+ return MUNIT_OK;
+}
+
+static MunitTest tests[] = { { "/test_cli_hello_file", test_cli_hello_file, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL },
+ { NULL, NULL, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL } };
+
+static const MunitSuite suite = { "/cli_test", tests, NULL, 1, MUNIT_SUITE_OPTION_NONE };
+
+int
+main(int argc, char *argv[])
+{
+ return munit_suite_main(&suite, NULL, argc, argv);
+ return EXIT_SUCCESS;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH olang] style: fix clang-format format indentation
@ 2024-02-14 23:36 Carlos Maniero
2024-02-14 23:41 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Carlos Maniero @ 2024-02-14 23:36 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Carlos Maniero
editorconfig was set to 4 spaces clang-format was expecting 2.
Signed-off-by: Carlos Maniero <carlos@maniero.me>
---
.clang-format | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/.clang-format b/.clang-format
index 2850fff..b76afee 100644
--- a/.clang-format
+++ b/.clang-format
@@ -81,8 +81,8 @@ ColumnLimit: 120
CommentPragmas: '^ IWYU pragma:'
QualifierAlignment: Leave
CompactNamespaces: false
-ConstructorInitializerIndentWidth: 2
-ContinuationIndentWidth: 2
+ConstructorInitializerIndentWidth: 4
+ContinuationIndentWidth: 4
Cpp11BracedListStyle: false
DeriveLineEnding: true
DerivePointerAlignment: false
@@ -124,7 +124,7 @@ IndentGotoLabels: true
IndentPPDirectives: None
IndentExternBlock: AfterExternBlock
IndentRequiresClause: true
-IndentWidth: 2
+IndentWidth: 4
IndentWrappedFunctionNames: false
InsertBraces: false
InsertTrailingCommas: None
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH olang] build: enable continuous integration through .build.yml
@ 2024-02-13 21:36 Johnny Richard
2024-02-13 20:34 ` [olang/patches/.build.yml] build failed builds.sr.ht
0 siblings, 1 reply; 35+ messages in thread
From: Johnny Richard @ 2024-02-13 21:36 UTC (permalink / raw)
To: ~johnnyrichard/olang-devel; +Cc: Johnny Richard
A small build pipeline with linter and compilation settings.
Signed-off-by: Johnny Richard <johnny@johnnyrichard.com>
---
.build.yml | 12 ++++++++++++
1 file changed, 12 insertions(+)
create mode 100644 .build.yml
diff --git a/.build.yml b/.build.yml
new file mode 100644
index 0000000..35a5da8
--- /dev/null
+++ b/.build.yml
@@ -0,0 +1,12 @@
+image: archlinux
+packages:
+ - gcc
+ - make
+ - clang
+tasks:
+ - lint: |
+ cd olang
+ make linter
+ - build: |
+ cd olang
+ make
--
2.42.0
^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2024-10-17 2:53 UTC | newest]
Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-10-15 12:14 [PATCH olang] fix: codegen: prevent stack overwrite Carlos Maniero
2024-10-15 12:14 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-10-15 23:03 ` Carlos Maniero
2024-10-16 22:33 ` [PATCH olang] fix: codegen: prevent stack overwrite Johnny Richard
-- strict thread matches above, loose matches on Subject: below --
2024-10-17 2:48 [PATCH olang v1 1/1] codegen: x64: deref returns pointer value Carlos Maniero
2024-10-17 2:49 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-10-17 2:52 ` Carlos Maniero
2024-10-11 3:42 [PATCH olang v1] fix: build: add missing dependencies for check-spec Johnny Richard
2024-10-11 1:43 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-10-08 16:33 [PATCH olang] parser: conform block line feeds with the spec Carlos Maniero
2024-10-08 16:33 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-08-13 18:16 [PATCH olang v1 2/2] ast: inline ast_node_data_t union definition Johnny Richard
2024-08-13 17:27 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-08-13 19:03 ` Johnny Richard
2024-04-20 13:54 [PATCH olang v1] build: rename linter to format to avoid confusion Johnny Richard
2024-04-20 12:57 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-28 15:58 [PATCH olang v1] fe: lexer: add single line comments support Johnny Richard
2024-03-28 14:59 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-28 16:46 ` Johnny Richard
2024-03-27 3:21 [PATCH olang v1 2/2] docs: spec: add variables and constants specification Carlos Maniero
2024-03-27 3:22 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-25 22:36 [PATCH olang v1 2/2] cli: remove code duplication Johnny Richard
2024-03-25 21:37 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-26 2:32 ` Carlos Maniero
2024-03-26 2:35 ` Carlos Maniero
2024-03-13 12:32 [PATCH olang v3] refactor: rename zero programming language to olang Fabio Maciel
2024-03-13 12:33 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-08 20:52 [PATCH olang] parser: abort when parser identifies a syntax error Johnny Richard
2024-03-08 19:54 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-05 8:44 [PATCH olang v2 3/3] cli: add compilation -o option with --save-temps Johnny Richard
2024-03-05 7:51 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-04 19:23 [PATCH olang v1 3/3] cli: add compilation -o option with --save-temps Johnny Richard
2024-03-04 18:33 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-04 19:39 ` Johnny Richard
2024-03-05 2:05 ` Carlos Maniero
2024-03-02 20:01 [PATCH olang] string_view: fix stack buffer overflow on to_u32 Johnny Richard
2024-03-02 19:02 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-03-02 19:02 [PATCH olang] string_view: add n + 1 test to string_view_to_u32 function Johnny Richard
2024-03-02 18:03 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-20 16:39 [PATCH olang v2] utils: add arena Carlos Maniero
2024-02-20 16:45 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-20 16:10 [PATCH olang] utils: add arena Carlos Maniero
2024-02-20 16:15 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-19 21:04 [PATCH olang v4 4/4] lexer: test: add integration tests for --dump-tokens Johnny Richard
2024-02-19 20:07 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-19 1:23 [PATCH olang v2 2/2] lexer: create --dump-tokens cli command Johnny Richard
2024-02-19 0:27 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-19 1:15 [PATCH olang 2/2] lexer: create --dump-tokens cli command Johnny Richard
2024-02-19 0:20 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-17 13:46 [PATCH olang 2/2] Revert "docs: add sphinx documentation support" Carlos Maniero
2024-02-17 13:51 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-17 4:23 [PATCH olang] docs: add HACKING documentation Carlos Maniero
2024-02-17 4:23 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-16 2:58 [PATCH olang v2 2/2] tests: add integration test setup Carlos Maniero
2024-02-16 3:03 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-14 23:36 [PATCH olang] style: fix clang-format format indentation Carlos Maniero
2024-02-14 23:41 ` [olang/patches/.build.yml] build failed builds.sr.ht
2024-02-13 21:36 [PATCH olang] build: enable continuous integration through .build.yml Johnny Richard
2024-02-13 20:34 ` [olang/patches/.build.yml] build failed builds.sr.ht
Code repositories for project(s) associated with this public inbox
https://git.johnnyrichard.com/olang.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox