From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:700:3204::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id iBJbHo0M8mWRSgAAbAwnHQ (envelope-from ) for ; Wed, 13 Mar 2024 21:29:01 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id EKiNGY0M8mU+rgAAe85BDQ (envelope-from ) for ; Wed, 13 Mar 2024 21:29:01 +0100 X-Envelope-To: patches@johnnyrichard.com Authentication-Results: aspmx1.migadu.com; none Received: from mail-a.sr.ht (mail-a.sr.ht [46.23.81.152]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 56CE720523 for ; Wed, 13 Mar 2024 21:29:01 +0100 (CET) DKIM-Signature: a=rsa-sha256; bh=hSopmtSnxtJ5CWxILZw7u631TTJBFU6H27KWdbIh1zE=; c=simple/simple; d=lists.sr.ht; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-Unsubscribe:List-Subscribe:List-Archive:List-Post:List-ID; q=dns/txt; s=20240113; t=1710361739; v=1; b=c1k0+FzIrCCKDbWqjQH+WRbAcEA7F3wA15m8jKnie7g0O8KGrcNozg3tshhnGesy77zMaImw O1v0IABMz2aN4LCzCSLV1iT+LmOxE7y/f0f/Sr/CEHwkNVfEYwZzSsutxDeGJzs0pWGZouqS9Uh Yr5Uk0SIMbp+LmwWo9IzEmlEuQAVLAneYbf9hI3L0ddM7fhrDbaKNX1P8lhpCPg9NbLVQz8UhPR niVmKiwsJIYmS0B022DuLJX9MhXCIRVnRdXtvbDk7dnHvpDf9hC49jI6sEpx3tMmzcSo0+7tuvq TJLKYIk06yt7Fi8VTXacqbTwWpiZW03yw/vQhxfnB/YlA== Received: from lists.sr.ht (unknown [46.23.81.154]) by mail-a.sr.ht (Postfix) with ESMTPSA id 712E020289 for ; Wed, 13 Mar 2024 20:28:59 +0000 (UTC) Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [IPv6:2001:41d0:1004:224b::b3]) by mail-a.sr.ht (Postfix) with ESMTPS id 922CE200E7 for <~johnnyrichard/olang-devel@lists.sr.ht>; Wed, 13 Mar 2024 20:28:58 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=johnnyrichard.com; s=key1; t=1710361737; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d5in9M3y9hl8wvWggVvuc3OhbO1TNI62N2qsz40gT+o=; b=c580CP6zcF/QxHHLUY3YjB4Gdwi17Q75FcPeCRjp6MUAx+ABNPaGDqkyMCXyyx6VbmwzUU d36DCefQGunTznCpwEyU6fNYVirv/qwJDZzDfrMq6uIzm1s93IF3WKA5j0pb8Jjbch67EW 4wxyq3kqQXmU1GuxMJhl4F2whRDRMw4gEH7NRgmKM/3BjJ8UVqRYp98+o1RX4gbkwKS2w2 5MHrerz+nXhVamtAMgll9TqWihKUipyPTZFrIeSQlf1fsgDhwM6rqir3YP56fNECBDrht0 AfPS0LjoSINijoVfSGZAn+TiYhSlShtOqHZcBSNaEdnNzb6L1m+SVJQZgjTpzg== From: Johnny Richard To: ~johnnyrichard/olang-devel@lists.sr.ht Cc: Johnny Richard Subject: [PATCH olang v1 1/3] lexer: add tokenize support for '+' '/' '*' '-' Date: Wed, 13 Mar 2024 22:21:50 +0100 Message-ID: <20240313212855.174554-2-johnny@johnnyrichard.com> In-Reply-To: <20240313212855.174554-1-johnny@johnnyrichard.com> References: <20240313212855.174554-1-johnny@johnnyrichard.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Sourcehut-Patchset-Status: UNKNOWN List-Unsubscribe: List-Subscribe: List-Archive: Archived-At: List-Post: List-ID: ~johnnyrichard/olang-devel <~johnnyrichard/olang-devel.lists.sr.ht> Sender: ~johnnyrichard/olang-devel <~johnnyrichard/olang-devel@lists.sr.ht> X-Migadu-Flow: FLOW_IN X-Migadu-Country: NL X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -4.00 X-Spam-Score: -4.00 X-Migadu-Queue-Id: 56CE720523 X-TUID: hBTzbnmaOBiU Signed-off-by: Johnny Richard --- examples/expression.ol | 3 ++ src/lexer.c | 24 ++++++++++++++- src/lexer.h | 4 +++ tests/integration/cli_test.c | 56 +++++++++++++++++++++++++++++++++-- tests/integration/proc_exec.h | 3 +- 5 files changed, 86 insertions(+), 4 deletions(-) create mode 100644 examples/expression.ol diff --git a/examples/expression.ol b/examples/expression.ol new file mode 100644 index 0000000..efa4ab5 --- /dev/null +++ b/examples/expression.ol @@ -0,0 +1,3 @@ +fn main(): u32 { + return (10 + 1 * 2) - (10 - (1 + 1) / 2) +} diff --git a/src/lexer.c b/src/lexer.c index dd6f11d..45a3d63 100644 --- a/src/lexer.c +++ b/src/lexer.c @@ -126,6 +126,26 @@ lexer_next_token(lexer_t *lexer, token_t *token) lexer_skip_char(lexer); return; } + case '+': { + lexer_init_char_value_token(lexer, token, TOKEN_PLUS); + lexer_skip_char(lexer); + return; + } + case '-': { + lexer_init_char_value_token(lexer, token, TOKEN_DASH); + lexer_skip_char(lexer); + return; + } + case '*': { + lexer_init_char_value_token(lexer, token, TOKEN_STAR); + lexer_skip_char(lexer); + return; + } + case '/': { + lexer_init_char_value_token(lexer, token, TOKEN_SLASH); + lexer_skip_char(lexer); + return; + } case '\n': { lexer_init_char_value_token(lexer, token, TOKEN_LF); lexer_skip_char(lexer); @@ -151,7 +171,9 @@ static char *token_kind_str_table[] = { [TOKEN_RETURN] = "return", [TOKEN_LF] = "line_feed", [TOKEN_OPAREN] = "(", [TOKEN_CPAREN] = ")", [TOKEN_COLON] = ":", [TOKEN_OCURLY] = "{", - [TOKEN_CCURLY] = "}", [TOKEN_EOF] = "EOF", + [TOKEN_CCURLY] = "}", [TOKEN_PLUS] = "+", + [TOKEN_DASH] = "-", [TOKEN_STAR] = "*", + [TOKEN_SLASH] = "/", [TOKEN_EOF] = "EOF", }; char * diff --git a/src/lexer.h b/src/lexer.h index cb91d7e..5ab9ccc 100644 --- a/src/lexer.h +++ b/src/lexer.h @@ -40,6 +40,10 @@ typedef enum token_kind TOKEN_RETURN, // Single char + TOKEN_PLUS, + TOKEN_DASH, + TOKEN_SLASH, + TOKEN_STAR, TOKEN_LF, TOKEN_OPAREN, TOKEN_CPAREN, diff --git a/tests/integration/cli_test.c b/tests/integration/cli_test.c index 8cc22f9..8481ba3 100644 --- a/tests/integration/cli_test.c +++ b/tests/integration/cli_test.c @@ -20,7 +20,7 @@ #include static MunitResult -test_cli_dump_tokens(const MunitParameter params[], void *user_data_or_fixture) +test_cli_dump_tokens_exmaple_main_exit(const MunitParameter params[], void *user_data_or_fixture) { cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/main_exit.ol"); munit_assert_int(compilation_result.exec.exit_code, ==, 0); @@ -42,6 +42,47 @@ test_cli_dump_tokens(const MunitParameter params[], void *user_data_or_fixture) return MUNIT_OK; } +static MunitResult +test_cli_dump_tokens_exmaple_expression(const MunitParameter params[], void *user_data_or_fixture) +{ + cli_result_t compilation_result = cli_runner_compiler_dump_tokens("../../examples/expression.ol"); + munit_assert_int(compilation_result.exec.exit_code, ==, 0); + munit_assert_string_equal(compilation_result.exec.stdout_buf, + "../../examples/expression.ol:1:1: \n" + "../../examples/expression.ol:1:4: \n" + "../../examples/expression.ol:1:8: <(>\n" + "../../examples/expression.ol:1:9: <)>\n" + "../../examples/expression.ol:1:10: <:>\n" + "../../examples/expression.ol:1:12: \n" + "../../examples/expression.ol:1:16: <{>\n" + "../../examples/expression.ol:1:17: \n" + "../../examples/expression.ol:2:3: \n" + "../../examples/expression.ol:2:10: <(>\n" + "../../examples/expression.ol:2:11: \n" + "../../examples/expression.ol:2:14: <+>\n" + "../../examples/expression.ol:2:16: \n" + "../../examples/expression.ol:2:18: <*>\n" + "../../examples/expression.ol:2:20: \n" + "../../examples/expression.ol:2:21: <)>\n" + "../../examples/expression.ol:2:23: <->\n" + "../../examples/expression.ol:2:25: <(>\n" + "../../examples/expression.ol:2:26: \n" + "../../examples/expression.ol:2:29: <->\n" + "../../examples/expression.ol:2:31: <(>\n" + "../../examples/expression.ol:2:32: \n" + "../../examples/expression.ol:2:34: <+>\n" + "../../examples/expression.ol:2:36: \n" + "../../examples/expression.ol:2:37: <)>\n" + "../../examples/expression.ol:2:39: \n" + "../../examples/expression.ol:2:41: \n" + "../../examples/expression.ol:2:42: <)>\n" + "../../examples/expression.ol:2:43: \n" + "../../examples/expression.ol:3:1: <}>\n" + "../../examples/expression.ol:3:2: \n" + "../../examples/expression.ol:4:1: \n"); + return MUNIT_OK; +} + static MunitResult test_cli_compile_minimal_program(const MunitParameter params[], void *user_data_or_fixture) { @@ -62,7 +103,18 @@ test_cli_compile_minimal_program(const MunitParameter params[], void *user_data_ } static MunitTest tests[] = { - { "/test_cli_dump_tokens", test_cli_dump_tokens, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL }, + { "/test_cli_dump_tokens_exmaple_main_exit", + test_cli_dump_tokens_exmaple_main_exit, + NULL, + NULL, + MUNIT_TEST_OPTION_NONE, + NULL }, + { "/test_cli_dump_tokens_exmaple_expression", + test_cli_dump_tokens_exmaple_expression, + NULL, + NULL, + MUNIT_TEST_OPTION_NONE, + NULL }, { "/test_cli_compile_minimal_program", test_cli_compile_minimal_program, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL }, { NULL, NULL, NULL, NULL, MUNIT_TEST_OPTION_NONE, NULL } }; diff --git a/tests/integration/proc_exec.h b/tests/integration/proc_exec.h index 135aa6a..45c2977 100644 --- a/tests/integration/proc_exec.h +++ b/tests/integration/proc_exec.h @@ -21,7 +21,8 @@ typedef struct proc_exec_result { int exit_code; - char stdout_buf[1024]; + // FIXME: output buffer shouldn't be fixed size + char stdout_buf[2048]; } proc_exec_result_t; typedef struct proc_exec_command -- 2.44.0