Re: [RFC] Namespaces in OLANG

[Johnny Richard <johnny@johnnyrichard.com>]

Thanks for the reply. 

I understand you want to build a very complex build system and module
system because you are frustrated on how C tackle these problems.  To
be honest I prefer to keep these problems stupid simple as C does.  This
would also reduce the impact of integrating with C (I mean, almost no
effort).

But if you really want to define right now how we organize the files no
problem, pick the one you think is the best, I have no knowledge to
contribute here.  But I can share my thoughts on this anyway.

On Thu, Mar 28, 2024 at 10:41:20AM -0300, Carlos Maniero wrote:
> > > 2. Full Compatibility with C:
> > > 
> > >    It is completely compatible with C!
> > > 
> > >      int olang_core_math__add(int, int);
> > > 
> > >    If you think it is ugly to call a function that way directly, you can create
> > >    macros in C to improve the readability, but completely optional.
> > > 
> > >      #define NSMATH(name) olang_core_math__##name
> >
> > I think this is too ugly and very hack.  I would prefer to call
> > olang_core_math_add instead.
> 
> You quoted the entire text (which I truncated), do you mean the macro is
> hacky? Or everything? The macro is just a suggestion.

Yeah, I was talking about the macro suggestion.

> You mentioned that you would prefer to use *olang_core_math_add*. Did you
> mean *olang_core_math__add*, or are you opposed to the use of double
> underscores to separate the namespace from the identifier?

Not actually.  I think it's okay if we keep double underscore to
separate namespaces from function. 

> > > 3. Manual namespacing is inconvenient: 
> > > 
> > >    You don't need to manually namespace every function with the cost of start
> > >    every single file with a *ns* statement.
> >
> > If we keep managing names manually, we already have the *1* and *2* for
> > free.  So, the only benefit of namespacing would be to avoid the
> > inconvenience of adding it manually.
> 
> That's partially correct. I mentioned points 1 and 2 because most modern
> system languages, such as C++ and Rust, mangle names to avoid conflicts.
> However, I made a mistake by not including this solution in the
> Alternatives section.

Sure, no problems.
 
> > > An important observation of the *ns* usage is that it must match the directory
> > > structure. The path of a file that declares the namespace *olang.core.math*
> > > must ends with *olang/core/math.ol*. This requirement is need for future
> > > import resolution.
> > > 
> > > Alternatives:
> > > -------------
> > > 
> > > 1. Automatically create namespaces based on the filename:
> >
> > I know we don't have written down nicely the goal of the language, but I
> > prefer being explicit and avoid convention over configuration.
> 
> Agree! That's why namespaced files are great \o/

Not actually.  I mean, adding suffix to the binary symbol automatically
based on directory structure is kind of convention over configuration.

> > > 2. Manual namespaces: ...
> > > 
> > > Conclusion
> > > ----------
> > > 
> > > In my opinion, the introduction of a namespace statement offers numerous
> > > benefits:
> > > 
> > > - It aids in resolving function name conflicts.
> > > - It facilitates deterministic code generation while maintaining compatibility
> > >   with C.
> >
> > The current suggestion doesn't solve the all compatibility with C.  We
> > have to provide a way of calling a C function from olang code without
> > namespacing (in case of namespace being mandatory).
> 
> Good catch! In my opinion, we should follow C's approach on this matter.
> 
>   extern fn pow(base: u32, power: u32)

Sure, I am okay with this one.

> Do you think that namespaces translation in between C and olang are
> necessary? In our arena implementation, all functions have the *arena_*
> prefix. By using *extern* the way I'm proposing we will call these
> functions in olang with their exactly name, ie, *arena_alloc* will be
> called using *arena_alloc* not just *alloc*. IMO, it is ok since it is
> an external.

I prefer to follow exactly the same extern name in this case.

> > > - It simplifies the resolution of imports.
> >
> > I would suggest to not go much further with import resolution (unless
> > you already want to define modules).  Perhaps we could have namespace
> > doing nothing else than namespacing...
> 
> If by "modules" you are referring to the file level, and not to
> something like packages or libraries, then that is exactly what I want
> to define! Influenced by Clojure, I recommended calling it a
> "namespace". However, I believe that naming it ‘mod' or ‘module' is more
> suitable for its purpose.
> 
>   mod olang.core.math
> 
>   fn add(a: u32, b: u32) {
>     return a + b
>   }

Not sure if I understand what you meant here. Maybe would be better
clarify what you understands as module.  This concept is still blurry to me.

> > > These advantages come with the minor stipulation of initiating all files with a
> > > namespace statement, which, in my view, is a small price to pay for the
> > > benefits gained.
> >
> > I'm not keen on the idea of enforcing strict adherence to the folder
> > structure.
> >
> > How about we introduce a namespace block instead? Within this
> > block, everything would automatically have the namespace added as a
> > prefix. This could offer more flexibility while still maintaining
> > organization.
> 
> Don't you think that in practice almost every single file will
> namespace? C++ follows this pattern, and look at this Qt mirror [1], 6k
> files, all namespaced, they even created a macro to facilitate the
> work.
> 
> [1] https://github.com/search?q=repo%3Aradekp%2Fqt+%2FQT_BEGIN_NAMESPACE%5Cn%2F&type=code&p=1

I prefer being explicit in this case (even if we need to write it for
every file).  If you land on a random file, you can say for what the
binary the code will translate to without checking the folder structure.
(Dir tree organization is completely up to the developers). 

> > I think module has a different meaning.  If you want to have modules,
> > for sure we have to discuss import resolution.  IMHO namespace shouldn't
> > do anything else than namespacing.
> 
> I believe you're right. It's almost impossible to discuss modules
> without bringing up imports. To me, the way C handles this is one of the
> most painful things in my life (hehe).
> 
> The main issue is that I never know where something is coming from,
> which is especially painful when I'm trying to replicate something I've
> already done. This also, often leads to unused includes over time
> because it's hard to determine if an include is actually being used.

Could you please enlighten me the unused definitions problem?  Does it
have any implications with performance?

If this is so bad, why we cannot make the compiler smart enough to
detect them?

> If we abandon modules and just go with C++-like namespaces, I believe we
> we may want to endure C's painful include system. This is because the
> language won't have control over function names. The way the include
> system is designed sends a message to developers that including a file
> is akin to concatenating all the definitions into a single file. But
> yet I think it would be ok to have names imports even if we don't
> control the language names but it would be just a semantic tool.
> 
> Named Imports
> -------------
> 
>   mod myprog
> 
>   import olang.core.math
> 
>   fn main(): u32 {
>     return olang.core.math::sum(1, 2)
>   }
> 
> And even associate identifiers to it.
> 
>   mod myprog
> 
>   import olang.core.math as ocm
> 
>   fn main(): u32 {
>     return ocm::sum(1, 2)
>   }
> 
> Note that there is no actually difference in between mangling and my
> module purpose, except the fact modules generates deterministic and
> friendly names that can be easily used in C and also easy to gen code,
> once to generate the assembly symbol of *olang.core.math::sum* we can
> just replace dots by underscores and double column to double
> underscores.
> 
> External Linkage
> ----------------
> 
> We probably don't wanna to make all function global for external
> linkage.

Could you please clarify what you mean with external linkage?

> So we may need a visibility keyword.

The C already has this functionality, the *static* keyword...
Everything else is "public" by default.  Do you see a problem on
following the same pattern?

> And to me, everything that can be imported should also be available
> for external linkage, even if we decide to do not generate an object
> file per module. I would recommend the usage of *export* or *pub*. I
> like *export* better.

Why this has to be so complex?  Seems like we would have to implement a
very complex system for compile a program with multiple files.  I would
love to have something very simple where you compile down to a `.o` file
and the linking is done simple as GAS or GCC does.

> Note that this is required no meter how we decide to handle imports.
> 
> No mangling
> -----------
> 
> But what if I really need something to have the exact name? Lets say you
> wanna integrate with a bootloader that is integrated on the link process and
> expects a symbol called *kmain* to jump into?
> 
> Ok, I admit, in that case namespaces it is gonna be a pain in the ass. But
> the good news is that these are exceptional, you don't need this for
> your entire application but only for a few functions.
> 
> I was wondering that we could have a *global* keyword where everything
> that is global assumes its own name and is always have public visibility.

You are assuming that all files has a mandatory namespace.  I still
prefer the namespace being optional.  The default behavior would be
every symbol wont have prefix. (1 to 1) with ELF binary symbols.

IMHO this is what makes the language simple as C.  Adding extra
functionality to protect the programming doing stuff and enabling
functionality with a lot of gymnastics feels like an unpleasant
experience.

> Summarizing
> -----------
> 
> We have a few options in the table.
> 
> 1. Use the names the way they are. (C approach)
> 
> Pros:
> - Simple, no magic, it is what it is.
> - Easy to produce debug info since the assembly symbol will be the
>   function name.

- No build gymnastics as well. The *make* program would solve our
  problems for free (Even with partial compilation).

> 
> Cons:
> - More challenge to keep the code out of name conflicts in large
>   codebases.
> - Requires developers to manually namespace functions.
> 
> 2. Use mangled names (C++, Rust approach)

The mangled names are basically to enable function overload.  I think we
don't want this functionality on the language.  I think we can ignore
this one.

> 3. Use modules (Zig approach (I think))
> 
> Pros:
> - Keep the code out of naming conflicts
> - Deterministic assembly symbols permit integrate with C without any
>   magic, you just need to follow the convention ns__fn.
> 
> Cons:
> - If you really need to have a specific name for your function you gonna
>   need a no-mangle approach.

Why? I mean you can still have the namespace and the freedom to use
namespace where you want.  Are you assuming the namespace is mandatory
here?

> - More debug info is required.
> - It is not entirely free of name conflicts, you can force a conflict by
>   create function that starts with double underscores which is not
>   recommended by C, since these names are reserved.