After the previous successful experiments, this will for the new direction. I'll plot down the plan first, then detail it and then reason about why i got here.
As at least the direction will be new to me i'll call this a research plan. Some elements will be known, but bringing it all together i think is new, as is the breadth of scope, i believe. The plan is to develop a sort of high level vm or compiler kit (hlvm as opposed to llvm), that can be used to compile dynamic object oriented languages.
The basic block, or foundation, is a multi layered compiler for a static language. The language shall be designed to facilite the stated objective (ie not general purpose). I envision something like simplified crystal.
The multi-layered here means that the compiler is a series of transpilers to lower languages. Each of these shall be quite well defined and may (if the need arises) get their own parser or Interpreter.
The languages should be integrated towards the goal of implementing a dynamic language, in the same sense that eg a c compiler, linker and assembler work together to implement the c language.
Concretely this means a common menory model that is object based. And also calling mechnism that allow features of dynamic languages to be implemented easily.
The system should provide core functionality and mechnisms for extensibility at every level. Core functionality in a compiler context means the minimum of being able to compile itself, and not super much more.
Extensibility means more functionality can be added by providing packages that were not part of the core (external packages). At different levels these can means different things. At an assembler level it could mean a broader support for specific instructions, eg SME. At higher levels it could be providing more or better primitives, or library code, or indeed syntax extensions (adding to or replacing the parser).
As detailed in the previous post, partial evaluation can play an essential role in the rather complex task of compilation, as it breaks it down into the easier tasks of interpretation and partial evaluation.
Specifically the means a static language would need to be build with a compiler for it. And then an Interpreter and a partial evaluator for the dynamic lanuage.
Possibly evaluation can continue in lower levels, but this wil have to be explored.
Going into more detail about it i'll outline some of the directly actionable things and will see how others evaolve. As it is reasearch it is more exploritory, though just having watched a video about 15 years of v8 developemnt, maybe the difference is not as big as i imagine.
From the last 10 years of experimenting i have a good idea of what is worth keeping and what needs to change, so let's start from bottom up (language 0...x in the last post).
Assembler languages are machine specific, more or less one to one mappings, of text to binary code. I'll start with Arm, as that is simpler.
Even it is close to machine code, normal assemblers already encode quite a bit of assumptions. Since we don't want c assumptions this will have to be redone, but quite contrary to my inital assumption, this is not difficult at all (it is after all one to one).
To be more explicit, we want a uniform memory model down to here (not the c model). Another typical c assumptions is taht code and data are seperated, but we want inline data (eg caches), and trampolines and possibly other constructs (like quajects), or changable call-ins.
In fact there is a suprising amount to be discovered here, and also some things to leave behind (like c-linking and the c-calling)
A Risc level has proven useful as a base from which different assemblers can be implemented (in the future). This level can be seen as quite close to arm or risc-v.
Another benefit of such a level is that we can write a relatively simple Interpreter for it and use that to verify the higher level functionality, when compiling to this level. Doing this we can save a lot of time debugging binary code (without a debugger)
This would classically be an IR, born out of the need to bridge the conceptual gap between a static language and assembler (risc).
It's called slot, because it already has objects (or chunks of data) and is all about accessing the slots of those. One could see them as arrays or as an equivalent lowering as self was to smalltalk. It's bout moving data in and out of those slots and having primitives defined on slots (and not int etc as c would)
Especially on arm this maps very well onto assembler while providing a good abstraction level.
The static language i am about to write. Kide means crystal in finnish and it will probably end up looking like that a little for several reasons.
Prime among those reasons is that i intend to use ruby to run kide while bootstrapping is not achieved, ie for a good while. So the synatx has to be 100% ruby compatible (at this point).
This will be achieved by anotation, naming convention and also changing the semantics of ruby. In the simplest way this may mean defining things to be static, when ruby has them dynamic.
I also want to mention some more things that will change from the current approach. The biggest is obviouly that i'll start by writing a static language and then an Interpreter and evaluator for ruby in kide.
But there was also too much ruby magic in the previous approach, so that goes
And also the stages of compilation, were not clean and during risc interpretation ruby code was partially used. That will go too.
Also the handling of primitives was not correct and needs to be revised.
