It has been roughly 2 months since I have provided any sort of update... certainly one of the longer gaps I took while working on this project.
I haven't done much work in this period, as I kept stumbling across little things that pop up while working on the parser, and that has been a major source of procrastination for me... and honestly, I didn't have the mental energy to deal with all of the small details involved in writing a parser by myself. All my mental energy was going towards other stuff in life, mainly my day job which puts food the my table... and the remaining I had to split on recreation and working on this project.
The problem is that this work has been more tedious than fun, so for that reason, I have decided to abandon the hand-written parser (for now at-least).
The sad part ends there, because I have no intention to stop work on this project, which is to finally bring a simple and easy to use scripting language to Linux 😁
After some discussion with more experienced members of the C community, I have come to know that I have been reinventing the wheel mostly with my parser. Initially I have been hesitant to use external tools to generate a parser, partly because I didn't want to learn how to use them.
Now after my raw experience with doing it the old straight-forward way, I have decided that being practical is more important... obviously I under-estimated what it would take to write a full parser by myself, it is task comparable to actually implementing the rest of the language, which actually does the work... and it is the fun bit, which parsing is not.
The tools I chose to use for the job are lex and yacc, these are the OG tools that were (and in some, still are) used in popular C compilers, so if those are good enough for that, they should definitely be more than sufficient for us!
And obviously, they are tightly integrated with C and produce C code for parsing, so that is easy to integrate them into our existing code.
I have been reading the book lex & yacc, 2nd Edition by John Levine, Doug Brown, Tony Mason to learn how to use them, the first chapter (available to read for free) gives a very good introduction, I was blown away at how simple it was, compared to reading all those technical research papers on parsing techniques 😵.
I regret not looking into it earlier, I probably would have made this decision sooner if I had.
I have still yet to get into the second chapter which explains it all in detail, and then I will start experimenting with code to replace our existing code (RIP).
Hopefully this means we can skip the hard parts of parsing and get into the meat, which is actually executing the code and making it do things.
I will keep you guys updated, hopefully the next one won't take another two months 🤭