Wyrm Progress and LLMs

Wyrm now has its own website: https://wyrm-lang.com. I’ve also sprouted organizations on both github and codeberg. I’ve made most of my own internal toying with Wyrm public - including editor support largely enabled via LLM generated stuff. Respecting codeberg’s wishes, the LLM driven elements are hosted elsewhere.

First - the big news here is that the Wyrm has a hand-rolled parser written in Wyrm.

The FOSS world is dividing itself on lines of LLM usage and acceptability. On one end, you have the pragmatic discussions of the Linux kernel team, and on the more idealogical end you have lists of the LLM sinners.

I’ve slowly transformed to more of the pragmatist in development than my younger far more idealogical self. Still, I find LLM use for any project something of a Faustian Bargain. The identity of the design is lost as the machine smooths the edges into a funhouse mirror combination of its source training data.

Wyrm is a massively ambitious project of the sort were a huge amount of uphill travel happens prior to seeing ANY satisfying reward for the work. My vision for the darned thing is a full-on Qt replacement (wyrm itself being the underlying data / query engine for the tooling). LLM generation let me move REALLY fast and enabled writing code in the language itself. I’ve thought of this process as ‘dog sledding’ - I take a code base and design doc, point the LLM at it, and see where the machine pulls to an end result. Still, with that process, I wanted to stay familiar with the underlying logic and design so the core repo stayed handcrafted.

My observation of launching LLMs to implement any is that there is a heavy tendency for models to outright copy the mechanics of similar pieces of software. There’s a lot of shared DNA for LUA within Wyrm, and of-course, the machine argues strongly to adapt more and more lua style patterns. Get a bit passive, and the docs suddenly start specifying conclusions from the grey goop of averaged designs close to what’s being built.

The GitHub repositories host a version of the Wyrm ecosystem miles ahead of the handcrafted variant. A lot of this work happened as I pulled various LLM subscriptions to see what the seemingly inevitable professional rollout would look like. Watching the LLM crank features and espouse the brilliance of my designs was damned addictive.

And then the cracks show. Despite the wyrm language hitting the self-hosting benchmark I tasked the LLM with, I found the interpreter failed some of the most basic primitives of the language. Specification gaps were filled in with wild and often nonsensical guesses. Worse, the models would on occasion reject out of hand basic design decisions while hallucinating others as concrete laws / rules to follow. And yet, the output is usable enough to hand to someone else to experiment with a new language. The full 9 yards of ecosystem - including VSCode integration and a debugger.

While I’ll have access to the models a bit more, I’m not planning on renewing my subscriptions. Professionally, it’s near impossible to fight the current tide of LLM use. It’s gotten very difficult to advocate for software quality as LLM code generates feature after feature so very quickly. I’ve actively seen the quality bar shift down multiple notches as excitement for fast results takes over. It’s easy to understand after accelerating so much work and quickly seeing slow arduous tasks done seemingly instantly.

Going forward, it’s a race to get the handcrafted implementation to a point where it can self-host before I lose the LLM subs. In the vanishing small chance the language becomes popular, it’ll be interesting to see how extreme the anti-LLM is crowd is. Will the earlier POC work permanently “taint” the language? I suspect the answer is yes. Sadly, I don’t know that it matters.