Working with LLMs

LLMs change the way I as a software engineer write code and design architectures. Here is my take and workflow.

How to read this post

This blog post is meant to be a living document and will be updated as things develop¹. Therefore I'll always include the current landscape of LLM tooling.

It is furthermore solely concerned with using LLMs for programming. It does not consider AI in other domains such as human computer interaction or image generation.

The Gist of it

LLMs are nothing more than a glorified snippet dictionary. Fight me. Embedding it more than that into my traditional workflow comes with the huge cost of frequent context switches from coding to prompting. More radical changes to my workflow would lead to arguably less satisfaction with questionable efficiency improvements.

Unfortunately, this snippet dictionary does not match my quality criteria, especially from an architectural point of view and using said pool exhaustively makes developing "dull"; it removes my incentive to think and to select the proper snippet.

The Landscape

The current tooling around AI seems very unstable. For the moment I don't spend too much time integrating tools but rather make them work with the least effort possible to actually be able to use them before they are outdated again.

I just recently finalized my nvim-setup and finally integrated codecompanion (Using claude-code via ACP) for quick in-buffer integration. I furthermore created a dedicated persistent floating terminal for interacting directly with the claude-code-TUI.

Before that I used roocode in VSCode, having to switch between LLM-driven development (VSCode) and finger-presses-key-driven development (nvim)².

I exclusively used Opus-4.5.

The Experience

For the last two months I used LLMs exhaustingly (for my usual consumption of tasty tokens). I was in between projects and had been given time (and 200$ on tokens/month) to play around with LLM assisted coding. It was... fun in the beginning, but quickly got to a point where I was more and more annoyed.

It just doesn't deliver on the promise of less headache. Especially DRY and Modularization are hard to enforce. And while it would have been able to code the challenges (this awesome blog, or iteraTinder) I gave it, it couldn't code the challenges in a way I wouldn't have the headaches just 3 months later when maintaining them or adding features. At this point I started asking myself what the point was.

Maybe you can grasp a little of the frustration and grief of the following prompts. You might want to join me in a minute of silence to connect to those feelings and pay your respect:

Please ONLY implement the endpoint. DO NOT add a rate limiter in again.

Claude doing claude things

NO RATE LIMITER! Delete it!

Ugh revert, I'll do it myself

You should NEVER expose the admin_ID to the frontend. Remove it from the endpoint.

...Thank you³.

Since the LLM did not hit the bar, I ended up implementing a lot myself, falling back to AI when I wanted my ideas challenged or an API integrated. Right now I would estimate to write 60% of the code myself.

Which leads to my main point: The time lost of switching the context from coding (pouring your thoughts into code) and prompting (pouring your thoughts into words, that hopefully result in a close-to-idea-implementation, checking this implementation and refining it further) is huge. And while I already do this for pull request of colleagues, it's different in my coding flow, because these context switches are much more frequent than in a pull request.

The Conclusion

For me, it is just not there yet to do the heavy lifting. Even in those cases where the community already cherishes LLMs.

It is hard to not lower my expectations both in terms of code quality and usability. I don't know if I am just more pedantic or think too different compared to the training data⁴. And I don't know if I actually want an answer to this question. I further wonder if there are other eccentric thinking people, that have a similar experience.

I observed, that I started to read docs more sloppy and shove bugs directly into the AI before understanding it. This automated routines lets me question my discipline to continually improve my skills.

To get the best results for my workflow, I consider LLMs as glorified snippet dictionaries. This mental model fits well with the actual nature of LLMs as well as the areas where they currently improve my workflow:

• Widening the solution space: AI knows a lot about framework/library surface-APIs. I don't. But now I know what I can look up and if it solves my problem.
• (Challenge your ideas): Since the output of LLMs is set into context by me and does not end up directly in the code, it cannot harm anyone. If it catches stuff I missed -- Great. If not, no harm was done.
• Boilerplate: Proof is left to the reader.

I might use LLMs too close to the code. Especially the last point raised in my experiences suggests that I could take a position further away from the code and closer to a PO/Mentor. But I will not mentor a herd of juniors without ever gaining the satisfaction of seeing their improvements and celebrating their success.

The Outlook

I will continue to use LLMs for the aforementioned tasks, but my trials showed rather depressing limits of even the current frontier models. Since I do not intend of publishing throw away projects, I don't see the reason of postponing headaches. I am eager to see the next releases and announcements of new models and hope LLMs get good enough to create pull requests without vast hand holding. That will my next landmark to reevaluate their capabilities.

Until then, the cost of focus-switching from coding to thinking of a prompt to implement what I want, waiting, reviewing is slower than writing the details by hand.

The Friends we made along the way aka. Footnotes