The Good, the Bad, the Ugly

शांतनु — Sat, 01 Aug 2026 09:55:21 GMT

“I don't want whatever I want. Nobody does. Not really. What kind of fun would it be if I just got everything I ever wanted just like that, and it didn't mean anything? What then?”

— Coraline

The Good:

LLMs are powerful. There are many novel ways of harnessing them. I got Claude implement a feature that I don't think I could have done myself within the time and budget constraints.

Task: Implement AirPlay sender to allow ESP32-based MoodPlay to stream content to any AirPlay compatible speaker.

I was able to use pyatv from my Linux machine to AirPlay an MP3 file to a HomePod mini. But if I targeted a Sonos speaker there was no audio. The Sonos app showed that it was playing something.

I had a starting point: an Open-source reference that worked and I wanted to reproduce it with an ESP32. I created a sandboxed environment for Claude on a Raspberry Pi4. I cloned repositories for the reference, connected an ESP32-S3-DevKitC-1 to it, and set up esp-idf so that Claude could flash the firmware and let it cook (--dangerously-skip-permissions). It took some time and lots of tokens but eventually I got the music streaming on HomePod mini.

Next step: get the AirPlay working with Sonos. I did some digging around. I was able to use AirPlay from an iOS device to Sonos. I captured the metadata from the speaker and added it to the CLAUDE.md file. Further, there is (audio) media server project, OwnTone, that allows streaming local media using AirPlay. Claude confirmed that using OwnTone it was able to stream the content to Sonos. Again we got a functioning open source reference that Claude was able to port to ESP32. Nicely done.

In the end it was ~2600 lines of portable C. It is a prototype to validate the idea and showcase a working setup/demo. I learned a lot about setting up tools for Claude and letting it work: the Good. That was the goal of the project: can we harness Claude for this? But, I have zero understanding of the code itself. If there's a bug in the code, I won't have any clue where to start debugging. And that is a problem.

The Bad:

This experience is from another project: Rast. I am using a PN532 NFC module to read NFC cards with a Raspberry Pi:

I have one working unit for myself and I am remotely setting up another one in Motionlab Berlin. This NFC board supports three communication modes— I2C, UART and SPI — and the mode is set using jumper pins. I had it working locally and I was struggling to get the same working on the remote setup. I could see the device under /dev , enabled the communication mode on RPi (config.txt), and confirmed the connection, but when I tried to read a card, it wouldn't work. With support from Felix, I kept on trying different modes but without success. I shared images of my working RPi, logs, and errors with Claude but beyond changing modes there was no progress. Claude was now suggesting that I shift kernel branches:

The real fix — downgrade the kernel on the broken Pi

Look at the confidence of that phrasing: ugh. I started talking to Felix and going through the wiring. Turns out I was using 5V for VCC on my system and Felix was using 3V. We switched to 5V and voila, it worked. UART NEEDS a 5V VCC connection and SPI works with 3V.

Claude was helpful with testing additional commands and configs that I was not aware of. But I also had some sense of when not to trust its recommendation and keep doing what I was doing. Its confidence and hallucinations were very telling: the Bad

The Ugly:

This is from a third project that is putting together a data pipeline and a user-facing dashboard built with Draxlr. The dashboard graph runs SQL queries and plots the result. The dashboard feature PR was created using an LLM and I was reviewing it. The changeset was 1390 lines of SQL (the Ugly):

Review is a strong word. I tested it. Ran a bunch of queries independently. Got an overall sense of how they contributed to the dashboard and "approved" the PR with a comment that we should also monitor the load on the postgres server when we visit the dashboard.

After a few days, there was a bugfix PR, from Claude:

-    ROUND(SUM(CASE WHEN er_kg > 0 THEN er_kg ELSE 0 END)::numeric, 0) AS "CO₂ Avoided (kg)",
+    NULLIF(ROUND(SUM(CASE WHEN er_kg > 0 THEN er_kg ELSE 0 END)::numeric, 0), 0) AS "CO₂ Avoided (kg)",

context: this query calculates the CO₂ emissions avoided. In the original statement, CASE WHEN er_kg > 0 THEN er_kg ELSE 0 END folds a NULL er_kg — i.e missing data — into 0. that's a bug. Missing data reads as zero emissions avoided, when the dashboard should show NULL or NA to say that data is missing.

Claude's bugfix wrapped the sum in NULLIF(..., 0), so a total of 0 comes back as NULL. It produces the right output, buts it's a patch on the symptom rathen than a fix of the cause..

I looked up the CASE statement:

If the ELSE clause is omitted and no condition is true, the result is null.

Delete the ELSE 0 and the CASE returns NULL for the missing row on its own; SUM skips NULL — the exact behavior we want. Claude had a better fix within one deletion, but it doubled down on its idea and instead of re-reading the statement. It took me a moment to realize this and I kept doubting myself. I ran a bunch of queries, confirmed things and followed up with the "requested changes" to the bugfix PR.

It is very easy to fall into this trap and click "Approve" on a PR. The LLM is delivering the functionality but I should really know my craft to be able to discern their outputs. And in the first case I mentioned I have no clue if the implementation is bad.

Concluding point:

In his book Skin in the game, Nassim bhai talks about the Greek phrase "Pathemata mathemata" — learning through pain. For me the pain is learning by debugging. With Claude and other LLM tools I have a feeling that I am getting away from that exercise and becoming complacent.

I'm not disciplined; I procrastinate and I rely on 'last-minute panic' mode.

Again, with LLM tools, I am leaning into my worst instincts. I rely on them to build a feature within limited time and budget, and I end up not learning or growing. There is an existential crises dimension to all this too, If the any and all features can be done by these LLMs, what am I for? Do I really want that? I think I got lucky catching Claude's mistake with the query. I can't be a mere meat proxy and I want to be an active human in the loop. As these tools get more powerful, I have to stay sharp, I don't know how yet, but that's the only way I'll survive as a programmer.

सतत (Posts about llm)

The Good, the Bad, the Ugly

The Good:

The Bad:

The Ugly:

Concluding point: