सतत

The Good, the Bad, the Ugly

शांतनु — Sat, 01 Aug 2026 09:55:21 GMT

“I don't want whatever I want. Nobody does. Not really. What kind of fun would it be if I just got everything I ever wanted just like that, and it didn't mean anything? What then?”

— Coraline

The Good:

LLMs are powerful. There are many novel ways of harnessing them. I got Claude implement a feature that I don't think I could have done myself within the time and budget constraints.

Task: Implement AirPlay sender to allow ESP32-based MoodPlay to stream content to any AirPlay compatible speaker.

I was able to use pyatv from my Linux machine to AirPlay an MP3 file to a HomePod mini. But if I targeted a Sonos speaker there was no audio. The Sonos app showed that it was playing something.

I had a starting point: an Open-source reference that worked and I wanted to reproduce it with an ESP32. I created a sandboxed environment for Claude on a Raspberry Pi4. I cloned repositories for the reference, connected an ESP32-S3-DevKitC-1 to it, and set up esp-idf so that Claude could flash the firmware and let it cook (--dangerously-skip-permissions). It took some time and lots of tokens but eventually I got the music streaming on HomePod mini.

Next step: get the AirPlay working with Sonos. I did some digging around. I was able to use AirPlay from an iOS device to Sonos. I captured the metadata from the speaker and added it to the CLAUDE.md file. Further, there is (audio) media server project, OwnTone, that allows streaming local media using AirPlay. Claude confirmed that using OwnTone it was able to stream the content to Sonos. Again we got a functioning open source reference that Claude was able to port to ESP32. Nicely done.

In the end it was ~2600 lines of portable C. It is a prototype to validate the idea and showcase a working setup/demo. I learned a lot about setting up tools for Claude and letting it work: the Good. That was the goal of the project: can we harness Claude for this? But, I have zero understanding of the code itself. If there's a bug in the code, I won't have any clue where to start debugging. And that is a problem.

The Bad:

This experience is from another project: Rast. I am using a PN532 NFC module to read NFC cards with a Raspberry Pi:

I have one working unit for myself and I am remotely setting up another one in Motionlab Berlin. This NFC board supports three communication modes— I2C, UART and SPI — and the mode is set using jumper pins. I had it working locally and I was struggling to get the same working on the remote setup. I could see the device under /dev , enabled the communication mode on RPi (config.txt), and confirmed the connection, but when I tried to read a card, it wouldn't work. With support from Felix, I kept on trying different modes but without success. I shared images of my working RPi, logs, and errors with Claude but beyond changing modes there was no progress. Claude was now suggesting that I shift kernel branches:

The real fix — downgrade the kernel on the broken Pi

Look at the confidence of that phrasing: ugh. I started talking to Felix and going through the wiring. Turns out I was using 5V for VCC on my system and Felix was using 3V. We switched to 5V and voila, it worked. UART NEEDS a 5V VCC connection and SPI works with 3V.

Claude was helpful with testing additional commands and configs that I was not aware of. But I also had some sense of when not to trust its recommendation and keep doing what I was doing. Its confidence and hallucinations were very telling: the Bad

The Ugly:

This is from a third project that is putting together a data pipeline and a user-facing dashboard built with Draxlr. The dashboard graph runs SQL queries and plots the result. The dashboard feature PR was created using an LLM and I was reviewing it. The changeset was 1390 lines of SQL (the Ugly):

Review is a strong word. I tested it. Ran a bunch of queries independently. Got an overall sense of how they contributed to the dashboard and "approved" the PR with a comment that we should also monitor the load on the postgres server when we visit the dashboard.

After a few days, there was a bugfix PR, from Claude:

-    ROUND(SUM(CASE WHEN er_kg > 0 THEN er_kg ELSE 0 END)::numeric, 0) AS "CO₂ Avoided (kg)",
+    NULLIF(ROUND(SUM(CASE WHEN er_kg > 0 THEN er_kg ELSE 0 END)::numeric, 0), 0) AS "CO₂ Avoided (kg)",

context: this query calculates the CO₂ emissions avoided. In the original statement, CASE WHEN er_kg > 0 THEN er_kg ELSE 0 END folds a NULL er_kg — i.e missing data — into 0. that's a bug. Missing data reads as zero emissions avoided, when the dashboard should show NULL or NA to say that data is missing.

Claude's bugfix wrapped the sum in NULLIF(..., 0), so a total of 0 comes back as NULL. It produces the right output, buts it's a patch on the symptom rathen than a fix of the cause..

I looked up the CASE statement:

If the ELSE clause is omitted and no condition is true, the result is null.

Delete the ELSE 0 and the CASE returns NULL for the missing row on its own; SUM skips NULL — the exact behavior we want. Claude had a better fix within one deletion, but it doubled down on its idea and instead of re-reading the statement. It took me a moment to realize this and I kept doubting myself. I ran a bunch of queries, confirmed things and followed up with the "requested changes" to the bugfix PR.

It is very easy to fall into this trap and click "Approve" on a PR. The LLM is delivering the functionality but I should really know my craft to be able to discern their outputs. And in the first case I mentioned I have no clue if the implementation is bad.

Concluding point:

In his book Skin in the game, Nassim bhai talks about the Greek phrase "Pathemata mathemata" — learning through pain. For me the pain is learning by debugging. With Claude and other LLM tools I have a feeling that I am getting away from that exercise and becoming complacent.

I'm not disciplined; I procrastinate and I rely on 'last-minute panic' mode.

Again, with LLM tools, I am leaning into my worst instincts. I rely on them to build a feature within limited time and budget, and I end up not learning or growing. There is an existential crises dimension to all this too, If the any and all features can be done by these LLMs, what am I for? Do I really want that? I think I got lucky catching Claude's mistake with the query. I can't be a mere meat proxy and I want to be an active human in the loop. As these tools get more powerful, I have to stay sharp, I don't know how yet, but that's the only way I'll survive as a programmer.

Collecting user feedback

शांतनु — Thu, 14 May 2026 03:16:53 GMT

I love simple basic things that come together to deliver a feature.

I am working on a new product around guided meditation. I am learning what makes a good meditation session and applying it to create lot of sessions on the fly (shoutout to ffmpeg, it rocks). While the hardware is coming together at its own pace and has its own challenges (maybe I will write a separate post around it), I wanted a quick way to share these sessions with the right people and collect their comments and feedback.

My requirements were clear: clean UX that allows users to tell us their name and what they thought of the session. For users, I didn't want them to create account, authenticate etc. For backend, I didn't want to over-engineer it. A central place where I can see all the comments, a notification when I get a new feedback.

Enter: Google Forms. Here are the steps I did:

created a google form with the questions
published it and got the public URL
I used a tool to get "entries" that represented my questions in the form
build a modal, custom UI that was consistent with my app (<iframe> embedding was not needed):

A POST (on backend using python requests or from javascript using a fetch call) request with all entries (questions) and their values. That's it. Response is registered, a new line is added to the linked spreadsheet

   const GOOGLE_FORM_URL = 'https://docs.google.com/forms/d/<formID>/formResponse';
   const ENTRY = {
     name:     'entry.1122112211',
     comment:  'entry.7766177172',
     segments: 'entry.1121121218',
     version:  'entry.178019201',
   };
   const formData = new FormData();
   formData.append(ENTRY.name,     name);
   formData.append(ENTRY.comment,  comment);
   formData.append(ENTRY.segments, segments);
   formData.append(ENTRY.version,  modal.dataset.version);

   try {
     await fetch(GOOGLE_FORM_URL, {
       method: 'POST',
       mode:   'no-cors',   // Google Forms doesn't allow CORS; submission still works
       body:   formData,
     });

I enabled email notification for new responses

When I first read about it from Claude and Stackoverflow I didn't believe it. In the past whenever I have tried to use google products, it was a pain, setting up authentication, configuration, permissions etc, it was always a PAIN. I thought Google would never allow a simple POST to register a response. But they do. Whenever a user leaves a comment, I get an email alert, I get neat breakdown of all the feedback on google forms response section

It is such a joy to stumble across this solution that just works. Plus, I was casually talking about it with Punch, it was TIL for him too, he loved it too and that also made me happy 😊.

Elevating the recipe

शांतनु — Fri, 06 Feb 2026 09:57:15 GMT

This is a follow-up on an earlier post: The right recipe. There I used Github actions to push data to a google spreadsheet that would update a looker studio dashboard.

The data we were handling didn't scale with google spreadsheet: both ingesting and accessing got slower. And we needed feature around data access controls, onboarding new clients. So we migrated to Supabase (a Postgres-based platform) and this is the new pipeline:

Hourly API pulls, store JSON response in a postgres table
Flatten part of JSON, subset of key value pairs into another table (materialized view)
A dashboard connected to this view

Infrastructure on Supabase is managed using SQLAlchemy models, I use Alembic for migrations, and Github actions for automation. Everything is version controlled with separate development and production environments.

New pipeline, new challenges. As the data grew, refreshing the materialized view started taking longer and longer. Github action failed as the query to flatten the data took longer and exceeded Supabase's timeouts. This is the query that I was running:

SELECT
  json_dump -> 'total' -> 'time' as time,
  json_dump -> 'total' -> 'value' as energey
FROM telemetry_table
WHERE telemetry_table.json_dump -> 'total' is not null;

We would adjust timeouts on Supabase roles and in a week or two the run would fail again. With close to 35k entries, it took almost 20 seconds to refresh the view.

At some point adjusting timeout didn't cut and I started looking into the query (explain analyze). Just parsing the JSON column was biggest factor. And as I extracted any field it would also increase the execution time.

Here is result from not including JSON column in query:

QUERY PLAN
Seq Scan on telemetry_table (cost=0.00..6843.29 rows=31629 width=8) (actual time=0.014..13.774 rows=31424 loops=1)
Planning Time: 0.230 ms
Execution Time: 15.178 ms

Here is result from using the JSON column, filtering null rows and extracting fields:

QUERY PLAN
Seq Scan on telemetry_table (cost=0.00..8181.60 rows=31471 width=8) (actual time=0.524..11074.447 rows=26012 loops=1)
Filter: ((json_dump -> 'total'::text) IS NOT NULL)
Rows Removed by Filter: 5412
Planning Time: 0.249 ms
Execution Time: 11079.209 ms

Regular search and llm both suggested me to use JSONB instead of JSON and GIN indexes. I created a backup table and altered the column to JSONB and that reduced the refresh time to less than 2 seconds. 10x improvement:

QUERY PLAN
Seq Scan on experimental_telemetry_table (cost=0.00..10696.63 rows=43026 width=8) (actual time=0.753..405.313 rows=37030 loops=1)
Filter: ((json_dump -> 'total'::text) IS NOT NULL)
Rows Removed by Filter: 6212
Planning Time: 0.778 ms
Execution Time: 931.380 ms

This was good enough for us and I didn't need GIN index. I am not filtering rows based on specific fields from the json data.

I will work on it more as we onboard new clients. We know the limitation of existing system: despite the best efforts we still have some manual steps for onboarding a new client. We have functions tied to the project. The data is time-series and there are other well suited databases specially meant for it. API response json varies for each client. Lets see how the project and solution (recipe) evolves.

Readable code

शांतनु — Fri, 30 Jan 2026 09:22:20 GMT

Background

I am revisiting Clojure by cleaning up and handling bad-data. Data is in a CSV file. There are two aspects to the problem - return clean rows and log/find bad ones.

This reminded me of a Clojure project I worked on in 2020 that handled drop shipping of orders from a third-party website. We would receive orders in CSV files, create these orders on our store, and respond with order tracking details in CSV format. Our store API worked with JSON data, so I had to handle the data transformation from CSV to JSON and back.

Problem: Handle the transition of data from flat to nested form

I was learning Clojure while working on that project, and I wrote the following function to flatten nested order data:

(defn single-row-per-sku
  "Convert consolidated processed orders to rows containing each SKU"
  [orders]
  (reduce (fn [result order] (concat result (map #(merge % (dissoc order :items)) (:items order)))) [] orders))

It works—I wrote unit tests to make sure it did. But it is dense and unreadable. I can't even recall how I explained this section during code review.

I used Clojure for the Brave and True for learning the language. It was a fun and refreshing read on introducing core concepts. The function that I wrote doesn't do justice to the quality of the content.

Brave Clojure also has this relevant quote from Alan Perlis:

It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.

I came from Python background. I also didn't know about threading macros in Clojure when I wrote above function. The quote really made sense to me once I learned about threading.

As I am revising Clojure concepts I am writing a lot of small function and testing them using the REPL. In the same vein I feel following is more readable equivalent:

(defn get-address [order]
  (dissoc order :items))

(defn get-items [order]
  (order :items))

(defn flatten-items [order]
  (let [address (get-address order)
        items (get-items order)]
    (->> items
         (map #(merge address %)))))

(defn single-row-per-sku [orders]
  (->> orders
       (map flatten-items)
       flatten))

Small functions. Threading macros. Names that make sense. Easy to read and functional. I don't think it's the final version-the ballad continues as I keep learning.

Getting started with powerful abstract tools: Emacs, Vim

शांतनु — Tue, 13 Jan 2026 15:35:00 GMT

Foundation: Text Editors

In his book The Dip, Seth Godin talks about snowboarding:

Snowboarding is a hip sport. It’s fast, exciting and reasonably priced; and it makes you look very cool. So why are there so few snowboarders? Because learning the basic skills constitutes a painful Dip. It takes a few days to get the hang of it, and, during those few days, you’ll get pretty banged up. It’s easier to quit than it is to keep going.

The brave thing to do is to tough it out and end up on the other side–getting all the benefits that come from scarcity. The mature thing to do is not even to bother starting to snowboard because you’re probably not going to make it through the Dip. And the stupid thing to do is to start, give it your best shot, waste a lot of time and money, and quit right in the middle of the Dip.

In the programming world, editors are an important basic skill-they are like the snowboard in this analogy. I would often read about vim and emacs. In college, I tried them a couple of times and got stuck. vim's swap file prompt was scary. When git opened vim's side-by-side buffer to resolve merge conflicts, I would start sweating.

Finding the Spark

Both VIM and Emacs are powerful and ubiquitous text editors and both are infamous for their steep learning curves. On my own, picking up such a tool was intimidating and frustrating. Here is what did the trick for me: watch a power user use them. It really hits you when this happens in person. I saw someone using org-mode in emacs, its table editor, exporting it in Markdown, HTML, and PDF, and then he used it to create a presentation using org-reveal, I was blown away 🤯. Later another friend of mine was using emacs and tramp to make code adjustments on a live server (avoid doing that) 🫡. During another pairing session, my colleague was using vim, navigating code, renaming variables with visual mode, and running macros-it was magic 🤹🏽. These moments made these editors cool for me-simple yet very pliable and that gave me enough escape velocity to push through the steep learning curve.

The Learning Curve

These editors are abstract tools that carry certain labels: "hard", "for smart people only", "set in their own ways". I struggled with two things when trying to pick them up. When I was working on a problem, I didn't want to get distracted by my editor, its mode, buffers, opening and closing etc. Second, I didn't consider myself smart and I kept my distance (thinking I was being mature) from tools that carried that impression.

But once I realized editors were a fundamental skill, I realized I should invest the time, don't make it (the editor) the last thing. I kept retrying, just like how I would approach a boss fight in a game. I didn't scare that easy in the next round, I noticed patterns, found my corner, got comfortable and started attacking. It grew on me. I acquired the taste. The same approach works for all crafts. Later, as I worked on new problem, these editors didn't get in the way-they provided the structure that will helped me focus.

I was lucky to work with such power users, and they allowed me the space to ask stupid questions. I still nag them about their setup when we meet in person. I use Emacs. I can't program without org-mode, magit, tramp et al. I am slowly working my way through its LSP support and configuring and tweaking things, one language at a time. I've tried other editors, I kept looking for these features and so far my attempts to install plugins/addons haven't worked. I keep returning to Emacs. As the landscape evolves with AI agents and MCP, I will explore whether I can integrate them with Emacs or if I'll need to adopt a more contemporary editor.

One Firmware for all ESP32 Variants: Solving Pin Mapping Pain

शांतनु — Sat, 16 Aug 2025 05:07:10 GMT

For my embedded project, I needed NFC card functionality for WiFi onboarding and user interactions. When integrating NFC functionality into ESP32 projects, pin mapping became a major pain point as I was working with different ESP32 boards.

Adafruit's arduino library example sets up the NFC reader in following manner:

#include <Wire.h>
// Using the SPI breakout, define the pins for SPI communication.
#define PN532_SCK  (2)
#define PN532_MOSI (3)
#define PN532_SS   (4)
#define PN532_MISO (5)
Adafruit_PN532 nfc(PN532_SCK, PN532_MISO, PN532_MOSI, PN532_SS);

These pins are placeholders and they would vary for each ESP32 variant and I had several: ESP-WROOM-32, ESP32-C3 Super Mini, S3, XIAO ESP32S3

I tested the reader across all these variants, different modes: SPI, I2C, wiring and reading different NFC cards.

First challenge was to figure out the mapping of pins. I found that espressif's arduino support is the best resource for it. For each board the file pins_arduino.h gives us the right pins. For the ESP32-C3 Super Mini board, the above example NFC reader code would become:

#include <Wire.h>
#define PN532_SCK  (4)
#define PN532_MOSI (6)
#define PN532_SS   (7)
#define PN532_MISO (5)
Adafruit_PN532 nfc(PN532_SCK, PN532_MISO, PN532_MOSI, PN532_SS);

But as I iterated through these ESP32 variants it became quite a pain adjusting those pins in the firmware and flashing it. Eventually I realized what would simplify this: I could have a single firmware that will work with everything:

#include <Wire.h>
Adafruit_PN532 nfc(SCK, MISO, MOSI, SS);

These pins, SCK, MISO are just like another constant: LED_BUILTIN, each Arduino board that doesn't use default pin 13, defines it in their pins_arduino.h header. Similarly all the variants have their own respective pins_arduino.h file with right pins. As I select the board inside Arduino IDE, these variables are always available. Further this pattern isn't limited to NFC readers, all the SPI peripheral using those pins would work.

CPR for a Docker container

शांतनु — Sat, 14 Jun 2025 10:35:55 GMT

Background

Picture this: You're experimenting with a Docker container, enable a feature, and suddenly your container is stuck in an endless restart loop. In a panic, you run docker compose down and watch everything disappear.

What You Should Do Instead

Don't lose the container, no matter how broken it is. Create a new image from the container using docker commit: docker commit <container-id> broken-image (you get container id using docker ps -a command)

From here there are a couple of things you could do. I was using Dockerfile to build and setup things. Modify it to use broken image as base and install the missing dependencies:

FROM broken-image:latest
RUN pip install <package-name>

And with this in place I can restart the docker compose stack.

Complicated changes

In case damage to your container is more severe there will be additional steps to revive the container.

Create image from your broken container: docker commit <container-id> broken-image
Create a new container from this broken image with a different entrypoint command (default command would restart the broken stack): docker run -it --entrypoint=/bin/bash broken-image:latest
Adjust or remove your changes, install missing dependencies, run migrations, what have you
Create a new image from this container that has all the fixes: docker commit <container-id-with-bash-running> fixed-image
Start a new container using this fixed-image and default command that brings up your original stack. In my case I removed build step from my docker-compose.yml file and pointed it to the new image:
```
   app:
     image: fixed-image:latest
```
Be careful with running faulty containers. I was really confused balancing docker compose commands and direct docker commands. I noticed that docker compose (re)start command would also (re)start all broken/existing containers and I had to manually remove them to be able to get the complete stack running again.

With this approach I was no longer building a new Docker image. Instead I used docker compose to start container using fixed-image and default command.

Summary:

Always test changes in development first
Use docker commit with your container before making risky changes

The right recipe

शांतनु — Fri, 30 May 2025 08:03:09 GMT

A colleague reached out to me for a pairing session on his project. I explained upfront that my hourly rate far exceeds most LLM subscription costs. He understood — while LLMs have their place, he needed help clarifying requirements and making real progress.

His existing setup: several machines pushing data to cloud, a looker studio dashboard connected to a placeholder google spreadsheet. To start, he wanted to make this dashboard live.

His requirements:

A live dashboard
Python based: his comfort zone
Daily data pulls: sufficient for non-temporal data

The solution was obvious, I have used it before: A GitHub action that runs daily, pulls cloud data and updates his spreadsheet — instantly making his dashboard live. I understand that I am contributing to the Github actions phenomena:

The trickiest parts? Wrestling with Google's Sheets API authentication and configuring GitHub secrets properly. But once those pieces clicked, everything flowed. Four one hour focused pairing sessions, that's it. He got a working solution plus he understood all the moving parts to continue experimenting on his own.

Four hours, serverless, an elegant solution. This is what going back to first principles looks like.

Setting Up Zephyr for ESP32: A Blinky LED Journey

शांतनु — Tue, 08 Apr 2025 11:49:10 GMT

I recently started working with a client on an ESP32 based project. Having prior experience with Zephyr on Nordic boards, I decided to leverage Zephyr's Espressif support for this project. While the documentation mentions the classic "blinky" example, I noticed it lacked the necessary device overlay file for my specific board.

Finding the Right Example

Fortunately, I discovered that the "PWM Blinky" example included sample overlay files I could reference. After compiling the example, I encountered this error when attempting to flash the device:

([Errno 5] could not open port /dev/ttyUSB0: [Errno 5] Input/output error: '/dev/ttyUSB0')

Running stat /dev/ttyUSB0 revealed that the device belonged to the dialout group. I fixed the permission issue by adding my user to this group:

sudo usermod -a -G dialout $USER

After rebooting to apply the new permissions, west flash worked perfectly.

Creating the missing overlay file

The basic Blinky documentation explains the need for an overlay file that points to the correct GPIO pin. Based on the PWM Blinky example, I discovered my board (esp32_devkitc_wroom) has an onboard blue LED connected to pwm_led_gpio0_2. I created following overlay file:

/ {
    aliases {
        led0 = &myled0;
    };
    leds {
        compatible = "gpio-leds";
        myled0: led_0 {
            gpios = <&gpio0 2 GPIO_ACTIVE_LOW>;
        };
    };
};

Success

With the overlay in place(samples/basic/blinky/boards/esp32_devkitc_wroom_procpu.overlay), I compiled the project and it worked:

west build -p always -b esp32_devkitc_wroom/esp32/procpu samples/basic/blinky

After flashing the board, there it was, blinking blue LED - a small victory that is still the first step and I keep revisiting it.

Next up: implementing the project's actual functionality, including WiFi provisioning, ePaper display integration, and LED ring controls.

Resolution / सलटारा

शांतनु — Fri, 08 Mar 2024 10:11:47 GMT

Continuing from the last post. The fact that Python interpreter didn't catch the regex pattern in tests but threw compile error on staging environment was very unsettling. Personally I knew that I am missing something on my part and I was very reluctant on blaming the language itself. Turns out, I was correct 🙈.

I was looking for early feedback and comments on the post and Punch was totally miffed by this inconsistency and specially the conclusion I was coming to

Umm, I'm not sure I'm happy with the suggestion that I should check every regex I write, with an external website tool to make sure the regex itself is a valid one. "Python interpreter ka kaam main karoon abhi?" :frown:

He quickly compared the behavior between JS(compiler complianed) and Python(did nothing). Now that I had full attention from him there was no more revisiting this at some later stage. We started digging. He confirmed that the regex failed with Python2 but not with Python3.11 with a simple command

docker run -it python:3.11 python -c 'import re; p = r"\s*+"; re.compile(p); print("yo")'

I followed up on this and that was the mistake on my part. My local python setup was 3.11, I was using this to run my tests and staging environment was using 3.10. When I ran my tests within containerized setup, similar to what we used on staging, Python interpreter rightly caught faulty regex:

re.error: multiple repeat at position 11

Something changed between python 3.11 and 3.10 release. I looked at the release logs and noticed this new feature atomic grouping and possessive quantifier. I am not sure how to use it or what it does 1. But with this feature, python 3.11 regex pattern: r"\s*+" is a valid one. I was using this to run tests locally. On staging we had python 3.10, and with it interpreter threw an error.

Lesson learned:

I was testing things by running things locally. Don't do that. Setup a stack and get is as close to the staging and production as possible. And always, always run tests within it.

You pull an end of a mingled yarn,

to sort it out,

looking for a resolution but there is none,

it is just layers and layers.