When I’m going on long walks, I like to keep a paper notebook. This is something I’ve been doing since my Very First Long Walk back in 2018.

I wrote the whole thing in German; Fernwanderwege (long distance hiking trails) are the best German language schools you could ask for.
On the first few pages, I’ll write my hike itinerary (from/to, accommodation, distance, food stops), and the rest is a journal. I like journalling on paper, and I like being mostly offline when I’m walking; I’m trying to disconnect from the internet a little.

the first pages are always the neatest.
I also like being able to search my journal, and have it backed up. I’m very good at losing things. Scanning and OCRing the journal would be really nice. There’s just a few problems.
First: my handwriting is terrible. My dad once described it “as if a dying spider limped through a puddle of ink”, which I think is unfair (to dying spiders).

Here I am getting mad about a poorly-signposted five kilometer detour due to highway works and trying to decipher my own writing a day later (it says ‘signpost anything clearly’).
Second: My journals are a bit of a stress test for what counts as a ‘real word’. Place names are sometimes a little crazy. Also, I make spelling mistakes sometimes (also sometimes on purpose!), and write in a mix of UK and US English, and sometimes the occasional word that’s only in my brain in German.

re-remembered. remememberd? reremembered?
Third: My journals are a mess of scribbles with words scratched out sometimes and with the occasional e.g. arrow symbol to denote the way a particular bird sings:

I don’t think I ever found out which bird this was.
I tried a few OCR solutions, including Transcribus, a new app called Flowstate, and OlmOCR. But they all struggled with my handwriting.
I had heard that the LLMs are pretty good at transcribing handwriting into text. But I don’t just want to just blindly trust the output without checking it, so I built a little app around proofing and correcting.
It is (imaginatively) named Notebook Tools.
How it works
Here’s a video:
Basically:
- You take a photo of every page / spread of a notebook
- You get them onto your computer somehow, and then drag-and-drop them into the app
- It ships the pages off to Anthropic for OCR, at a cost between USD 0.05 and USD 0.10 / image (the video above skips past the delay there, it’s about 20s and we do up to 4 pages in parallel)
- You can then buzz through the images, check that the text is correct, fix anything that looks particularly ambiguous to the robot over
claudeslords - Export to PDF with the text embedded, or Markdown.
Why LLMs?
They have an idea of a world model which is uniquely powerful for figuring out what the hell I’m writing about. Because they’re trained on ~everything, they can figure out likely words from context in ways that aren’t possible with other models. This works great with place names – “Cotswolds Way” and “Doddington” implies that that messy word is likely “Tormarton” and that scribble at the end of the sentence is likely “Old Sodbury”. It means when I talk about the pub I’m staying in and scrawl “Dave, the Hauswirt” – a German word that might not even really belong in context1 – the LLM can still figure it out.

Flagged as unusual, but still accurately transcribed.
You also see evidence of the world model in other, weirder, ways:

I am sorry but there is no way this says ’toasted'.
Anthropic APIs for transcription
I am using the Anthropic API for this instead of just using a consumer-grade Claude.ai account. I couldn’t just plug in to my existing subscription because of some restrictions I don’t understand. This has implications on cost (it’s worse) and privacy (it’s better by default). The worst thing is that the UX for enabling transcription is kinda terrible as a result. Let’s talk about each of these in turn.
Cost
Transcription costs about US$0.05 - US$0.10 per photo, but each photo can be roughly a two-page notebook spread (two 13 x 21cm pages). The cost depends on the model in use: the less-powerful Sonnet costs about $0.05 per spread and the more-powerful Opus costs about $0.10. I found Opus was better enough to warrant the extra cost, but I guess Sonnet will work fine for people with halfway-legible handwriting. I added a little option switch for this:

I am team Opus
Part of why it’s expensive is the production of bounding-boxes and alternatives. If you just get the LLM to dump text, you can get away with much cheaper transcription, but the bounding boxes make the transcription UI and the exported PDFs really nice.

I love that mousing over a line on one view highlights it on the other.
Privacy
Privacy is a concern; I feel a little uncomfortable about the fact that I’m shipping my journal to a company whose business model craves human-authored text (but not uncomfortable enough; and I guess I think that Anthropic are one of the better ones). Stuff that’s sent over the APIs is apparently not used for training and theoretically mostly deleted after 30 days.
Login is a pain
This is the big one. Currently, to use transcription in the app, you need an API key of some kind, and that means:
- getting an Anthropic developer / console account
- loading up some credits
- copying a key into the app, or signing into the
antCLI on the same computer.
This really bothers me, because it means that Notebook Tools isn’t something that I’d ever really productionise / distribute outside of developer circles. If I wanted to distribute more broadly and make the UX halfway reasonable for ‘regular people’, I’d probably have to:
- Setup my own infrastructure to call through to the Anthropic API
- Build user account management
- Figure out how to bill people
- Figure out what to do in the cases where someone scans a page of handwritten text with something illegal written or drawn on it.
I don’t want to have to deal with any of that. It feels like there’s something powerful about the idea of being able to “BYO machine intelligence”, instead of an LLM provider being something that every app needs to deal with (and, probably, bill for) individually.
Other models?
The post that sparked this idea explicitly mentioned Gemma 4 31B because it’s way closer to ‘commodity’: you can run it in loads more places, and much more cheaply. Maybe it’d be good enough for me? I prototyped Gemma 4 running on Tinfoil (which runs in a secure enclave and therefore has strong privacy guarantees!) and found that it was able to transcribe a two-page spread for less than one US cent ($0.01). For my handwriting, it was a little less accurate than Sonnet again, and much worse at (a) place names (b) identifying areas of uncertainty and alternatives. But at a fraction of the cost of the Anthropic models, it’s probably not a bad option for most people.
It’s got a slightly lower max image resolution than Opus / Sonnet, which might be a contributing factor – at some point, I’ll (maybe) experiment with splitting a spread into two individual pages before sending to the LLM, and see if that helps with accuracy.
Gemma 4 seemed to not be able to fathom that there’d be more than one interpretation of a single word though. Here’s some junk from the ‘reasoning’ where I’d written the word Dorfromantik2:
I wonder how it would change if I’d written the prompt in German.
Development approach
100% of the code in the repo was written by Claude Opus 5.5. I’ve skim read every change, and banned the agent harness from making its own commits. It is definitely faster than making software by hand (and weirdly more capable; there’s loads of tasks where I’d just have no idea of how to get started, like edge detection). You get more ‘software’ for the same amount of effort.
I’m trying to work at a level of abstraction that’s higher than writing code, but in the weeds about all the details, which pretty much precludes ‘one-shotting’ or ‘slot-machining’ anything. The problem, is of course:
- Software is the accumulation of thousands of decisions, and you don’t know which ones you’re skipping when you’re not writing the code (though, to some extent this has always been true, go read You Already Don’t Understand Your Code).
- Software is also about craft. You don’t know what’s possible with the material and techniques unless you’re wrestling with them.
I tried a few new (to me) ‘knowledge management’ things in an attempt to make working with the agents more… useful? maintainable? In particular, I’ve got a decision log stored as a set of markdown files in decisions/, and an evolving list of forward-looking tasks in tasks/. I mostly write the tasks myself, but sometimes we’ll3 spin parts of the current task out into future tasks if it seems like the scope on the current task is creeping, or the task is more complex than I initially believed (it is software development, this happens constantly). The decisions/ are mostly agent-maintained based on conversations that we have when planning a feature. The idea is to capture who made certain choices and why, so that e.g. I can tell if a certain behaviour exists because I / the LLM explicitly had a reason for it, or if it exists by accident. I’m hoping that this’ll help me avoid Chesterton’s Fence-style situations in the future. But I think it’s also I think a way of wrestling with the fact that I want the effort to be seen, and the existence of the code doesn’t prove much anymore.
When implementing tasks/, I’ll typically read through the task file and add stuff to it, especially for things that I’ve got a lot of opinions about, and then kick off work with a prompt in plan mode like this:
/plan
Hello, today we're working on `tasks/missing-page-export.md`,
please ask lots of questions to resolve ambiguity. ty
Then, a bunch of back and forth on decisions and eventually execution. I’ll occasionally add a task for a refactor if I’ve spotted something design-ish that’s frequently getting in the way, and sometimes ask the agent explicitly to look at the list of upcoming tasks and identify refactor candidates that will make future work easier. Or ask, “what did you get stuck on this session? Are there any changes that we should make to reduce friction in future?”
Overall, it’s worked pretty well, and I haven’t had many problems with maintainability. But the project is still very small, and also greenfield.
Building just for yourself
Both before LLMs and after them, the temptation has always been Scope Creep. Figuring out how to cut scope, deciding ahead of time that a project is going to be just for you and then actually sticking to that and calling it finished earlier – giving the system enough grace to make it fun to use, but treating code changes as part of that use – is all the battle.
Building a demo – or something that’s useful on just one machine, in just one point in time, on just a limited dataset – is about 20% of the work. Making it distribution-grade is the other 80%.
There are lots of examples of that ‘other 80%’ work in this project:
I’ve got a project save / load system that I don’t strictly need, which includes being able to double-click a project in OSX and have it open, and the ability to re-link missing images using EXIF data. None of this is necessary to the core task of ‘OCR a notebook’.
I’ve got a settings dialog for model selection and for what to do with underlines (do they convert to
*text*or_text_?)I made loads of tiny decisions around what happens if you were logged in, and there are cached transcripts, but you are no longer logged in. Definitely not necessary if “works on my machine” is the end-goal.
This is maybe all unnecessary, but I’ve still put a lot of thought into small details because they make things ergonomic for future me. I will forget how this all works in 6 months, probably, and so the more the tool is ‘complete’ (with tooltips! and inline help! and install instructions! And a cute dock icon!) The more likely it is to be useful to me in the future.
Also, just deep in my bones, I like making things that are polished. I’m convinced that it’s possible to take an anti-slop (or at least, min-slop?) approach to LLM-driven-development, and I’m trying to illustrate that approach in this project.
That’s it
That’s all for now! The software is on Github, it works on recent MacOS.
I thought this meant the owner / innkeeper / proprietor, but now that I’m writing about it, I wonder if I’ve learned it incorrectly from context and/or gotten confused with Gastwirt. But it doesn’t matter! The LLM still figured it out. ↩︎
Dorfromantik: A “nostalgic feeling for the countryside”, and also a great board / videogame. ↩︎
I’m a bit uncomfortable using the word ‘we’ to refer to development that I’m undertaking with an agent harness, but don’t really have any good alternatives yet. ↩︎