Go on vnykmshr

autobreaker: adaptive circuit breaking

Sat, 15 Nov 2025 00:00:00 +0000

The circuit breaker post from last year used a common trigger: trip after N consecutive failures. This works when traffic is predictable. It falls apart when it’s not.

At 10,000 requests per second, 10 failures is noise – a 0.1% error rate. A static threshold trips the circuit on what’s essentially a healthy service. At 10 requests per second, 10 failures is total collapse – 100% error rate over one interval. The same threshold that false-positives under high traffic is too slow to protect under low traffic.

Replacing OCR with Gemini

Thu, 10 Jul 2025 00:00:00 +0000

The previous post covered an address sanitizer that fixes mangled OCR output using multi-strategy matching. It works, but it’s treating a symptom. A smarter OCR step would make most of it unnecessary.

Traditional OCR extracts characters, then downstream code figures out what they mean. A separate pipeline handles structure, validation, error correction. The address sanitizer is part of that pipeline. It exists because the OCR engine doesn’t understand what it’s reading.

Fixing OCR addresses

Sat, 05 Jul 2025 00:00:00 +0000

OCR on government documents works well until you look at the address fields.

State: "DKI JAKRTA"
City: "JAKRTA PUSAT"
District: "MENTNG"
Village: "MENTENG"

Dropped vowels, character substitutions, truncated names. Indonesian place names get mangled in predictable ways – ‘A’ becomes ‘R’, characters vanish mid-word. The OCR engine reads the image fine. It just can’t spell.

The problem: take these broken strings and map them back to real administrative divisions. Province, city, district, village – the hierarchy matters, and every level needs to resolve correctly.

Circuit breaking in Go

Sat, 28 Sep 2024 00:00:00 +0000

A service calls a dependency. The dependency is slow or down. The service waits, ties up a goroutine, maybe a connection. Multiply that by every request in flight, and the caller is now as broken as the dependency it called.

Circuit breaking stops this. Instead of waiting on something that’s failing, stop calling it. Let it recover. Try again later.

Three states

A circuit breaker wraps external calls and tracks their outcomes.

Scout, plan, wait

Tue, 20 Aug 2024 00:00:00 +0000

The legacy codebase still runs the business. It is not small. It is six vertical functions deployed as separate services, sharing a data layer and a code tree, so “service” is a deployment unit here, not a boundary. It reads like a place rather than an architecture – rooms we know the shortcuts of, walls not quite where a greenfield build would put them. It has been running for years. It works.

Redis caching patterns

Thu, 20 Jun 2024 00:00:00 +0000

Put Redis in front of a database and reads get fast. The cost is a cache layer that’s now load-bearing, and a set of failure modes that come with that.

Three write patterns, three hard problems. The patterns determine consistency. The problems determine whether your cache layer is a net positive or a source of outages.

Write patterns

Cache-aside (lazy loading). The application checks cache on read. On miss, it reads from the database and populates cache. Writes go directly to the database; cache entries are either invalidated or left to expire.

The compiler as first reviewer

Wed, 18 Jul 2018 00:00:00 +0000

When I joined, a few of us who had recently come in would skip the daily standups. Clock in, clock out, heads down. The rest of the team wasn’t sure what we were working on. This went on for a while. We were reading the existing codebase and quietly rewriting pieces in Go – proving it could handle production before anyone had to commit to it.

That changed when Go got formally introduced. The direction came from above – Go was the language for the new services. Now it wasn’t a side experiment, it was the stack. And the PRs were coming in faster than anyone could review them all.

The GraphQL buffer

Fri, 20 Apr 2018 00:00:00 +0000

The GraphQL gateway started as a practical problem. We had mobile apps, web clients, and a growing number of backend services. Every client talked to every backend directly. When a new backend came up or an old one changed its API, every client needed updating. The gateway was supposed to fix that – one schema, one endpoint, clients talk to GraphQL, GraphQL talks to backends.

We built it in Go, starting from a fork of graphql-go. The fork grew over time – custom resolvers, caching layers, request batching, things we needed that the upstream didn’t have. We’d sync the fork every few months, but our changes kept growing. Five of us on the team, and most of the early days went into getting other teams to migrate their APIs onto the gateway. We built the base, got teams to add and own their own modules, then moved into a gatekeeping role – reviewing what went in, making sure the schema stayed coherent.

The same tree, twice

Fri, 22 Sep 2017 00:00:00 +0000

I am building the promocode engine again.

This is the second time. The first was in Node.js, at a previous company, on top of a small library called business-rules. The engine worked. I thought the shape was brilliant.

I am building it again in Go, from scratch, and the shape is exactly the same.

The first time

The first promocode engine was an AND/OR decision tree. A rule had conditions in groups, groups nested inside groups, leaves that compared a fact to a value. A rule was data in the database – a flattened tree stored across MySQL rows – but it evaluated against facts that had to be computed live.

Designing a wallet

Sun, 20 Aug 2017 00:00:00 +0000

I’m building a wallet service in Go. Users add money from their bank, the balance sits in the wallet, and they spend it on the platform. Small payments – the kind where going through a full bank authentication flow every time is more friction than the transaction is worth.

The pitch is simple: top up once, spend without thinking. No OTP for every payment. No redirect to the bank’s page. One click, done.

The first service

Wed, 15 Mar 2017 00:00:00 +0000

The monolith is Perl. No framework I can identify – just a large codebase that’s been growing for years. One Perl line does an alarming amount. I’m not a Perl developer and had to read through the code several times before I was confident I understood what it was doing.

Production goes down for days sometimes. When it does, the team spends hours tracing through the code to figure out what broke. That’s the context. The system works, mostly. When it doesn’t, nobody quite knows why.

Switching to Go

Wed, 07 Dec 2016 00:00:00 +0000

I’d been building production services with Node.js since 2013. Custom blog engines, API services, real-time backends. The ecosystem was rich, iteration was fast, and for moderate-load services, it worked well.

The limits showed up at scale. A latency-sensitive payment processing API exposed three problems that were manageable at lower traffic but compounded at higher volumes.

GC pauses. Node’s garbage collector introduced latency spikes that were unpredictable and hard to tune. For payment processing, where consistent response times matter more than peak throughput, this was the biggest concern. A p99 spike during a GC pause means a user staring at a spinner while their payment hangs.