Ops on vnykmshr

The dismissal

Fri, 27 Mar 2026 00:00:00 +0000

A validation layer that checks 3 of 4 fields is worse than one that checks none.

Zero checks, the developer tests everything. Three checks, they assume the fourth is covered. That gap – between nothing and almost everything – is where the actual damage hides.

I keep running into this. Filed a security report recently – clear bug, one-line fix, obvious PoC. Response: “not applicable.” The code did exactly what I said it did. But the team’s threat model said “caller is trusted,” and three other fields had validation, so the missing one looked intentional. It wasn’t. It was just the one nobody got to.

Trust boundaries

Fri, 20 Mar 2026 00:00:00 +0000

I use coding agents on my own private repos every day. Security research, side projects, things I wouldn’t put on a public GitHub. Not something I’d do blindly with work source code though.

So when someone turns off WiFi to prove the agent needs a network connection, I get it. But that’s the architecture. It’s on the pricing page. The agent works on your local files, the reasoning runs on a remote model. Both true, neither a secret.

The personal agent trap

Sat, 28 Feb 2026 00:00:00 +0000

Spent a week going through the personal agent ecosystem – OpenClaw, ZeroClaw, PicoClaw, the whole *Claw family. Channel testing, security audit, the whole thing.

If you want a personal assistant that messages you reminders, triages your inbox, schedules things, posts updates – these frameworks are actually good at that. OpenClaw connects to 50+ channels out of the box, the setup is real, it works. For that, a $7 VPS and an afternoon gets you something useful.

Prescaling for a known spike

Fri, 15 Mar 2019 00:00:00 +0000

Our biggest sale event of the year is on the calendar. The date is fixed, the hour is fixed, and when it starts, traffic hits a multiple of normal within minutes. The engineering challenge isn’t handling surprise. It’s handling certainty at a scale we’ve never seen before.

We prepare for months. Six months out, teams start thinking about what their services need. Backend teams work with SRE and infra to define prescale configurations and autoscale rules. Terraform handles the provisioning. Every service team shares their estimates with infra, and the configurations get codified.

Hazard lights

Sat, 10 Jun 2017 00:00:00 +0000

There are about fifteen of us in the enclosure. Backend engineers, SRE, devops, infra – handpicked from across the floor. The rest of the team, about a hundred people, sit outside. They call us the fishes in the aquarium.

The aquarium has hazard lights. Physical ones – wired to fire on any 5xx in the system. When something breaks in production, the room goes red.

It sounds like a gimmick. It isn’t.

Nginx load balancing decisions

Thu, 18 May 2017 00:00:00 +0000

Nginx as a reverse proxy and load balancer is well-documented. The configuration syntax is not the hard part. The decisions are.

Algorithm selection

Three algorithms cover most workloads.

Round-robin (the default). Requests cycle through backends sequentially. Weights let you bias toward higher-capacity servers. Simple, predictable, works well when request processing times are uniform.

upstream api {
 server api-01:8080 weight=3;
 server api-02:8080 weight=2;
 server api-03:8080 weight=1 backup;
 keepalive 32;
}

The backup directive keeps a server in reserve – it only receives traffic when all non-backup servers are down. Useful for a smaller instance that can keep the service alive during a partial outage but shouldn’t take production load normally.

Zero-downtime deploys

Mon, 15 Jul 2013 00:00:00 +0000

I deploy everything from the terminal. No web interface, no CI service, no dashboard with green buttons. Just deploy production from my laptop, and the code goes live.

The setup is two pieces. A bash script that handles the remote work – SSH in, pull the latest code, run hooks. And a Node.js process that watches a file on the server and reloads the app cluster when the file changes. Between them, they do zero-downtime deploys in under ten seconds.

Running Node.js in production

Wed, 29 May 2013 00:00:00 +0000

We’ve been running Node.js in production since the 0.4 days. The language is easy to get started with. Keeping it running under real traffic is a different problem.

Process management

The application needs to start at boot, restart on crash, and respond to system signals. Upstart handles this on Ubuntu without additional dependencies:

description "myserver"

env APP_HOME=/var/www/myserver/releases/current
env NODE_ENV=production
env RUN_AS_USER=www-data

start on (net-device-up and local-filesystems and runlevel [2345])
stop on runlevel [016]

respawn
respawn limit 5 60

pre-start script
 test -x /usr/local/bin/node || { stop; exit 0; }
 test -e $APP_HOME/logs || { stop; exit 0; }
end script

script
 chdir $APP_HOME
 exec /usr/local/bin/node bin/cluster app.js \
 -u $RUN_AS_USER \
 -l logs/myserver.out \
 -e logs/myserver.err >> $APP_HOME/logs/upstart
end script

respawn limit 5 60 prevents a crash loop – if the process dies 5 times within 60 seconds, Upstart stops trying. The pre-start script verifies that Node and the log directory exist before attempting to start.

MySQL on XFS

Thu, 11 Apr 2013 00:00:00 +0000

XFS handles database workloads better than ext4 – better concurrent I/O, more efficient metadata operations for tables-heavy schemas, and delayed allocation that improves write throughput. The obvious approach is to change MySQL’s datadir in the config. The less obvious approach is bind mounts, which keep every path where the system expects it.

Setup

Install XFS utilities alongside MySQL:

sudo apt-get install -y xfsprogs mysql-server

Create the filesystem on the dedicated volume: