Node on vnykmshr

Graph queries for a social feed

Tue, 10 Feb 2015 00:00:00 +0000

We’re building a platform with a social feed. Not a timeline of photos or opinions – a feed of plans. Things people intend to do. The interesting question isn’t “what did my friend share” but “what is my friend’s friend doing this Saturday, and can I join.”

That second question – friend-of-friend discovery – is what made me reach for a graph database.

Why graph

At first glance, a feed is just a list. The interesting part is underneath – a web of relationships: who knows whom, who created what, who joined what.

The middleware chain

Wed, 20 Aug 2014 00:00:00 +0000

I wrote about Express middleware chains last year as a pattern for keeping route handlers flat. That was a toy example. This is what it looks like in a real application.

We’re two people building a platform. The home page needs to show the user’s feed, friend recommendations, today’s happenings, friend request count, and unread notifications. That’s five data sources, each requiring its own database query, each needing the user to be authenticated first.

Node.js on a Raspberry Pi

Sun, 05 Jan 2014 00:00:00 +0000

When I first heard about the Raspberry Pi, I had to get one. A $35 computer that runs real applications. In India in 2013, getting one was the hard part.

Element14 showed “6 qty available.” I ordered. The status changed to “8-9 weeks lead time.” Forty-five days later, the Pi arrived. I plugged it in – nothing. A blinking red light, no display. I tried reloading Raspbian, different cables, different SD cards. Nothing worked. I packed it away and forgot about it for the better part of a year.

Express middleware chains

Tue, 15 Oct 2013 00:00:00 +0000

A typical Express route handler fetches some data and renders a template:

app.get('/book/:book', function(req, res, next) {
 Book.fetch(req.param('book'), function(err, book) {
 if (err) return next(err);
 Author.fetch(book.author, function(err, author) {
 if (err) return next(err);
 res.render('book', { book: book, author: author });
 });
 });
});

Two async calls, three levels of nesting. Add authentication, input validation, permission checks – and you’re five levels deep in a single route definition. The callback alignment pattern helps with formatting, but the nesting is structural. The route handler is doing too many things.

Middleware as steps

Express lets you pass multiple handler functions to a route. Each one gets req, res, and next. Do your work, attach results to req, call next() to pass control to the next handler. Bail out early if something fails.

Sorting by multiple fields

Fri, 28 Jun 2013 00:00:00 +0000

Array.sort takes a comparator. One field is straightforward:

data.sort(function(a, b) {
 return a.SCORE - b.SCORE;
});

Multiple fields gets ugly fast. Sort by score descending, then time ascending, then age ascending – you end up chaining comparisons:

data.sort(function(a, b) {
 return (b.SCORE - a.SCORE) || (a.TIME - b.TIME) || (a.AGE - b.AGE);
});

This works for numbers but breaks down with strings, custom transforms, or when you want to configure the sort dynamically. I wanted something reusable.

The pattern below is adapted from a Stack Overflow answer I kept coming back to – writing it up here for my own reference.

A fifty-line rate limiter

Wed, 05 Sep 2012 00:00:00 +0000

I needed rate limiting. Not a sophisticated system – something that stops a single IP from hammering the app. The options were a Redis-backed solution or an npm module I hadn’t vetted. I wrote one instead.

var hits = {};
var timers = {};

function ratelimit(max, mins) {
 max = max || 600;
 mins = mins || 1;
 var window = 60 * mins * 1000;

 return function(req, res, next) {
 var ip = req.headers['x-real-ip'] || req.socket.remoteAddress;
 var key = 'rl:' + ip;
 var n = hits[key] || 0;

 res.setHeader('X-Ratelimit-Limit', max);
 res.setHeader('X-Ratelimit-Remaining', max - n);

 if (n >= max) {
 var retry = Math.ceil(
 (timers[key]._idleStart.getTime() + timers[key]._idleTimeout - Date.now()) / 1000
 );
 res.setHeader('X-Ratelimit-Message', 'Enhance Your Calm');
 res.setHeader('X-Ratelimit-Retry-After', retry + 's');
 res.statusCode = 420;
 return res.end('Rate limit exceeded. Try again in ~' + retry + 's.');
 }

 hits[key] = n + 1;
 if (!timers[key]) {
 timers[key] = setTimeout(function() {
 delete hits[key];
 delete timers[key];
 }, window);
 }
 next();
 };
}

That’s it. Wire it into Express as middleware:

Connection pooling

Fri, 20 Jul 2012 00:00:00 +0000

The mysql npm module creates one connection per client instance. Every database call opens a connection, runs the query, and you’re responsible for closing it. Under concurrent requests, connections pile up. MySQL has a max_connections setting – default 151 – and when you hit it, new queries fail.

I hit this in production. The app was handling maybe forty concurrent requests and MySQL started refusing connections. The error is Too many connections and there’s no graceful fallback. Queries fail.

URL mapping in Redis

Sun, 10 Jun 2012 00:00:00 +0000

I’m working on an e-commerce app in Node.js. The store has SEO-friendly URLs – /blue-widget/SKU12345/p/ for products, /category-name.html for categories, /brand-name/ for brands. Those URLs are indexed, bookmarked, linked from everywhere. They can’t change.

Express routes are defined in code. The catalog has enough products, categories, and brands that I’m not writing a route for each one. I need a way to map the friendly URL to an internal route at runtime.