Skip to content

Rate limiting

This server's expensive surfaces are unauthenticated by design. Anyone may read a record, a leaderboard or an analysis without an account, and that openness is the point. But open does not have to mean unbounded. Without limits, one loop could pin the CPU on /api/analysis, or hold every socket the process can open. src/server/limits.js adds in-process limits with no dependencies.

Token buckets

Each caller has a token bucket per category. The bucket holds up to burst tokens, refills continuously at rate tokens per minute, and each request spends one token.

A token bucket is used instead of a fixed window because a fixed window lets a caller spend its whole allowance at the end of one window and again at the start of the next, a 2× burst at the boundary. A bucket refills continuously, so the average rate is the limit and short bursts are still allowed, up to the bucket's capacity.

bucketapplies toburstrefill per minuteenv
connectopening a WebSocket120240LIMIT_CONNECT_BURST, LIMIT_CONNECT_RATE
readGET/HEAD on /api/* and /records/*120240LIMIT_READ_BURST, LIMIT_READ_RATE
analysisanything under /api/analysis/1530LIMIT_ANALYSIS_BURST, LIMIT_ANALYSIS_RATE
writeany other method (POST, …) on any path3060LIMIT_WRITE_BURST, LIMIT_WRITE_RATE

Values must be positive numbers. Anything else falls back to the default.

Not limited by these buckets: static files (left to the platform or CDN), GET /healthz, the GET /email/* links, and all /gammonet/* routes. Workers authenticate with a shared key, they are supposed to be busy, and a volunteer worker polling for jobs is exactly the traffic pattern a limiter would punish.

When a bucket is empty:

  • HTTP gets 429 {"error":"too many requests","retryAfter":n} with a Retry-After header. The value is whole seconds and always at least 1, so a client that honours it really backs off instead of retrying immediately.
  • A WebSocket opened with an empty connect bucket is closed at once with code 1013 ("try again later"), before any session exists.

Why the numbers are this high

A key is an IP address, and an IP address is often a crowd

An office, a school, a café, and for most phone users carrier-grade NAT, where thousands of unrelated subscribers share a handful of addresses. The first version used connect: { burst: 10 }. Measured, that refused 30 of 40 simultaneous sockets from one address. On a phone network that does not look like "attacker throttled". It looks like "the site is broken" for everyone behind that exit.

The ceilings are set so that a busy shared address passes comfortably while a script still cannot run freely. They are a backstop against runaway loops, not a quota. The real limits on expensive work are elsewhere and are not per IP: analysis is queued and stored once per match, so a thousand requests for the same match cost one computation.

connect is tunable on purpose. No single value is right for everyone: 120 is generous for a home connection and tight for a carrier NAT pool serving a city. It is also exactly the value that made the project's own load test hang, because test/load.js opens every socket from one address. If a limit can break your load test, it can break a real population of mobile users, so the operator can change it.

TRUST_PROXY

The bucket key is the client address:

  • TRUST_PROXY=1: the left-most entry of X-Forwarded-For, which is the original client. The socket address is used if the header is missing.
  • otherwise: the TCP peer address (req.socket.remoteAddress).

Get TRUST_PROXY right. Both mistakes are silent, and they fail in opposite directions.

  • Behind a proxy without TRUST_PROXY=1: every request appears to come from the proxy, so all users share one bucket, and one busy player can rate-limit everyone.
  • TRUST_PROXY=1 without a proxy: X-Forwarded-For is whatever the client sends. A caller can send a new value with every request and bypass every limit. That is worse than no limiter, because each fake address also costs memory for a new bucket.

Set it to 1 behind Render, Railway, Fly, Caddy or nginx. deploy/docker-compose.prod.yml sets it, because Caddy always sits in front.

On the Cloudflare + Caddy setup, Caddy appends the peer address to X-Forwarded-For and leaves the rest of the header alone, so Cloudflare's real client address stays left-most. Replacing the header in Caddy would make every request look like a Cloudflare edge address. See Deploy to a VPS.

A consequence of trusting the left-most entry

Caddy passes through whatever X-Forwarded-For it receives. The committed configuration does not restrict ports 80/443 to Cloudflare's address ranges. So a client that connects to the origin IP directly, bypassing Cloudflare, can choose its own left-most X-Forwarded-For value and with it its rate-limit key.

Other limits

limitvaluewhere
WebSocket messages per connectiontoken bucket, burst 120, 40/s. A flooding client is closed (1008 too many messages), not silently ignoredSession.allow() in index.js
WebSocket message size64 KBmaxPayload
HTTP request body1 MB, then the request is destroyedreadBody()
emails per key and per address5 per hourmayMail()
worker batch64 positions per acquire/gammonet/acquire
analysis queue5,000 pending positions, then analysis queue fullAnalysisQueue
rate-limit buckets in memorywhen there are more than 50,000 keys, full buckets are sweptRateLimiter

Per process, deliberately

A shared limiter (Redis) would be exact across a fleet. It would also add a network hop to every request, plus a service to run and pay for. With N instances behind a round-robin proxy, a caller gets at most N times the limit. The code comment considers that the right trade at this size: the goal is to stop one script from hurting the box, not to meter an API anyone pays for. Sharding by match ID keeps N small.

Checking it

The default analysis burst is 15, so a 429 appears from the 16th request in a row:

bash
for i in $(seq 20); do curl -so/dev/null -w"%{http_code} " https://your-host/api/analysis/x; done

Rules engine, fairness protocol, verifier, analysis and worker: MIT. Server and client: AGPL-3.0-or-later.