At the end of Post 1, my best Postgres version topped out at around 700 reservations a second. Beyond that, requests just piled up waiting for the same row.

So for this part, I tried something different: keep the stock count in Redis and do the check-and-decrement in a small Lua script. Same machine, same load test. It held up to about 6,000 reservations a second without overselling a single ticket.

It wasn’t free, though. Moving the count out of the database fixed the speed problem and created a couple of new ones, which I’ll get to at the end.

The idea

The problem in Post 1 was never really the SQL. It was that every reservation had to go to the database, lock the same row, and wait its turn.

So the obvious question was: what if the stock count didn’t live in the database at all?

Redis is a good fit for this. It keeps everything in memory, so reads and writes are fast, and it handles commands one at a time. That second part matters more than it sounds. If I send Redis a small Lua script, it runs the whole script atomically, so no other request can come in between and no race conditions.

Why a script and not just a few normal Redis commands? Because I’d be back to the same bug as the no-lock version in Post 1. If I read the stock with one command, check it in Node, and then decrement it with another, two buyers can still both read “1 left” in between and we will oversell. Putting the check and the decrement in one atomic script closes that gap.

So the plan is simple: one Lua script that checks the stock, decrement the stock, and records a hold for the buyer, all in a single step.

The script

Here's the part of reserve.lua that matters for this post: checking the stock and decrement the stock.

-- KEYS[1] = stock counter for the event
-- KEYS[2] = holds (sorted set, score = expiry time)
-- ARGV[1] = qty, ARGV[2] = holdId, ARGV[3] = expiresAtMs

local stock = tonumber(redis.call('GET', KEYS[1]) or '0')
local qty   = tonumber(ARGV[1])

if stock < qty then
  return {'SOLD_OUT'}
end

redis.call('DECRBY', KEYS[1], qty)                 -- take the tickets
redis.call('ZADD', KEYS[2], ARGV[3], ARGV[2])      -- hold them until expiry

return {'OK', ARGV[2]}

It reads the current stock and, if there isn't enough, returns right away with "sold out". Otherwise it takes the quantity off and record the hold. Because Redis runs the whole script atomically, no other request can change the stock between the check and the decrement. That's the gap that caused the overselling in Post 1, and here it simply can't happen.

The rest of the script does one more thing. It records a hold for the buyer with an expiry time, so if they don't complete the payment in time(i took 10 minutes) the hold is released and the ticket goes back into stock.

The setup

Focus on the top row: k6 → Ticket Service → Redis Instead of k6 → Ticket Service → Postgres. The tracing and metrics underneath are just how I watched it, they get their own post later in the series.

Correctness

Here's what happened when 1,000 reservations hit 100 tickets at once:

How fast is it:

I ran the same kind of ramp as in Post 1: start at 500 requests a second and keep adding more.

Up to 1,000 requests a second it was fine. p99 stayed under 50 ms and there were no errors. That's more than the 700 I got from Postgres, but I expected a bigger jump.

At 1,500 requests a second it still handled the load and still returned no errors, but it got slow. p99 went over 200 ms and reached about 256 ms by the end of that step. Using the same 200 ms rule as Post 1, I'm counting the knee as 1,000 requests a second.

To find out why, I looked at the bottom three charts in the above image. The ticket service was running as a single Node.js process, which can only use one CPU core, and its CPU usage went up to 1 core, so it was fully busy. The event-loop lag went from around 10 ms to almost 70 ms, which means requests were waiting in Node's queue before my code even started handling them. Redis, on the other hand, used at most 0.15 of a core the whole time.

So the row lock problem from Post 1 is gone, but now the Node process is the limit, not the database.

What happens with more Node processes?

If one Node process is the limit, the next step is to have more workers. So I ran the same ramp again with 1, 2, 4 and 8 ticket workers. I did it for both the Redis version and the best Postgres version from Post 1, the atomic update. Nothing else changed between runs.

Postgres didn't really move. It went from 600 to 800 and then back down to 700 with 8 workers. Adding workers just meant more requests waiting for the same row, so the row lock was still the limit.

Redis was different. Every time I doubled the workers, the knee roughly doubled too: 1,000, then 2,000, then 4,000. At 8 workers it reached 6,000. That's still more, but not double this time, because by then the laptop was running out of CPU cores to share between the workers, Redis, Postgres and k6.

This is the result I was really want. With Postgres, adding servers didn't help, because the database was the ceiling. With the counter in Redis, the ceiling goes up when I add servers.

Side by side

The setup that reached 6,000 requests a second

Same as before, with two changes: the counter lives in Redis, and the ticket service runs as 8 Node processes instead of one. Postgres isn't on this path any more. It still stores orders, but it's no longer touched for every reservation. (That part comes in Part 3.)

What it costs

Redis keeps everything in memory, so if it crashes I can lose recent stock changes unless it saves to disk or has a replica. I'll test that later in the series.

The bigger issue: stock now lives in Redis, but orders live in Postgres and they need to be in sync eventually.

What I'd try at 10×

One Redis server will eventually become the limit, because it handles commands one at a time. The usual next step is a Redis Cluster: several Redis servers, each holding a different part of the data.

There's one catch. Lua script touches several keys at once (stock, holds, events), and in a cluster a script only works if all its keys are on the same server. I already name every key with the event ID in curly braces, like stock:{event42}, which tells Redis to keep all of one event's keys together. So the script would work unchanged.

That spreads the load when there are many sales at the same time. One very big sale would still sit on a single server, though, so it would still be limited by one Redis.

Repo: https://github.com/manideep2810/Flash-Sale.git · Raw results: docs/results/post-2/ · Next: Post 3, Keeping Postgres and Redis in sync

My first version sold 115 of them. I ran it again: 106. Again: 107. Same code, a different wrong answer every time.

Getting it back to exactly 100 turned out to be the easy part. Making it fast was the hard part.

The problem

The rule is simple: never sell what you don’t have.

Each sale does two things: it takes one unit off the stock, and it creates a reservation (I call it a hold). Stock sold and holds created must always match.

All the stock for an event lives in a single row: the available column in the inventory table. Every buyer has to read that number and change it. One buyer at a time, that’s trivial. A thousand in the same second, all reaching for the same row, is where it breaks: two of them can read the same number before either one writes.

(Per-user limits come later in the series. This post is only about not overselling.)

How I tested

• Setup: one laptop (Ryzen 7, 16 threads, 32 GB RAM), Postgres 17 in Docker on default settings, one Node.js process with a pool of 20 connections. That means at most 20 database queries in flight; request 21 waits.

• Load: k6, ramping from 100 to 1,000 requests a second (+100 every 30 s). Requests keep arriving on schedule even when the server slows down, just like real buyers.

• Two kinds of runs:

  • Correctness: 100 tickets, 1,000 reservations fired at once. Did it oversell?

  • Speed: 200,000 tickets, so it never sells out. How much load can it take?

I report two numbers:

• p99: 99% of requests were faster than this. It’s what your unluckiest users feel.

• Knee: the highest rate where p99 stayed under 200 ms. Past it, things fall apart.

The Setup

Focus on the top row: k6 → Ticket Service → Postgres. The tracing and metrics underneath are just how I watched it, they get their own post later in the series.

1. No lock

The idea: read the stock, check it in code, then write.

SELECT available FROM inventory WHERE event_id = $1;
-- in Node: if available < qty → SOLD_OUT
UPDATE inventory SET available = available - $2 WHERE event_id = $1;
INSERT INTO holds (event_id, user_id, qty) VALUES ($1, $2, $3) RETURNING id;

Looks reasonable. Here's what happened when 1,000 reservations hit 100 tickets at once:

Three runs, three different wrong answers.

The classic race: two buyers read 1 left before either one writes, and both buy. Three runs, three different wrong answers.

Verdict: wrong, so speed doesn't matter.

2. Pessimistic Locking

The idea: lock the row before reading it. Only one transaction can hold the lock; everyone else waits until it commits. No gap, no race.

BEGIN;
SELECT available FROM inventory WHERE event_id = $1 FOR UPDATE;
-- in Node: if available < qty → ROLLBACK, SOLD_OUT
UPDATE inventory SET available = available - $2 WHERE event_id = $1;
INSERT INTO holds (event_id, user_id, qty) VALUES ($1, $2, $3) RETURNING id;
COMMIT;

Correctness: Same test, 1,000 reservations for 100 tickets

How fast is it:

It kept p99 under 200 ms up to 260 req/s (p99 173 ms). At 300 req/s p99 jumped to 200 ms. Then the line goes flat . That's the knee.

To see where the time went, I traced a single request just past the knee:

That one request took 6.6 seconds:

• ~6.4 s waiting for a database connection (the span for pool acquire)

• 225 ms waiting for the row lock (the SELECT … FOR UPDATE)

• ~2.5 ms doing the actual work (UPDATE, INSERT, COMMIT)

So 99.9% waiting, 0.1% working. And most of the wait wasn't even for the lock.

Here's what happens. One transaction holds the lock. The other 19 connections are each stuck inside their SELECT … FOR UPDATE, waiting their turn, and holding their connection while they wait. With all 20 connections busy, every new request has to wait in the app before it can even send BEGIN.

The lock creates the line, but the line forms at the pool.

Verdict: correct, but everyone waits in line, one buyer at a time.

3. Optimistic Locking

The idea: don't lock anything. Read the stock along with a version number, and only update if the version hasn't changed since you read it. If someone else got there first, try again.

SELECT available, version FROM inventory WHERE event_id = $1;

UPDATE inventory
   SET available = available - $2, version = version + 1
 WHERE event_id = $1 AND version = $3;
-- 0 rows updated → someone else won; wait 0–2 ms and retry (up to 200 times)

INSERT INTO holds (event_id, user_id, qty) VALUES ($1, $2, $3) RETURNING id;

Correctness

Same test, 1,000 reservations for 100 tickets:

How fast is it:

The worst of the three correct versions. It stayed quick up to 200 req/s (p99 just 8.9 ms), then fell off a cliff: at 300 req/s p99 hit 1.4 seconds.

At low load it's fast because there's no lock to wait for. Under a crowd, that's exactly what hurts it.

Where does the time go?

One ticket, 4 attempts. Each of the first 3 attempts are a SELECT followed by an UPDATE that comes back with 0 rows, because someone else changed the version first.

So under contention, optimistic locking still waits on the row, just like Pessmistic Locking. But it also throws away the work and starts over. Every retry keeps the connection busy longer, the pool runs dry, and new requests queue behind it.

Verdict: correct until the crowd shows up. Then it argues with itself.

4. Atomic conditional update

The idea: let Postgres do the check. Put the condition inside the UPDATE itself, so reading and writing happen in one step.

UPDATE inventory
   SET available = available - $2
 WHERE event_id = $1 AND available >= $2
RETURNING event_id;
-- no row returned → SOLD_OUT

INSERT INTO holds (event_id, user_id, qty) VALUES ($1, $2, $3) RETURNING id;

Is it correct?

Yes. Same test, 1,000 reservations for 100 tickets:

How fast is it?

The best by far. It kept p99 at just 51 ms up to 700 req/s, more than double Pessimistic Locking.

Then it hit a wall. At 800 req/s p99 jumped to 645ms.

Why is it so much faster? It still locks the row; every UPDATE does. But the lock is held only for that one statement. There's no trip back to Node between "check" and "write" while everyone else waits. Each turn on the row is as short as it can be.

Where does the time go?

The trace is boring: one UPDATE of 8ms and an insert of 7ms while the waiting time for acquring a connection from thepool reduced to 2s. The interesting part is Postgres CPU.

Through most of the ramp, CPU grew slowly and steadily, staying under 0.6 cores. That's healthy: more buyers, a bit more work, more tickets sold.

Then, at the knee, CPU shot up to 2 cores, nearly 4× in under a minute, while throughput rose only about 12% (700 → 783 req/s). Postgres was working much harder but selling barely any more tickets. The extra effort went into passing one row back and forth between 20 connections all waiting for it.

Verdict: the best of the four, and still capped by one row.

Side by side

The lesson

The SQL isn't the problem. One hot row is. Every buyer takes turns on the same row. Pessimistic makes them take turns politely, Optimistic makes them argue, and atomic makes each turn as short as possible. But at the end all the turns wait for a single row i.e available.

The fix is to take the counting off the database. In Post 2, stock moves to Redis, and a Lua script checks and decrements in one step: no row locks, no pool to run dry.

What I'd try at 10×

• Sharded counters: split 200,000 tickets into, say, 20 rows of 10,000, and send each buyer to a random row. This spreads the contention out. The cost is that the total becomes a sum you must keep right, and selling out gets fiddly when one row is empty but another isn't.

• A bigger machine buys time, not a fix. More cores make each turn a bit faster, but two buyers still can't change one row at the same moment.

Repo: https://github.com/manideep2810/Flash-Sale.git · Raw results: docs/results/post-1/ · Next: Post 2, Reserving in Redis with Lua