PostgreSQL 14 and the price of a connection

A few hundred connections, mostly idle, and the whole server gets slower. That was Postgres for as long as I remember it: a process per connection, and past some count the snapshot bookkeeping starts to eat everything. PostgreSQL 14 is released today. I had the RC on a test box for a week, so a few notes from a backend seat. The headline for me is that this internal work got a serious rewrite. On our connection-heavy profile, many workers, short queries, lots of idle time, the difference is visible without squinting. Idle connections finally cost close to what people always assumed they cost. ...

September 30, 2021 · 2 min · Murat Useinov

Redis Streams is a log, not a better queue

“Should we move the queue to Streams? They are the proper way now.” A colleague, this week, about our Redis list queue. Wrong question. Streams are not a newer version of the same thing. A list queue is a hand-off. LPUSH on one side, BRPOP on the other, and once a consumer takes the message, it is gone. Crash after the pop and the message died with you, unless you built the pending-list dance yourself. No history, no second reader. For “send this email eventually” it is honestly enough. ...

August 20, 2021 · 2 min · Murat Useinov

Nobody closes your database connection anymore

“MySQL server has gone away”, first thing in the morning, on a worker that had no traffic all night. That was week one after we moved a project from FPM to RoadRunner. For all these years FPM managed our database connections. Not with clever code. By dying. Process ends, connection closes, transaction rolls back, session variables reset. Nobody thought about connection lifecycle, because there was none. Move the same code to RoadRunner or Swoole workers and the topic is back. A connection lives for hours now. Two things go wrong, and both showed up within a week. ...

July 20, 2021 · 2 min · Murat Useinov

Attributes in Symfony 5.3

#[Route('/orders/{id}')] right above the method, and nothing in config/routes. Symfony 5.3 came out last week, and with PHP 8 attributes configuration finally found its place. Routes, autowiring hints, some validation, native syntax next to the code: #[Route('/orders/{id}', methods: ['GET'])] public function show(int $id): Response { // ... } I spent years defending YAML routing. The argument was separation: code is code, wiring is wiring, one file shows the whole URL map. The argument was never wrong. It lost to practice. In every real project the first thing you do with a route is jump to the controller, and the first thing you do with a controller is wonder which route hits it. Two files, one mental join, forever. Attributes remove the join. Rename a method, the metadata moves with it. Delete the class, no orphaned YAML block stays behind to confuse the next person. ...

June 4, 2021 · 2 min · Murat Useinov

Octane and state that outlives the request

One customer sees another customer’s cart. No crash, no error in the log. That is the bug Laravel Octane brings to an old codebase, and the benchmarks are the least interesting part of the story. Octane is out in beta, Swoole and RoadRunner became first-class overnight. In FPM every request gets a fresh application. Boot, handle, die. The model forgives everything. Under Octane the framework boots once and workers reuse it. Bootstrap cost drops to near zero, hence the pretty numbers. But every singleton is now shared between requests, and between users. ...

May 5, 2021 · 2 min · Murat Useinov

Symfony Runtime: front controller as a callable

public/index.php, unchanged since I learned Symfony. Create the request from globals, run the kernel, send the response, terminate. Symfony 5.3 arrives next month, and the Runtime component rewrites this file. It looks like a small refactoring. It is a statement about where PHP is going. Baked into those few lines is one big assumption: one process, one request, then we die. FPM made the assumption true for fifteen years, so nobody saw it as an assumption. ...

April 16, 2021 · 2 min · Murat Useinov

The outbox table

About once a month an order existed and nobody heard about it. The order service saved the order, then published order.created to RabbitMQ, and once a month the process died between the two. Rare enough to be mysterious. Frequent enough to ruin a weekend. Two operations, two systems, no common transaction. Whatever order you do them in, you lose. Commit first, then publish: the order exists, the event never leaves. Publish first, then commit: the commit fails, consumers are already busy with an order that does not exist. We had the first variant in production. ...

March 23, 2021 · 2 min · Murat Useinov

TTL is not an invalidation strategy

Support ticket: a user changed his name, the site still shows the old one. The profile sits in Redis with TTL one hour. Fine, we set five minutes. A week later the hit rate drops and MySQL feels it. This seesaw has no good position. TTL is insurance, not invalidation. The obvious upgrade is cache-aside with explicit delete. Read: try the cache, miss, load from the database, write to the cache. Write: update the row, delete the key. Looks correct. It has a race. ...

February 24, 2021 · 2 min · Murat Useinov

PHP 8 in production: dependencies first

composer why-not php 8.0. That is the first command of the migration, and it has nothing to do with syntax. PHP 8.0 is six weeks old. Twitter is full of match expressions and constructor promotion. Meanwhile the real upgrade of a working project is a dependency problem. Your code is maybe twenty percent of what runs in production. The rest is the framework, thirty packages and a dozen extensions, and any one of them can be the blocker. ...

January 8, 2021 · 2 min · Murat Useinov

Attributes: metadata moves in with the code

#[Route('/orders/{id}', methods: ['GET'])] public function show(int $id): Response Symfony 5.2 accepts this on PHP 8, and the annotation era quietly starts to end. Same shape as the annotation, but now it is language, not a comment. The engine parses it, static analysis sees it, a typo is a compile-time complaint instead of a route that silently does not exist. Doctrine and the validator are heading the same way. Everything that lived in docblocks will move over the next year or two. ...

December 18, 2020 · 2 min · Murat Useinov