Multi-stage builds for PHP images

720 MB. That was our production PHP image, and most of it was composer, git, unzip, build headers and a pile of apt cache. None of it runs in production. It was there because composer install needs it, and a Dockerfile was one linear script. People worked around this with two Dockerfiles and a shell script gluing them. Ugly, and everyone’s glue was different. Docker 17.05 brought multi-stage builds. The first Docker feature in a while that fixes a problem I actually had. Now it is one file: ...

July 7, 2017 · 2 min · Murat Useinov

Keyset pagination instead of OFFSET

LIMIT 50 OFFSET 500000. Page 10001 of an events table, from a paginator someone wrote in an afternoon. Postgres has no shortcut to row 500000. It walks the index through half a million entries, fetches them, throws them away, and returns fifty. Page one is fast. Page ten thousand is slow, and every page after it is slower. Run EXPLAIN ANALYZE on both: the plans are identical, the numbers are not. ...

June 19, 2017 · 2 min · Murat Useinov

Private services: the container steps back

$container->get('app.mailer') stopped working this week. Symfony 3.3 is out, and in the new configuration services are private by default. The container refuses the call. Everyone is writing about autowiring. This is the change I like more. That call was always a smell. A class that pulls dependencies from the container by string id is a class with secrets. Its constructor says nothing, and to test it you boot half the framework, because it may ask for anything at any moment. ...

May 30, 2017 · 2 min · Murat Useinov

ALTER TABLE is part of the deploy

Forty million rows, three application instances behind a balancer, one migration that renames a column. The deploy script runs migrations first, then rolls the code instance by instance. For a few minutes old code was writing to a column that no longer existed. The error rate graph looked like a wall. I had approved that migration. It was one line. Migration tools trained us to think a schema change is a code change. Write a file, run it, done. A schema change on a big table takes time, takes locks, and while it runs the previous version of the application is still serving traffic. Both versions must survive both schemas. That is the whole discipline, and it has a name: expand and contract. ...

April 24, 2017 · 2 min · Murat Useinov

Autowiring is coming to Symfony 3.3

Our services file repeats every class name three times. Once as the id, once as the class, and again in the arguments of every service that needs it. Add a constructor parameter, edit YAML, clear cache, repeat. Symfony 3.3 lands in May and promises to delete most of that file: services: _defaults: autowire: true autoconfigure: true AppBundle\: resource: '../../src/AppBundle/*' Autowiring and autoconfiguration on by default, the whole source folder registered in five lines. I have mixed feelings. Sorted, they look like this. ...

March 8, 2017 · 2 min · Murat Useinov

Cache stampede: the hot key problem

Cache hit rate 98 percent. The database still fell over. Both facts are true, and the second one does not care about the first. What happened. One key holds the result of a heavy query for the main page. TTL one minute. At some second the key expires. In that second a few hundred requests all get a miss, and all of them go to recompute the same heavy query. Redis is fine. MySQL is not. This is a cache stampede, and hit rate will not warn you, because hit rate is an average. The stampede lives in the worst second. ...

February 7, 2017 · 2 min · Murat Useinov

Laravel Dusk and what browser tests are for

Laravel 5.4 came out on Tuesday, and the first thing I installed was Dusk. Browser tests without a Selenium server: it talks to ChromeDriver directly, the API is fluent, and a failed test leaves a screenshot behind. My first test was the login path: $this->browse(function ($browser) { $browser->visit('/login') ->type('email', 'user@example.com') ->type('password', 'secret') ->press('Log in') ->assertPathIs('/home'); }); It passed on the third run. The first run failed because the button on the test box had different text. The second failed because a JS animation was slower than the wait. This is the normal life of a browser test. It sees the real application, and the real application moves. ...

January 26, 2017 · 2 min · Murat Useinov

PHP 7.1: nullable is honest

// returns User or false, see wiki public function findByEmail($email) Every legacy codebase has this method. The comment is the type system. Half the callers check for false, some check for null because a sister method returns null, one caller checks nothing and works by luck. PHP 7.1 came out last week. Nullable types, void, iterable, multi-catch. Small features, but together they continue the direction 7.0 started: less implicit agreement, more signature. The one I care about is ?Type: ...

December 9, 2016 · 2 min · Murat Useinov

Order status is a graph

if ($order->status == 'paid') in the shipping module. Another one in the refund module. A slightly different one in the admin panel. Then someone cancels a shipped order from an admin button, because the button just writes the string, and the warehouse learns about it a week later. Symfony 3.2 is out this week and ships a Workflow component. Finally a framework says out loud what every order table has been whispering for years: status is a state machine, not a string column you assign wherever convenient. ...

November 30, 2016 · 2 min · Murat Useinov

The queue will run your job twice

The job called the payment provider. The call succeeded. Then the worker timed out before it marked the job done. The queue did what queues do: it retried. The provider did what it was asked: it charged again. The customer did what customers do and wrote an angry email. Nobody made a mistake here. The queue promises at-least-once delivery, and “at least” is written in the contract. Network partitions, worker crashes, deploy restarts. Sooner or later every job runs twice, and the jobs that hurt are exactly the ones with external effects: payments, emails, webhooks, API calls. ...

October 3, 2016 · 2 min · Murat Useinov