Parallel query in PostgreSQL 9.6

38 seconds to 11. Same query, same forty million rows, no index added, no SQL changed. The only difference is PostgreSQL 9.6, released yesterday, and one setting. The query is a typical report: count and sum over an events table, grouped by day, three months of data. On 9.5 the plan is one process grinding through the table. On 9.6 with parallelism enabled: Finalize HashAggregate -> Gather Workers Planned: 4 -> Partial HashAggregate -> Parallel Seq Scan on events Four workers scan their own chunks, aggregate partially, the leader merges. Just more hands. For years we optimized SQL as if the database had exactly one worker per query. That assumption expired yesterday. ...

September 30, 2016 · 2 min · Murat Useinov

Laravel 5.3: the app is not a website anymore

Mail::send() inside the checkout controller. A second copy inside the API controller. Then someone adds the chat webhook to one copy and forgets the other. Every project has this code. I have written it more than once. Laravel 5.3 came out this week, and the two big pieces, Notifications and Passport, both point the same way. The application is no longer a thing that renders HTML. It is a core that talks to browsers, mobile clients and third parties, and HTML is one of the outputs. ...

August 26, 2016 · 2 min · Murat Useinov

Column order in a composite index

Fifty rows. Three indexes on the table, one per column. Still slow. The query is the standard one from any multi-tenant application: SELECT * FROM orders WHERE tenant_id = ? AND status = ? ORDER BY created_at DESC LIMIT 50; “Add an index” is the typical reaction, and one index on each column is the typical result. The number of indexes is not the point. The order of columns inside one index is. ...

July 4, 2016 · 2 min · Murat Useinov

Doctrine and the 100 000 row import

Row sixty thousand. Allowed memory size exhausted. The import script on one project died there, and the code was the obvious loop: read a row, persist() an entity, next row, flush() at the end. On a hundred test rows it worked perfectly. It was my loop. The reason is the Unit of Work. Doctrine keeps every managed entity in memory, plus a snapshot of its original data for change tracking. Persist a hundred thousand entities and you hold a hundred thousand objects twice. This is not a bug. It is the price of the ORM’s main feature, and on a normal web request the price is invisible because the request dies young. ...

June 19, 2016 · 2 min · Murat Useinov

Symfony by the piece

A folder of cron scripts, each one a .php file with hand-parsed $argv. That is what CLI looked like on our legacy Kohana application until last month. One line fixed it: composer require symfony/console Now each script is a small Command class with named options, --help for free, and exit codes that cron can actually check. The framework around it did not notice anything. Console does not care who serves your HTTP. ...

May 10, 2016 · 2 min · Murat Useinov

Docker instead of a setup README

Two days. That is how long the new colleague spent last month getting one project to run on his laptop. The README is two pages: PHP with a specific set of extensions, Nginx, MySQL, Redis, and a paragraph that starts with “on OS X it is a bit different”. Every laptop in the team is a slightly different snowflake. So I finally tried Docker for local development. The compose file: ...

April 13, 2016 · 2 min · Murat Useinov

Freezing the Kohana layer

Kohana 3.3, on a project that earns money every day. The framework is effectively finished. The repository barely moves, the community left years ago. Nobody will approve a rewrite, and I have stopped asking. This is a normal situation and it deserves a better plan than “someday we migrate”. The plan we settled on is a freeze. Pin the exact framework version and vendor it. Not “3.3.*”, the exact commit. The build must be reproducible in five years, when the original download link is dead. ...

March 4, 2016 · 2 min · Murat Useinov

Middleware groups in Laravel 5.2

An IP whitelist check placed after auth. That was the whole bug. Every scanner bot on the internet was going through session start and a user lookup just to be told to go away. The fix was to move one line up in the Kernel. Laravel 5.2 makes that line easier to see. Middleware groups, and the request pipeline is finally written down in one place: protected $middlewareGroups = [ 'web' => [ \App\Http\Middleware\EncryptCookies::class, \Illuminate\Session\Middleware\StartSession::class, \App\Http\Middleware\VerifyCsrfToken::class, ], 'api' => [ 'throttle:60,1', 'bindings', ], ]; Before 5.2, sessions and CSRF were global middleware. They ran for everything, including API routes that have no use for cookies. Now web and api are two separate pipelines, and you can read each one top to bottom. ...

February 8, 2016 · 2 min · Murat Useinov

Upsert in PostgreSQL 9.5

$row = $db->fetchOne('SELECT id FROM counters WHERE name = ?', array($name)); if ($row) { $db->execute('UPDATE counters SET value = value + 1 WHERE name = ?', array($name)); } else { $db->execute('INSERT INTO counters (name, value) VALUES (?, 1)', array($name)); } Works on the laptop. In production two requests arrive in the same millisecond. Both SELECT, both see nothing, both INSERT. One dies with a duplicate key error. Or worse, there is no unique constraint, and now you have two rows and a bug report you cannot reproduce. The window between SELECT and INSERT is tiny, so it fires once a week, always for someone else. ...

January 12, 2016 · 2 min · Murat Useinov

PHP 7 measured on real code

One endpoint. The heaviest catalog page of one project, same data, same opcache settings, PHP 5.6 against 7.0. That is the only benchmark I trust. 7.0 has been out for three weeks, everyone has seen the hello-world numbers, and I do not believe hello-world numbers on principle. Response time dropped about 40 percent. Memory per request, more than half. No code changes. I ran it again because I did not believe the first numbers either. The engine rewrite is real: smaller internal structures, cheaper function calls, and typical framework code is exactly that, thousands of small calls and arrays. ...

December 26, 2015 · 2 min · Murat Useinov