Posts on Murat Useinov

A single server and a database is enough here

Tue, 01 Sep 2026 19:40:00 +0300

Ten years ago a system design conversation was short. A server, a monolith, Postgres, maybe a load balancer in front. For most products that was the entire architecture, and it held.

Now the same conversation opens with API gateways and doesn’t stop: Kubernetes, service meshes, Kafka, Redis, three databases because each one is good at one thing, gRPC, GraphQL, Elasticsearch, Terraform, Prometheus, tracing, sharding, replication, autoscaling.

The awkward part is that none of it is junk. Every one of those tools exists because someone hit a real wall: more users, more data, more teams, more regions, more nines of uptime.

Reading PHP 8.6 RFCs as constraints

Mon, 17 Aug 2026 21:05:00 +0300

Accepted, declined, discussion drama. That is how most posts about PHP 8.6 RFCs read, and the vote cycle is in full swing this month. I read RFCs for a different reason. Not “what do I get”, but “what will my current code look like next year”.

One RFC is a feature. Ten RFCs are a direction. When several push the same way, toward stricter types, explicit declarations, deprecating some old freedom, the direction tells you which of your habits are becoming legacy while you still type them. Syntax sugar is the least interesting part. Deprecations are the most interesting, because every deprecation is a migration with a countdown attached.

Testing PostgreSQL 19 beta without guessing

Wed, 08 Jul 2026 21:45:00 +0300

A production-like dump, a separate PostgreSQL 19 beta instance, one evening plus machine time. That is the whole method, and it is the only thing a database beta is for: finding out where the new version changes the behavior of your application while the change is still a bug report and not your incident.

Restore the dump first. Data size matters. A plan that is fine on a thousand rows goes bad on fifty million.

When version numbers stop talking

Thu, 11 Jun 2026 22:30:00 +0300

Major means breakage possible, minor means safe. For twenty years a MySQL version number carried that promise, and you could plan by reading it. With the move to calendar-style releases the number tells you one thing: when it was built. The promise moved elsewhere, and you have to know where to look.

The signals now are the channel and the dates. Is this an LTS or an innovation release. When does support end. What do the compatibility notes actually say, because “no major version” does not mean “no behavior change”. Optimizer changes ship in any release. Your query plans do not read version numbers.

A worker is not a controller

Mon, 25 May 2026 21:25:00 +0300

All the logic inside execute(), exit code always zero, no lifecycle thinking at all. A controller without a request. I still see this console command in every second codebase. Symfony stopped being only an HTTP framework a long time ago, but habits are slower than releases, and with 8.1 giving console and workers first-class attention the excuse is gone.

An HTTP request lives for milliseconds. The kernel builds services, handles, throws everything away. A worker lives for hours. Same container, very different lifecycle. Every service that quietly keeps state, an in-memory cache, an accumulating buffer, an entity manager full of tracked objects, is invisible in HTTP and becomes a memory leak in a worker. If your consumer needs a nightly restart by cron, this is where the night went.

MySQL 9.7 LTS is an operational choice

Fri, 10 Apr 2026 20:10:00 +0300

An SQL mode set in 2019 that nobody remembers why. An authentication plugin the driver happened to support. A charset and collation nobody chose. That is what a MySQL upgrade is made of. MySQL 9.7 LTS is out, so I am making the list again.

An LTS release of a database is a date more than a feature list. The date until which somebody else worries about patches, and the date after which the worry is yours. For a working PHP project the question is not “what is new”. It is “what did we assume”.

An LLM call is just IO

Wed, 11 Mar 2026 20:15:00 +0300

One method, response back. Laravel 13 makes calling a model feel like calling a mailer, and the demo level is now twenty minutes of work. Which is exactly when architecture starts to matter, because everybody will ship the demo.

An LLM call is IO. Slow IO, expensive IO, IO that sometimes fails and sometimes lies. We know how to handle IO like that. We forget, because the response text looks smart and the API looks native to the framework.

Dependency upgrades without a big bang

Fri, 27 Feb 2026 19:45:00 +0300

A composer.lock that had not changed in three years. A colleague showed it to me last week, not as a confession, as a normal fact about the project. Nobody decided this. Every single upgrade looked small and risky, so everybody postponed it. Three years of postponed small risks equals one migration project with a budget and a name.

The alternative is dull and it works. Upgrades as normal operations, like backups.

PHP 8.5 one month later

Mon, 19 Jan 2026 21:50:00 +0300

Last week of December, low traffic, good test suite. That is when we moved one project to PHP 8.5. Not brave, convenient. A month later I can sort the release into two piles.

Pile one is the demo features. Every release has them. They look great in a tweet-sized example, then you search the codebase and find two places to use them. Fine. Not why you upgrade.

Pile two is the boring changes. Less boilerplate in places you touch every day. Types that say what a function really returns. Small things that make PHPStan complain earlier instead of production complaining later. This pile is why you upgrade, and you only see it after some weeks, never in the release notes.

From Kohana to PHP 8.5

Thu, 11 Dec 2025 21:40:00 +0300

A Kohana ORM, an N+1 on a catalog page, and a blog post about it. That was November 2014. It is December 2025, the code around me runs PHP 8.5, and I went back and read that first post.

The surface changed completely. A 2014 controller was a fat method: it read the request, validated by hand, called the ORM, formatted the response, sometimes sent an email on the way out. Types lived in docblocks and in hope. Deploy was files over FTP and a prayer. Today the same feature is a typed handler with constructor injection, validation at the boundary, a use case object, a message on a queue for the email, a trace id through the whole thing. The engine checks what a comment used to promise. I do not miss the old way.

The November upgrade train

Fri, 21 Nov 2025 22:30:00 +0300

PHP 8.5 came out yesterday. Symfony 7.4 LTS and 8.0 land at the end of the month, same as every year. November is upgrade month now. First the cargo, then the timetable.

From 8.5 I care about three things. The pipe operator makes transformation chains read left to right:

$slug = $title
    |> trim(...)
    |> strtolower(...)
    |> (fn ($s) => preg_replace('/\s+/', '-', $s));

The new URI extension puts standards-correct URL parsing in core. parse_url had opinions instead of a specification, I will not miss it. And #[\NoDiscard] lets a method declare that ignoring its return value is a bug. Made for result-style APIs, where a dropped return is a swallowed error.

Coding agents meet legacy PHP

Fri, 10 Oct 2025 21:30:00 +0300

A strange-looking condition in a 2015 module, “fixed” by an agent. All tests green, static analysis clean, and the condition was a business rule with history. My review caught it. Barely.

That came out of an experiment I ran this month: a coding agent on one legacy PHP module, since agents can do multi-file changes now and every old codebase suddenly looks like a target.

The setup matters more than the model. I did not ask “modernize this”. I gave a narrow migration goal on one bounded module: replace a deprecated API, add proper namespaces, keep behavior. The same shape of task I would give to Rector. The difference is that the agent also handles the irregular cases Rector cannot express, the places where people wrote creative code in 2015.

PostgreSQL 18, the upgrade notes

Fri, 26 Sep 2025 22:45:00 +0300

PostgreSQL 18 went GA yesterday. I ran the beta in July. Now it is not a preview, it is a ticket, and these are the notes from planning it.

The quiet feature that matters most for the upgrade itself: pg_upgrade now carries planner statistics over. Before, the minutes or hours after switching versions were the scary part. Fresh cluster, empty statistics, the planner guessing, production limping until ANALYZE finished. That cliff is mostly gone. This alone changes how a major upgrade feels at 2 a.m.

vendor/autoload.php is not free

Wed, 20 Aug 2025 20:10:00 +0300

46 milliseconds. That is how long the bootstrap of one large application took before the framework even started, in a trace I pulled this week. A good share of it was autoloading.

require vendor/autoload.php looks like a constant of nature. With plain PSR-4 rules it is a loop: every class load walks the prefixes and asks the filesystem whether a file exists. A few thousand classes on a cold request, and the loop becomes a number you can see in a flame graph.

A weekend with PostgreSQL 18 beta

Wed, 02 Jul 2025 18:50:00 +0300

A staging copy of one database, restored over the weekend onto PostgreSQL 18 beta. GA is expected in autumn. Two things I wanted to touch: asynchronous I/O and the built-in uuidv7().

Start with uuidv7, because it is about a mistake many of us already made, me included. Random UUIDv4 primary keys scatter inserts across the whole index. Every insert lands on a random page, the working set is the entire index, buffers churn. UUIDv7 is time-ordered: new rows go to the same few pages, like a sequence, but still globally unique.

One TTL does not fit the whole page

Mon, 16 Jun 2025 21:10:00 +0300

“Hello, Ivan” in the header. That is what broke the full-page cache on one project: either the whole page became uncacheable, or Ivan’s greeting was served to strangers. We managed both, in the same month.

Full-page cache behind a reverse proxy is a great first step. It dies on the first personalized element.

The way out is to stop thinking of the page as one object. It is regions with different speeds. The header with the user name changes per user. Navigation changes per deploy. The product list changes every few minutes. Prices and stock change every few seconds. Give all of that one TTL and you must pick the minimum, which means almost no cache at all.

Symfony 7.3 and the code I get to delete

Fri, 30 May 2025 15:25:00 +0300

Symfony 7.3 was released this week. I opened the release notes with a text file next to them. The file is a list of things to delete.

Every project older than three years carries a layer of custom helpers that were reasonable when written. The framework moves, the helpers stay. Nobody reviews them, because they work.

Two examples from one project.

A base console command class with sugar for arguments and options, written around 2021. Symfony 7.3 has invokable commands, arguments come as parameters with attributes:

Native lazy objects, the ORM view

Wed, 09 Apr 2025 22:35:00 +0300

Open the Doctrine proxy folder of any project older than a year. A generated subclass for every entity, getters overridden, one file per class in the cache dir. It worked for fifteen years, and the seams always showed: final classes, public typed properties, instanceof surprises, a folder of code you pretend not to see.

PHP 8.4 lazy objects were designed with exactly this in mind. The native way needs no generated class:

Observability has a budget

Mon, 10 Mar 2025 22:30:00 +0300

Last year we turned tracing on everywhere. This year the telemetry invoice is a line item a manager asks about by name. The dashboards did not get more useful. More data, same confusion at 2 a.m.

Time to design instead of collect.

The design starts from the questions I actually ask when the pager goes off. In practice there are four: is the latency of key endpoints normal, did the error rate jump, how far behind are the queues, is the database saturated. That is the core set. Metrics for these must be cheap, always on, with alerts. Everything else can be sampled traces I pull up on demand.

Laravel 12 is a boring release

Wed, 26 Feb 2025 23:30:00 +0300

Laravel 12 came out on Monday. Tuesday evening I upgraded one mid-size project. The whole diff is composer constraints and the lock file. Tests green, deployed before midnight.

People in chats are disappointed. Where are the big features. A major that mostly refreshes dependencies and starter kits, is it even a major.

I remember 4.2 to 5.0. New folder structure, new config system, half of the packages dead on arrival. We planned it as a separate project, because it was one. Skip a major, and the next one cost double.

PHP 8.4 after the hype

Wed, 15 Jan 2025 15:50:00 +0300

A private field, a getter, a setter with one line of normalization. Three members for one idea, repeated through every entity folder I have ever opened. PHP 8.4 is two months old, the conference demos are done, and the honest question is which features earn a place in a working codebase.

My filter is simple. A feature is good when it deletes code.

Property hooks pass:

class User
{
    public string $email {
        set => strtolower(trim($value));
    }
}

The class is shorter and the normalization cannot be bypassed. But keep hooks boring. A hook that talks to the database is a magic getter from 2008 in new syntax.

The fastest request never reaches PHP

Tue, 03 Dec 2024 12:10:00 +0300

One header, and the FPM load graph fell off a cliff.

Cache-Control: public, s-maxage=60

A public catalog page on one project renders the same HTML for every anonymous visitor. Same queries, same JSON from the search service, same template, thousands of times per hour. We profiled it, we tuned it, and only then asked the obvious question: why is PHP involved in the second request at all.

HTTP had the answer before my career started. public says a shared cache may store the response. s-maxage gives the CDN or reverse proxy its own lifetime, separate from browser max-age. With Varnish in front, the request path splits in two. MISS: full stack, FPM, database, sixty milliseconds. HIT: the proxy answers from memory, the PHP process never starts, the database never hears about it. Nothing in the application got faster. There was simply less application running.

Lazy objects move into the engine

Sat, 23 Nov 2024 12:50:00 +0300

Fifty ghosts initialized in a loop are still fifty one queries.

PHP 8.4 is out since Thursday. Property hooks get the headlines, but the feature I keep coming back to is native lazy objects. It legalizes a trick ORMs have done with generated code for fifteen years.

The trick: you load an order, $order->customer should be a Customer, and you do not want a query until someone actually touches it. Doctrine generates a proxy class at build time, a subclass that overrides every method with “initialize first, then call parent”. It works. It is also a pile of magic: generated files, edge cases with final classes and private properties, strange things in var_dump.

Property hooks, before the hype settles

Mon, 14 Oct 2024 20:50:00 +0300

Six lines of getter and setter ceremony around one strtolower. PHP 8.4 is in release candidates, and property hooks delete exactly that:

class Customer
{
    public string $email {
        set => strtolower(trim($value));
    }
}

Assignment stays assignment, $customer->email = $input, and normalization happens on the way in. For this exact case, one value, no dependencies, I am convinced. The property remains a property.

Where I get careful is everything past normalization. A setter method is ugly, but it announces itself. setEmail() in a call stack tells you code ran. $obj->email = $x looks free, and with hooks it is not. Put validation that throws into a hook, and every plain assignment becomes a possible exception, invisible at the call site. Put a side effect in, and you have built magic the next person discovers through a debugger at midnight. We spent years removing __get and __set from codebases for this exact crime. Hooks are their respectable cousins, better typed, analyzable, and the temptation is identical.

PostgreSQL 17 and why VACUUM is an API concern

Fri, 27 Sep 2024 15:20:00 +0300

A reporting transaction open for hours, and n_dead_tup on the hot table climbing all day while autovacuum ran non-stop and changed nothing.

That was one project. PostgreSQL 17 came out yesterday, and the headline work is exactly this plumbing: vacuum got a new memory structure for tracking dead tuples, far less memory and no longer capped the old way, plus better throughput for high-concurrency writes and bulk loads. Maintenance internals. Which is why backend people should care.

Pick the library with a profiler, not a chart

Wed, 07 Aug 2024 20:50:00 +0300

Two hundred thousand ops per second against eighty. That was the bar chart, and the team was ready to pick a serializer from it. I asked for one day.

The day went like this.

Take a representative workload. Not a synthetic three-field object, but a hundred real payloads from production logs, with the nested structures, the nullable mess, the one field that is sometimes a list and sometimes a map. Anonymize them, save as fixtures. This step is the whole method. The bar chart was measured on someone else’s data with someone else’s configuration, and a serializer is exactly the kind of code whose cost depends on the shape of the input.

WebSockets change your operational model

Wed, 03 Jul 2024 23:10:00 +0300

Every client reconnects at once. That is what a deploy means now on one project of mine, since Reverb made WebSockets a first-party Laravel feature this spring and order updates go to the browser instead of polling. The code was the easy part.

A classic PHP app is stateless between requests. An FPM worker takes a request, answers, forgets. Capacity planning is requests per second. A WebSocket server is the opposite animal: thousands of open connections that mostly do nothing, but each one holds memory and a file descriptor, and each one is state that dies with the process. Your quiet realtime feature has a thundering herd built in. Plan for reconnect storms, raise descriptor limits, and make the client reconnect with jitter, never on a fixed timer.

A faster sprintf will not fix your p99

Tue, 18 Jun 2024 23:15:00 +0300

Three times faster. That is the number in the benchmark posts about PHP 8.4, where simple sprintf() calls with basic placeholders get turned into plain string operations at compile time. The function call disappears.

for ($i = 0; $i < 1000000; $i++) {
    $s = sprintf('%s-%d', $prefix, $i);
}

I like this work. I do not like how people read it.

A microbenchmark measures one function in a loop, alone, with a warm cache and nothing else to do. Now profile a real request from the same application. In every backend I have profiled the picture has the same shape: most of the wall time is waiting. SQL queries, Redis round trips, an HTTP call to some partner API. The CPU spent inside PHP is a modest slice, and sprintf inside that slice is a rounding error. Three times faster rounding error is still a rounding error.

Symfony 7.1 and the value of boring releases

Fri, 31 May 2024 12:40:00 +0300

composer update, the changelog over coffee, two deprecation notices fixed, done before lunch. That was Symfony 7.1 on one project this morning. Nothing in the release will change your architecture, and I mean this as a compliment.

There are nice small things. An attribute that maps an uploaded file straight into a controller argument, one more piece of request plumbing gone. A new experimental component for type introspection that libraries will quietly benefit from. A layer of deprecations preparing the next major. That is the whole show.

MySQL 8.4 gives us a place to stand

Tue, 30 Apr 2024 21:05:00 +0300

Half the industry is sitting on MySQL 5.7 right now, unsupported since last autumn, and calling it stability.

MySQL 8.4 came out today, and the important word in the announcement is not a feature. It is LTS. First long-term release of the new model: innovation releases every quarter for the brave, an LTS every couple of years for production, bugfixes only after that.

People underestimate how much pain the old model caused. 8.0 was a rolling target. A “minor” 8.0.x could change optimizer behavior or deprecate something you used. Staying current meant re-testing, staying behind meant missing security fixes.

Laravel 11 removes files, not decisions

Mon, 25 Mar 2024 23:05:00 +0300

The diff of a Laravel 11 upgrade on one mid-size API this week is mostly deletions. Almost empty app/, configuration in one fluent bootstrap file, the slim skeleton I wrote about in January now real. It feels great.

Now look at what survived untouched. The use case class that wraps an order state change in a transaction. The listener that must fire only after commit, because it queues an email about a row that must exist. The cache invalidation that follows every write to a heavily read table. The decision that validation lives in a request class and business rules one layer deeper. None of this is skeleton. None of it got smaller.

One trace through HTTP, queue and database

Thu, 08 Feb 2024 19:50:00 +0300

A request creates a job. The job calls an external API and writes to the database. Something in this chain is slow, and the logs tell four disconnected stories.

Grepping a request id across services is our folklore. OpenTelemetry finally makes the boring standard version practical in PHP.

The core is small. One trace id for the whole causal chain, a span id for every operation, and one rule: pass the context along. Over HTTP it is a single header:

The shrinking Laravel skeleton

Sat, 06 Jan 2024 22:25:00 +0300

No app/Http/Kernel.php. That is the first thing you notice in the Laravel 11 preview, and the first thing every comment thread asks: where did my middleware go.

The skeleton is the loudest change this quarter, louder than any feature. An app/ directory that is almost empty, no folder of middleware stubs, a pile of service providers collapsed into one, config files trimmed.

The middleware did not go anywhere. It moved into the framework. The mechanism exists exactly as before, only the file does not, and bootstrap/app.php gets a small fluent API to reconfigure the stack when you actually need to.

AI assistants do not know your production

Sat, 23 Dec 2023 10:40:00 +0300

A DTO with a dozen fields, a framework migration, a slow query incident. Three tasks from the last few months, all done with Copilot and ChatGPT open. Here is where the line runs for me.

Task one, boilerplate. The DTO, its validation rules, a test with fixtures. The assistant writes this faster than I type, and the result needs only a quick read. Same for converting an array-shaped legacy structure into typed classes. This is honest time saved, maybe an hour a day. The code is the kind where being generic is correct.

PHP 8.3 and Symfony 7 in one month

Thu, 30 Nov 2023 10:45:00 +0300

public const int TIMEOUT_SECONDS = 30; One week old, and the one thing I waited for from PHP 8.3.

Busy November. 8.3 released last week, and yesterday Symfony shipped 6.4 LTS and 7.0 on the same day. Time to plan, not to jump.

Typed class constants are a small thing, but constants were the last untyped island in a class, and a child class could silently redefine one to a different type. Also json_validate(), which checks JSON syntax without building the whole tree in memory. Everyone had a helper that did json_decode and threw the result away. Now the helper can retire.

FrankenPHP: the server absorbs PHP

Fri, 27 Oct 2023 18:25:00 +0300

One binary, my code, a Caddyfile. That is the whole deployment of a small internal service I moved to FrankenPHP in staging this week.

The classic PHP deployment is a couple: Nginx in front, PHP-FPM behind, FastCGI between them. Two configs, two processes to supervise, one socket to misconfigure. It works, we all know it by heart, and knowing it by heart is the only reason we call it simple.

PostgreSQL 16, read through one workload

Sat, 16 Sep 2023 20:25:00 +0300

A nightly import, a few hundred million rows through COPY, then reporting queries in the morning. That is the only workload I care about, so that is how I read the PostgreSQL 16 release notes: with one pipeline in mind and everything else skipped.

Three items survived the filter.

COPY got faster, noticeably when several loads run at once. Our import is exactly this shape, parallel COPY streams into partitioned tables. I will take the free speed. But the old lesson stays: if your import still does row-by-row INSERT from PHP, no Postgres release will save you. Batch into COPY first, then talk about versions.

UUID primary keys and the B-tree bill

Thu, 03 Aug 2023 21:35:00 +0300

The same table, the same rows, about 1.5 times the disk. That was the measurement after someone made UUIDv4 the primary key in InnoDB, and that is before counting secondary indexes.

Every new API wants UUIDs. Distributed generation, no coordination, nothing leaks about row counts. All true. Then the database quietly starts paying.

InnoDB clusters the table by primary key. Auto-increment inserts always land in the rightmost page, warm and predictable. Random UUIDs land anywhere in the tree: page splits everywhere, half-empty pages, and the set of hot pages becomes the whole index. Secondary indexes store the primary key in every entry, so each of them inherits the full 16 bytes too. In PostgreSQL the heap does not cluster, so the pain is smaller, but index bloat and lost locality are still there.

MySQL now has two speeds

Thu, 20 Jul 2023 22:35:00 +0300

MySQL 8.1 came out this week, and its support window is measured in months. That is by design. Oracle changed the release model: two tracks now. Innovation releases every quarter or so, 8.1 is the first, short support, new features fast. And an LTS line with years of support, promised for next year. 8.0 stays on bugfixes until its end of life in 2026.

For years the question was “when do we finally move to 8.0”. Now the question has a second dimension: which track are you even on. That is a product decision, and it is worth making explicitly instead of by inertia.

Outbox: events that do not lie

Wed, 07 Jun 2023 20:40:00 +0300

An email confirming an order that does not exist. A colleague showed me this one last week, together with the mirror bug: an order that exists and never got its email.

The code was the obvious kind. Commit the Doctrine transaction, then dispatch to the AMQP transport. Two systems, two writes, no shared transaction. Publish after commit and the process can die between them, event lost. Publish inside the transaction and the broker gets an event for data that may still roll back. Both orders are wrong, and under real load this fires weekly.

Scheduler component, or cron as messages

Wed, 31 May 2023 20:20:00 +0300

crontab -l under a login nobody remembers creating. That is where the real business logic of one server lived: a nightly cleanup, an export, a retry script.

Symfony 6.3 came out yesterday with a Scheduler component. Periodic tasks defined in PHP, executed as Messenger messages. Experimental, but the idea deserves a note.

#[AsSchedule('default')]
final class MainSchedule implements ScheduleProviderInterface
{
    public function getSchedule(): Schedule
    {
        return (new Schedule())->add(
            RecurringMessage::every('10 minutes', new CleanupExpiredCarts()),
        );
    }
}

Then messenger:consume scheduler_default, and a worker fires the messages.

Persistent PHP workers need discipline

Mon, 03 Apr 2023 15:45:00 +0300

A static array with the comment “cache, cleared per request”. It went to production on RoadRunner last month, and the memory graph turned into a staircase.

PHP had one superpower nobody advertised: the request died. Every leak, every forgotten static, every open transaction was erased when the process shut down. Shared-nothing was not architecture. It was amnesia, and amnesia forgave us everything.

RoadRunner and Swoole take it away. The worker lives for thousands of requests, and the class of bugs changes on the first day. That “per request” cache became a cache per worker lifetime, and the supervisor killed the process when it ran out of memory. A logger kept the request id in a property, so entries from user B carried the id of user A. And the best one: an exception in the middle of a Doctrine transaction left the connection with the transaction open, and the next request on that worker silently joined it.

DI attributes: config moves into the class

Mon, 27 Mar 2023 23:10:00 +0300

Three lines of YAML in a file two directories away, for one integer that one service reads. That was the first thing I converted when I started moving a project’s service config from YAML into attributes, to see where the line is.

public function __construct(
    #[Autowire('%env(int:IMPORT_BATCH_SIZE)%')]
    private int $batchSize,
) {}

Now the class tells you everything about itself. Same with #[TaggedIterator] for collecting all implementations of an interface, and #[AsDecorator] for wrapping a service. These are local facts. A local fact belongs next to the code it describes. When I open the class, I want to stop searching.

Laravel 10 is a quiet major

Thu, 16 Feb 2023 20:40:00 +0300

Process::run('gzip -k dump.sql'). Out of the whole Laravel 10 release this week, that is the line I will actually use.

The rest is quiet. Skeleton and framework code got native type declarations instead of docblocks, PHP 8.1 is the floor, and there is the Process facade. That is roughly the whole story. Some people are disappointed. I am not. A major release that is mostly maintenance means the framework is an adult.

Before Laravel 10: raise the baseline first

Sat, 07 Jan 2023 22:40:00 +0300

composer why-not php 8.1. One command, and it tells you who holds you back. Run it before making any plans.

Laravel 10 comes next month and requires PHP 8.1. One project I help with is on Laravel 9 and PHP 8.0. The temptation is one heroic branch: new PHP, new framework, new package versions. When that branch breaks in production, you will not know which of the three changes broke it.

PHP 8.2: stricter by default

Fri, 16 Dec 2022 21:35:00 +0300

PHP 8.2 was released on December 8. My checklist for it looks like every other December, which is the whole point of a yearly cadence.

On the surface a quiet release. Readonly classes, DNF types, standalone true, false and null types, constants in traits. Underneath, the more important half: another round of cleaning old dynamic behavior. Dynamic properties deprecated, ${var} string interpolation deprecated, utf8_encode deprecated. The language keeps trading looseness for predictability, and I keep voting for the trade.

Readonly classes for value objects

Tue, 08 Nov 2022 22:45:00 +0300

Two readonly keywords for two properties in a two-property class. That is what an immutable Money looks like in PHP 8.1:

final class Money
{
    public function __construct(
        public readonly int $amount,
        public readonly string $currency,
    ) {}
}

PHP 8.2 is at release candidate stage, and the feature I am waiting for is this one. The keyword moves up and says it once:

final readonly class Money
{
    public function __construct(
        public int $amount,
        public string $currency,
    ) {}
}

Every property is readonly, and the class refuses dynamic properties on top. Cosmetics, yes. But cosmetics that make the right thing the short thing, and that changes what people actually write.

MERGE arrives in PostgreSQL 15

Thu, 27 Oct 2022 20:20:00 +0300

A supplier price list, loaded into staging_prices, then reconciled against prices. Since PostgreSQL 15 came out two weeks ago, that is one statement:

MERGE INTO prices p
USING staging_prices s ON p.sku = s.sku
WHEN MATCHED AND s.price IS NULL THEN
    DELETE
WHEN MATCHED AND p.price <> s.price THEN
    UPDATE SET price = s.price, updated_at = now()
WHEN NOT MATCHED THEN
    INSERT (sku, price) VALUES (s.sku, s.price);

Before 15 this was three statements in a transaction, or a stored procedure, or a loop in PHP. Now it is one statement that says what it does. Update changed rows, delete withdrawn ones, insert new ones. People coming from Oracle and SQL Server waited a decade for this. The conditional WHEN MATCHED AND ... branches are the real value. ON CONFLICT cannot express “delete when the source says so” at all.

Dynamic properties are leaving

Sat, 03 Sep 2022 20:30:00 +0300

$invoice->totall = 250; One extra letter. No error, no warning. A second property appears, the real total stays null, and the bug surfaces three screens later as “total is empty sometimes”.

I have hunted this exact bug. Twice. Both times it took hours, because the write looks correct and the read looks correct. They just disagree on one letter.

PHP 8.2 lands in December, and the deprecation that will touch the most code is this one. Writing to an undeclared property becomes deprecated. In some future major it becomes an error. For fifteen years it was legal:

Fast query, slow endpoint

Tue, 23 Aug 2022 15:10:00 +0300

The profiler said the query takes 20 ms. The export endpoint took 900 ms and a quarter of a gigabyte of memory. The missing 880 ms was Doctrine turning five thousand rows into five thousand entities.

Hydration is not free. For every row the ORM builds an object, fills properties through reflection, registers it in the unit of work, creates proxies for relations. Per row it is nothing. Times five thousand, it is the endpoint. The profiler shows SQL because SQL is easy to show.

JSONB or a normal table

Sat, 16 Jul 2022 10:05:00 +0300

Audit snapshots, product attributes, and a document with two hot keys. Three cases from one Postgres project, one question for each: column or JSONB.

I answer it by looking at queries, not at data.

Audit snapshots. When an order changes, we store the whole previous state. Nobody queries inside it. It is read as a blob, whole, rarely, by a human during an incident. Perfect JSONB. Normalizing it would mean a dozen tables for data nobody joins.

Argument resolvers and the HTTP border

Mon, 13 Jun 2022 20:35:00 +0300

Forty lines of $request->get() before the first line of real logic. That was the controller nobody wanted to touch, and the Symfony 6.1 upgrade was my excuse to open it.

An action that reads ten request fields by hand does two jobs. It translates HTTP into data, and it runs the use case. The first job is boring and repeated in every action, which is exactly why it should not be written by hand ten times. Symfony has argument resolvers for this since 3.1. People just do not use them for their own types.

Redis 7 functions, same old rule

Thu, 05 May 2022 20:35:00 +0300

FCALL update_if_newer 1 user:42 17. One round trip, a name instead of a SHA, and it survives a restart. Redis 7 came out last week, and Functions are the feature I care about. Everything EVAL scripts should have been.

The problem they solve is real. Take compare-and-update: set a value only if the stored version is older. From PHP it is three round trips and a race between them. WATCH/MULTI works but is ugly and retries under contention. A Lua script does it atomically in one trip, and that is why we all wrote Lua scripts. But scripts are anonymous blobs. The application carries the source, sends it on every deploy, and debugging starts with “which SHA is this”. Functions fix the operational part: the library lives in Redis, has a name, has a version.

Retry is a strategy, not a loop

Wed, 27 Apr 2022 20:20:00 +0300

A failed job that retries every second, one hundred times, against a payment provider that is already down. That is a small DDoS with a queue in front of it. If the provider needs five minutes to recover, hammering it for the first two achieves nothing except log volume.

Symfony Messenger has everything needed. You just have to configure it on purpose:

framework:
    messenger:
        transports:
            async:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                retry_strategy:
                    max_retries: 5
                    delay: 2000
                    multiplier: 4
        failure_transport: failed

Two seconds, eight, thirty two, and so on. Exponential backoff gives the external service room to breathe. I also add jitter in a custom retry strategy, because a hundred messages that failed together will otherwise retry together, in one synchronized wave. Same failure, five times.

Backfill in batches, not in one UPDATE

Thu, 17 Mar 2022 22:40:00 +0300

Sixty million rows in orders, a new nullable column total_cents from expand-and-contract, and a Friday afternoon. The question is how to fill it.

The naive answer is one statement. UPDATE orders SET total_cents = ROUND(total * 100). On a test database it works. On production it holds row locks on the whole table for the whole run, writes one giant chunk of WAL, and the replica falls minutes behind. On MySQL the binlog gets the same present. One project I worked on learned this on that Friday. Since then I backfill only in batches.

Laravel 9 and the boring major

Fri, 25 Feb 2022 12:50:00 +0300

One evening. That was the Laravel 9 upgrade on a mid-size project last week. Most of the diff was composer.json and a few config files.

Laravel 9 is out this month. Symfony 6 components under the hood, PHP 8.0 minimum, Flysystem 3. And from now on, one major per year. The upgrade guide is short, and the “high impact” section is shorter. This is a policy: small majors, often.

I want to defend this policy, because I remember the other kind. A project that sat on an old major for three years, because “we will upgrade when there is time”. There is never time. Then the version goes EOL, a security fix forces the jump, and you pay three years of drift in one horrible month. Packages you depend on drop support one by one, and suddenly it is not a framework upgrade anymore. It is archaeology.

Order status is an enum now

Sat, 08 Jan 2022 23:15:00 +0300

One row in orders had status 'canceled', with one l. Nobody knows how long it sat there. The enum found it on the first day.

PHP 8.1 enums are six weeks old, enough time in one project to say something practical. The first candidate was obvious: order status. For years it was a class with string constants, Order::STATUS_PAID, Order::STATUS_SHIPPED. Constants look safe, but nothing stops a function from receiving 'payed'. The parameter type was string, and string accepts everything.

Symfony 6.0 is boring, take the compliment

Thu, 09 Dec 2021 22:40:00 +0300

The deprecation counter in the profiler, three digits. That is where one project stands this week, and that number is the whole Symfony 6 migration.

Symfony shipped 5.4 and 6.0 together at the end of November, same code, same day. 4.4 and 5.0 worked exactly like this two years ago, so the story is rehearsed. 6.0 is 5.4 with the deprecated code deleted. You never migrate to a major. You migrate to the last minor, and the major is a formality.

PHP 8.1: enums, readonly and a word about Fibers

Fri, 26 Nov 2021 20:45:00 +0300

'payed'. One letter, half a day of debugging, a few years ago. The string walked straight through OrderStatus::isValid(), because someone had added it to the constants list “for compatibility”. Every project I touched has that class: a bag of string constants, an isValid() helper, and a prayer. The type system never knew these strings were special.

PHP 8.1 came out yesterday. Point release on paper, and it brings the feature I wanted since forever. Enums.

Generics we already have

Sat, 09 Oct 2021 20:05:00 +0300

A template calls getEmail() on an Order at three in the morning. The method returned array, the docblock said “array of User”, and eight months later someone put an Order in it. Everything was fine until it was not.

PHP still has no generics and will not get them soon. Meanwhile PHPStan and Psalm shipped them anyway, in docblocks. Checked at analysis, not at runtime. For everyday backend work that turns out to be most of the value.

PostgreSQL 14 and the price of a connection

Thu, 30 Sep 2021 10:05:00 +0300

A few hundred connections, mostly idle, and the whole server gets slower. That was Postgres for as long as I remember it: a process per connection, and past some count the snapshot bookkeeping starts to eat everything.

PostgreSQL 14 is released today. I had the RC on a test box for a week, so a few notes from a backend seat. The headline for me is that this internal work got a serious rewrite. On our connection-heavy profile, many workers, short queries, lots of idle time, the difference is visible without squinting. Idle connections finally cost close to what people always assumed they cost.

Redis Streams is a log, not a better queue

Fri, 20 Aug 2021 22:05:00 +0300

“Should we move the queue to Streams? They are the proper way now.” A colleague, this week, about our Redis list queue. Wrong question. Streams are not a newer version of the same thing.

A list queue is a hand-off. LPUSH on one side, BRPOP on the other, and once a consumer takes the message, it is gone. Crash after the pop and the message died with you, unless you built the pending-list dance yourself. No history, no second reader. For “send this email eventually” it is honestly enough.

Nobody closes your database connection anymore

Tue, 20 Jul 2021 23:20:00 +0300

“MySQL server has gone away”, first thing in the morning, on a worker that had no traffic all night. That was week one after we moved a project from FPM to RoadRunner.

For all these years FPM managed our database connections. Not with clever code. By dying. Process ends, connection closes, transaction rolls back, session variables reset. Nobody thought about connection lifecycle, because there was none.

Move the same code to RoadRunner or Swoole workers and the topic is back. A connection lives for hours now. Two things go wrong, and both showed up within a week.

Attributes in Symfony 5.3

Fri, 04 Jun 2021 21:05:00 +0300

#[Route('/orders/{id}')] right above the method, and nothing in config/routes. Symfony 5.3 came out last week, and with PHP 8 attributes configuration finally found its place. Routes, autowiring hints, some validation, native syntax next to the code:

#[Route('/orders/{id}', methods: ['GET'])]
public function show(int $id): Response
{
    // ...
}

I spent years defending YAML routing. The argument was separation: code is code, wiring is wiring, one file shows the whole URL map. The argument was never wrong. It lost to practice. In every real project the first thing you do with a route is jump to the controller, and the first thing you do with a controller is wonder which route hits it. Two files, one mental join, forever. Attributes remove the join. Rename a method, the metadata moves with it. Delete the class, no orphaned YAML block stays behind to confuse the next person.

Octane and state that outlives the request

Wed, 05 May 2021 22:20:00 +0300

One customer sees another customer’s cart. No crash, no error in the log. That is the bug Laravel Octane brings to an old codebase, and the benchmarks are the least interesting part of the story.

Octane is out in beta, Swoole and RoadRunner became first-class overnight. In FPM every request gets a fresh application. Boot, handle, die. The model forgives everything. Under Octane the framework boots once and workers reuse it. Bootstrap cost drops to near zero, hence the pretty numbers. But every singleton is now shared between requests, and between users.

Symfony Runtime: front controller as a callable

Fri, 16 Apr 2021 23:40:00 +0300

public/index.php, unchanged since I learned Symfony. Create the request from globals, run the kernel, send the response, terminate. Symfony 5.3 arrives next month, and the Runtime component rewrites this file. It looks like a small refactoring. It is a statement about where PHP is going.

Baked into those few lines is one big assumption: one process, one request, then we die. FPM made the assumption true for fifteen years, so nobody saw it as an assumption.

The outbox table

Tue, 23 Mar 2021 15:30:00 +0300

About once a month an order existed and nobody heard about it. The order service saved the order, then published order.created to RabbitMQ, and once a month the process died between the two. Rare enough to be mysterious. Frequent enough to ruin a weekend.

Two operations, two systems, no common transaction. Whatever order you do them in, you lose. Commit first, then publish: the order exists, the event never leaves. Publish first, then commit: the commit fails, consumers are already busy with an order that does not exist. We had the first variant in production.

TTL is not an invalidation strategy

Wed, 24 Feb 2021 22:35:00 +0300

Support ticket: a user changed his name, the site still shows the old one. The profile sits in Redis with TTL one hour. Fine, we set five minutes. A week later the hit rate drops and MySQL feels it. This seesaw has no good position. TTL is insurance, not invalidation.

The obvious upgrade is cache-aside with explicit delete. Read: try the cache, miss, load from the database, write to the cache. Write: update the row, delete the key. Looks correct. It has a race.

PHP 8 in production: dependencies first

Fri, 08 Jan 2021 23:20:00 +0300

composer why-not php 8.0. That is the first command of the migration, and it has nothing to do with syntax.

PHP 8.0 is six weeks old. Twitter is full of match expressions and constructor promotion. Meanwhile the real upgrade of a working project is a dependency problem. Your code is maybe twenty percent of what runs in production. The rest is the framework, thirty packages and a dozen extensions, and any one of them can be the blocker.

Attributes: metadata moves in with the code

Fri, 18 Dec 2020 12:35:00 +0300

#[Route('/orders/{id}', methods: ['GET'])]
public function show(int $id): Response

Symfony 5.2 accepts this on PHP 8, and the annotation era quietly starts to end. Same shape as the annotation, but now it is language, not a comment. The engine parses it, static analysis sees it, a typo is a compile-time complaint instead of a route that silently does not exist. Doctrine and the validator are heading the same way. Everything that lived in docblocks will move over the next year or two.

PHP 8.0: ignore the JIT, take the rest

Fri, 27 Nov 2020 21:25:00 +0300

JIT on, JIT off, the same Symfony endpoint. Difference within noise. Same for a Laravel endpoint. PHP 8.0 came out yesterday and this was the first thing I checked, because every headline is about the JIT.

Not a scandal. A web request spends its time in I/O, in MySQL, in framework code full of method calls the JIT cannot do much with. JIT is for tight numeric loops, and your controller has none. If you compute fractals in PHP, congratulations. The rest of us can leave it off.

Stop building running totals in PHP

Tue, 06 Oct 2020 15:40:00 +0300

Two hundred thousand rows over the wire to compute forty numbers. That was our monthly report on MySQL 5.7: fetch all transactions for the period, loop in PHP, accumulate a running balance, compare each row with the previous one, rank customers by volume. Classic 5.7 shape, because the database could not say “previous row” or “rank within group”.

We are finally moving that project to MySQL 8. Two years after GA, which by database standards is reckless haste. The first win had nothing to do with performance. We deleted PHP.

Laravel 8: the interesting parts are not on the marketing page

Fri, 11 Sep 2020 23:15:00 +0300

->refundedTwice(). That is the line from Laravel 8 I care about, and it is nowhere on the release page. Jetstream gets the screenshots. Class-based factories and the queue changes get my attention, because both are about production.

Factories used to be closures registered through a global function. Now they are classes with states:

Order::factory()
    ->paid()
    ->has(OrderItem::factory()->count(3))
    ->create();

Test data quality decides test quality. Most integration suites on our project test the happy path fifty times, because the default factory returns a fresh, valid, boring record. The bugs live somewhere else. An order refunded twice. A user registered before the migration added the column. A subscription that expired in the middle of renewal. A factory state gives such a monster a name, and once the name exists, people write tests with it. Named ugly data is the cheapest test improvement I know.

Average response time explains nothing

Tue, 04 Aug 2020 20:40:00 +0300

Average response time: 180 ms. Ticket from support: “the API is slow”. Both true. The average is a diplomat. It offends nobody and tells you nothing.

Latency is a distribution. Our 180 ms hides a p50 of 90 ms and a p99 above four seconds. One request in a hundred is terrible, and with thirty requests per page load, most users hit that unlucky one regularly. The people complaining are not imagining things. They live in the tail, and the average never visits there.

PHP 8 will retire some of our workarounds

Fri, 03 Jul 2020 22:50:00 +0300

@param int|string $id. I grepped one service this morning: dozens of those, and every one is a wish. The docblock says what the signature could not. First alpha of PHP 8 came out last week, and native union types turn the wish into a contract, checked at runtime and by tooling. Half of my phpdoc can go.

Everyone writes about the JIT. I keep thinking about the less shiny thing: how many of our daily patterns exist only to compensate for a missing language feature, and are about to become legacy.

Types are for reading, not for the compiler

Sat, 27 Jun 2020 22:20:00 +0300

OrderRepository in a constructor. Click. I am there. That is how I explore a big Symfony project now, and I noticed my main reason for types has changed. Not bug catching anymore. Navigation.

Symfony 5.1 is out, PHP 7.4 is everywhere I work, and the IDE knows every caller and every implementation. Compare with the array-passing style we all wrote for years:

public function register($data)
{
    // what is in $data? read three call sites to find out
}

versus

A deadlock is the database doing its job

Sat, 02 May 2020 20:30:00 +0300

Error 1213 in the logs, and the chat lights up: the database is broken. It is not. Two transactions locked rows in opposite order, A waits for B, B waits for A, and InnoDB did the only sane thing: picked a victim and killed it. The cycle is gone. This is a feature.

The classic shape is a money transfer. One request moves funds from account 1 to account 2, another from 2 to 1, both update the first account and then the second. Opposite order, instant cycle under load. Tests never show it, because tests do not run two of these in the same millisecond.

Waiting for three APIs, one at a time

Tue, 21 Apr 2020 21:40:00 +0300

Three external calls on one page: prices, stock, delivery estimate. Each answers in about 300 ms. The page waits a full second, because we call them one after another. PHP is synchronous, what can you do.

Turns out, something. Symfony HttpClient is lazy. request() sends and returns immediately. The waiting happens when you read the response. So start all three, read later:

$prices   = $client->request('GET', $pricesUrl);
$stock    = $client->request('GET', $stockUrl);
$delivery = $client->request('GET', $deliveryUrl);

$data = [
    'prices'   => $prices->toArray(),
    'stock'    => $stock->toArray(),
    'delivery' => $delivery->toArray(),
];

Under the hood it is curl multi. Three requests fly at once, wall time is the slowest one instead of the sum. Our second became 350 ms. No swoole, no reactphp, no async rewrite. Same boring controller.

Sanctum and choosing how much auth you need

Thu, 26 Mar 2020 23:05:00 +0300

Airlock lived for about a week. Laravel 7 shipped it this month, a trademark scare followed, and it became Sanctum. Fast rename, same idea. And the idea is good, because it names a problem people solve badly.

The problem: your own SPA needs to talk to your own API. For years the reflex was OAuth2. Install Passport, stand up an authorization server, issue JWTs to a frontend on the same domain as the backend. All that machinery to authenticate first-party code against itself. OAuth is a delegation protocol, it lets a third party act for a user. When there is no third party, you are running a passport office for your own family.

A Redis lock is a promise you cannot fully keep

Wed, 26 Feb 2020 21:45:00 +0300

Two workers, one order, processed twice. Every project gets this day. Someone writes SETNX, calls it a distributed lock, closes the ticket. I want to slow down here, because the ticket is not closed.

The small bugs first. A lock needs a TTL, or a crashed worker holds it forever. A lock needs an owner token, or worker A releases the lock of worker B:

$token = bin2hex(random_bytes(16));
$ok = $redis->set('lock:order:'.$orderId, $token, ['nx', 'ex' => 30]);

And the release must be atomic: compare the token and delete in one Lua script. Check in PHP, delete in a second command, and there is a gap. Something will land in that gap.

Cursor pagination is part of your API contract

Wed, 08 Jan 2020 21:30:00 +0300

Someone inserts a row while the client walks the pages, and the whole window shifts. Page two shows an item the client already saw on page one. Or an item falls between pages and the client never sees it. For a feed this is annoying. For an export or a sync endpoint this is a data loss bug that nobody can reproduce.

I used to think keyset pagination is a performance trick. Deep OFFSET is slow, keyset is fast, end of story. Now I think the performance part is the boring half. Offset pagination over a changing dataset lies to the client. That is the interesting half.

Static state outlives the job

Sun, 22 Dec 2019 23:50:00 +0300

One static array, from one of our projects:

class Settings
{
    private static $cache = [];

    public static function get($tenantId, $key)
    {
        if (!isset(self::$cache[$tenantId])) {
            self::$cache[$tenantId] = self::load($tenantId);
        }
        return self::$cache[$tenantId][$key] ?? null;
    }
}

A reasonable per-request cache. Under PHP-FPM it died with the process. In a queue worker it lives forever.

PHP had one great architectural feature nobody put in the manual: the process died after every request. Leak memory, cache nonsense in a static, forget to close things. Did not matter. The dying process forgave everything.

PHP 7.4: typed properties first, preloading later

Fri, 29 Nov 2019 15:25:00 +0300

Two lines of a class, and half of my docblocks can be deleted:

class Money
{
    private int $amount;
    private string $currency;
}

PHP 7.4 came out yesterday, and this is the part I start using on Monday. Those docblocks existed only to say @var int. Now the language says it, and unlike the docblock, it checks. Assign a string, get a TypeError at the assignment, not a strange bug three layers later.

There is one new state to learn. A typed property without a default is uninitialized, and reading it before the first write throws. This is the feature. “Object exists but is not filled yet” finally fails loudly instead of pretending to be null.

PostgreSQL 12 inlines your CTE

Fri, 18 Oct 2019 21:45:00 +0300

grep -rn "WITH " src/ was the first thing I ran after upgrading to PostgreSQL 12 this month. I was looking for a trick that had stopped working.

For years WITH was an optimization fence. The planner materialized the CTE first and only then ran the outer query. Everyone used this both ways. As a bug: you wrap a subquery into a CTE for readability, the planner stops pushing conditions inside, a fast query becomes a scan of half a table. As a feature: you write a CTE on purpose, to pin evaluation order and stop the planner from being creative. Half of the CTE advice on the internet is really advice about the fence.

Cleaning up before Symfony 5

Wed, 25 Sep 2019 10:25:00 +0300

Symfony 5.0 is 4.4 minus everything deprecated. That one sentence is the whole upgrade plan. Both land in November, and if the application runs on 4.4 with zero deprecation warnings, the major is a version bump. Two months is enough to get there in small chunks, with no freeze and no heroic branch that lives for six weeks.

The order that works for me.

First, get to 4.3 and make the deprecation report visible. The PHPUnit bridge prints the summary after the test run. In dev, the profiler collects the same warnings per request. You cannot clean what you do not see. The first report will be depressing. Fine. It is a todo list, not a verdict.

200 OK is not health

Wed, 07 Aug 2019 18:05:00 +0300

Route /health, returns ok, everyone happy. That is the first version of every health endpoint on the project we are moving to Kubernetes. Then somebody pastes the same path into livenessProbe and readinessProbe, and a thirty second database hiccup becomes a long evening.

The two probes ask different questions.

Readiness asks: should this pod get traffic right now. Here it is correct to check dependencies. Database unreachable, cache cold, migrations still running: answer no. Kubernetes takes the pod out of the Service, traffic goes to the others, the pod returns when the world improves. Failing readiness is cheap and reversible. A polite “not now”.

N+1 hides on your laptop

Fri, 05 Jul 2019 22:35:00 +0300

Ten rows in the dev database, on the same machine, 0.1 ms per query. Ten extra queries is one millisecond. The page feels instant, the code ships. Production has a thousand rows and the database one network hop away. Round trip is about a millisecond even in a good datacenter. A thousand queries is a second of pure network waiting. Not slow SQL. Each query is fast. The plural is slow.

One use case, one class

Wed, 26 Jun 2019 15:30:00 +0300

Four hundred lines in a checkout action, half of them inside one try. That is one way a project dies. The other is OrderManager: thirty methods, six injected services, and nobody can say what the class is for, because it is for everything about orders.

Both diseases have the same cure and it is embarrassingly simple. One use case, one class, one public method.

final class PlaceOrder
{
    public function __construct(
        OrderRepository $orders,
        PaymentGateway $payments,
        EventDispatcher $events
    ) { ... }

    public function handle(PlaceOrderCommand $command): OrderId
    {
        // the whole story, top to bottom
    }
}

The shape answers three questions by itself. What comes in: one command object, a thing you can log, validate, put in a queue. What it needs: the constructor lists dependencies of this use case, not of orders in general, so when PlaceOrder suddenly needs the mailer you see it and can ask why. What comes out: one result. Reading handle() from top to bottom tells the whole story of placing an order. No chapter hidden in a base class or a trait.

Messenger retries and the failure transport

Thu, 30 May 2019 23:25:00 +0300

A log full of the same stack trace, every second, all night. That is what a queue looks like when a handler throws on a malformed payload and the retry policy is “forever”. Symfony 4.3 is out this week, and Messenger in it finally has an answer for that night.

Retry is good for transient errors. Network blinked, deadlock, remote API returned 503. Wait, try again, it passes. But some errors are permanent. Malformed payload, entity deleted, a bug in the handler. Retrying those forever means a worker grinding the same poison message until morning. Infinite retry is not persistence, it is denial.

What Redis does when memory ends

Tue, 02 Apr 2019 15:10:00 +0300

Users logged out at random. Not all, not always. Only sometimes, only in the afternoon. Afternoon is when traffic peaks. Traffic peaks fill the cache. The cache lived in the same Redis as the sessions, the instance hit maxmemory, and the eviction policy was allkeys-lru. Redis did exactly what we asked: threw away the least recently used keys, and some of them were sessions of people who went for lunch.

Transactional DDL is not portable

Mon, 18 Mar 2019 22:45:00 +0300

Statement three of a migration fails. Column name typo. On PostgreSQL the first two ALTERs roll back with it, the schema returns to the exact state before the migration, you fix the typo and run again. On MySQL the first two ALTERs are already permanent.

Same up() method, same php artisan migrate, same green output. The migration tool gives one abstraction over two very different databases. The abstraction covers syntax. It does not cover what happens on failure.

Eloquent observers and hidden control flow

Wed, 27 Feb 2019 12:40:00 +0300

Three thousand welcome emails. A colleague ran a user import on one project last month: loop over a CSV, $user->save(), go home. Next morning we found the script had also warmed the search index three thousand times and invalidated cache after every row. Nobody wrote that in the import script. The observers did.

Laravel 5.8 came out yesterday, and reading the changelog brought that evening back, so here is the note.

One payment is enough

Mon, 21 Jan 2019 20:35:00 +0300

Charge the card, send the receipt, ack the job. The worker lost its Redis connection between step one and step three. The queue delivered the job again. The customer paid twice. Support learned some new words from him.

Retries are not an edge case. Laravel retries failed jobs by design, and you want that, because networks blink. So every job with a side effect must answer one question: what happens if this runs twice. “It will not run twice” is not an answer. It will.

PHP 7.3 and the value of small releases

Thu, 27 Dec 2018 15:35:00 +0300

A trailing comma after the last argument of a function call. PHP 7.3 came out at the start of the month, and this is the feature I noticed first, because I meet the missing comma in every second diff:

$this->logger->info(
    'order created',
    ['order_id' => $order->id],
);

A tiny thing that kills a whole class of noisy diffs, the ones where adding an argument touches the previous line too. Arrays got this years ago. Calls only now.

Deprecations are a to-do list, not noise

Fri, 30 Nov 2018 15:15:00 +0300

SYMFONY_DEPRECATIONS_HELPER='max[total]=20' ./bin/phpunit. Twenty is the number of deprecation warnings we had on the day we started counting. The rule is that it only goes down.

Symfony 4.2 came out yesterday, on the six month schedule. With this cadence deprecation warnings stop being an event and become weather. Every minor release brings a new batch, teams train themselves not to see them, and that is a mistake, because a deprecation is the next major upgrade delivered early, in small pieces, with instructions.

PostgreSQL 11: partitions grow up, JIT arrives

Fri, 26 Oct 2018 19:10:00 +0300

jit = on, restart, run the API test suite. Same numbers as before. That was my first evening with PostgreSQL 11, released last week, and it was the correct result.

JIT is the loud feature of this release and the misunderstood one. Postgres can now compile expression evaluation into machine code through LLVM. People read “compilation” and expect their endpoints to get faster. They will not. A primary key lookup takes a fraction of a millisecond. There is nothing in it worth compiling, and the compilation itself costs more than the whole query. JIT is for the other kind of query: an aggregate chewing through millions of rows, where the same expression runs so many times that generating machine code for it pays back.

Anatomy of email verification

Tue, 18 Sep 2018 18:10:00 +0300

email_verified_at, a timestamp. Laravel 5.7 came out this month with email verification built in, and this column is the first thing I noticed. A boolean would cost the same and answer less. A timestamp answers not only whether, but when, and when a support ticket arrives half a year later, “when” is the question.

The tutorials say: implement MustVerifyEmail, put the verified middleware on routes, done. True, and boring. The interesting part is how the feature is put together. It is a small example of a cross-cutting feature done right.

Where the checkout logic goes

Mon, 13 Aug 2018 22:15:00 +0300

Validate input, reserve stock, create the order, charge the card, fire events. Five steps in one checkout action, and the fat model versus fat controller argument offers only two rooms for them. Both rooms are wrong.

Put it all in the controller and you cannot run checkout from anywhere except HTTP. No console command, no queue job, no test without the kernel. Put it in the Order model and the model now knows about payments, stock and notifications, a strange set of friends for an Eloquent class.

Covering index and the price of SELECT *

Sun, 15 Jul 2018 12:35:00 +0300

Using index in the Extra column of EXPLAIN. That is the cheapest read MySQL can do, and you often get it almost for free.

SELECT user_id, created_at
FROM orders
WHERE user_id = 42
ORDER BY created_at DESC
LIMIT 20;

With an index on (user_id, created_at) everything the query needs is in the index leaves. The table is never touched. A composite index finds rows fast, a covering index answers the query on its own. Postgres calls this Index Only Scan, with one condition: the visibility map must be fresh, so a table that vacuum never visits quietly falls back to heap fetches.

Messenger: in-process today, queue tomorrow

Tue, 19 Jun 2018 22:50:00 +0300

$bus->dispatch(new SendWelcomeEmail($user->getId())); and then, in the same request, a handler:

class SendWelcomeEmailHandler
{
    public function __invoke(SendWelcomeEmail $message)
    {
        // load user by id, send the email
    }
}

That is Messenger in Symfony 4.1, still marked experimental. By default everything is synchronous, so at first it is a function call with extra steps.

The extra steps are for the transport. Change configuration, route this message class to AMQP, and the same handler runs in a worker process. The calling code does not change. Start synchronous, go async when you need it. I like this order much more than “install RabbitMQ on day one”.

GDPR: where the user actually lives

Wed, 09 May 2018 10:45:00 +0300

The row in users. Then orders, with the delivery address and phone. Then nginx access logs with emails inside GET parameters of an old unsubscribe endpoint. That is how far I got in the first ten minutes of drawing where one user’s data lives on one project, and the map did not fit on one page.

May 25 is close and everyone discusses consent banners. The lawyers can have the banners. The engineering part is the right to erasure. “Delete the user” sounds like one DELETE statement until you sit down with a pen.

MySQL 8.0: finally, modern SQL

Tue, 24 Apr 2018 22:15:00 +0300

@row := @row + 1. I have a report in production that stands on that trick, and the trick was never guaranteed to work. Evaluation order of user variables in SELECT is undefined, it just happened to behave. MySQL 8.0 went GA last week, and for me the release is about SQL. Window functions and CTEs, the things Postgres people stopped noticing years ago, are here.

The classic task: top three orders per customer. In 5.7 you had a self-join nobody could read a month later, or the variable trick. Now:

.env is not a secrets store

Wed, 14 Mar 2018 15:40:00 +0300

A .env file committed to git, “temporarily”. Database password, API keys, mailer credentials, all in one file. Symfony 4 moved configuration to environment variables, and this is the second project this month where I see the same thing.

.env is a developer convenience. It exists so local setup does not require exporting fifteen variables by hand before running the app. That is the whole job of this file. .env.dist goes to git with placeholder values, .env stays in .gitignore with your local ones. This part is not negotiable.

Rector: refactoring by machine

Thu, 08 Feb 2018 15:40:00 +0300

Search for ->fetch( in a project with old Kohana code and count the hits. A method call on a model, a comment, a line in a test fixture, an unrelated class that happens to have a method with the same name. That is what regex refactoring looks like. Regex does not know a method call from a string literal.

Rector does. A young tool I found this month: it parses PHP into an AST with nikic/php-parser, applies transformation rules, prints the code back. Rename a class across the whole project. Change a method call, add an argument everywhere. The AST sees that this fetch is called on that type and touches only those places. Mechanical change becomes exact.

Horizon, or the queue becomes visible

Sat, 20 Jan 2018 10:30:00 +0300

redis-cli llen queues:default and hope. That was my queue monitoring for years. Horizon replaces it with a real dashboard, and I did not know how much I needed one until I saw it.

The mental shift matters more than the UI. dispatch() is the beginning of the work, not the end. The job still has to wait in Redis, run, maybe fail, maybe retry. All of that was invisible. Now it is on one screen: throughput, wait time per queue, failed jobs with the full payload and the exception.

Sodium in PHP 7.2 core

Sun, 10 Dec 2017 20:20:00 +0300

PHP 7.2 came out on the last day of November. Mcrypt is out of core, libsodium is in. That trade alone makes it a good release.

Search any forum for “php encrypt” and you find the same folk recipe: openssl_encrypt with AES-256-CBC, an IV made from who knows what, no authentication of the ciphertext. Every choice in that recipe is a place to be wrong, and CBC without a MAC is wrong in a way that has published attacks. The developer is not careless. The API hands an application developer decisions that belong to a cryptographer.

Symfony 4: the framework as recipes

Thu, 30 Nov 2017 21:35:00 +0300

A kernel, an index.php, a config folder with almost nothing in it. That is a Symfony 4 project on day one, and 4.0 came out today together with 3.4 LTS. The code changes are the smaller story. How a project starts is the bigger one.

The Standard Edition is gone. You start from the empty skeleton and ask for what you need. composer require orm pulls Doctrine, and the recipe writes the config files, registers the bundle in bundles.php, adds the variables to .env. You watch your own repo grow file by file as you add dependencies.

PostgreSQL 10: partitions I can finally use

Thu, 26 Oct 2017 19:20:00 +0300

DROP TABLE events_2017_10. That is how you delete October now, and it is the reason I am reading PostgreSQL 10 release notes on a Thursday evening instead of waiting a year like usual.

Partitioning existed before, through inheritance, CHECK constraints and an insert trigger you wrote yourself and hoped was right. Now the database owns it:

CREATE TABLE events (
    id bigserial NOT NULL,
    created_at timestamptz NOT NULL,
    payload jsonb
) PARTITION BY RANGE (created_at);

CREATE TABLE events_2017_10 PARTITION OF events
    FOR VALUES FROM ('2017-10-01') TO ('2017-11-01');

The win is the data lifecycle. An events table grows forever, and deleting a year of history with DELETE is a night of I/O plus a bloated table in the morning. With partitions, retiring a month is one DROP. Instant. And a query that filters by created_at visits only the partitions in range, the planner skips the rest.

First run of PHPStan on legacy code

Sat, 09 Sep 2017 15:30:00 +0300

Level 0, a legacy codebase, a few hundred errors on the first run. That was my week with PHPStan. Most of the output was noise about magic the tool cannot see. But in the first hour of reading I found three real bugs, live in production for months.

One: a repository method returns an entity or null, and a caller chains a method right on the result. The not-found branch was never written. It survived because that path needs a deleted record, and deleted records are rare. Rare is not never.

Laravel 5.5 and package auto-discovery

Thu, 31 Aug 2017 15:45:00 +0300

Laravel 5.5 came out yesterday, an LTS: two years of bug fixes, three of security fixes. Good news for projects that outlive the hype cycle, which is most of them.

The feature in every screenshot is package auto-discovery. A package declares its service provider in its own composer.json, and after composer require it is registered. No editing of config/app.php, no provider line copied from the readme.

Convenient. Now look at what got deleted. Installing a package used to be two steps, and the second step was consent: a line in my own config saying this code runs inside my application at boot. Now composer require is the whole ceremony. The list of code that runs at boot lives in vendor/, spread over other people’s files. php artisan package:discover shows it, and dont-discover in composer.json switches it off per package. But the default flipped from explicit to implicit.

Multi-stage builds for PHP images

Fri, 07 Jul 2017 18:35:00 +0300

720 MB. That was our production PHP image, and most of it was composer, git, unzip, build headers and a pile of apt cache. None of it runs in production. It was there because composer install needs it, and a Dockerfile was one linear script. People worked around this with two Dockerfiles and a shell script gluing them. Ugly, and everyone’s glue was different.

Docker 17.05 brought multi-stage builds. The first Docker feature in a while that fixes a problem I actually had. Now it is one file:

Keyset pagination instead of OFFSET

Mon, 19 Jun 2017 10:20:00 +0300

LIMIT 50 OFFSET 500000. Page 10001 of an events table, from a paginator someone wrote in an afternoon. Postgres has no shortcut to row 500000. It walks the index through half a million entries, fetches them, throws them away, and returns fifty. Page one is fast. Page ten thousand is slow, and every page after it is slower. Run EXPLAIN ANALYZE on both: the plans are identical, the numbers are not.

Private services: the container steps back

Tue, 30 May 2017 15:15:00 +0300

$container->get('app.mailer') stopped working this week. Symfony 3.3 is out, and in the new configuration services are private by default. The container refuses the call. Everyone is writing about autowiring. This is the change I like more.

That call was always a smell. A class that pulls dependencies from the container by string id is a class with secrets. Its constructor says nothing, and to test it you boot half the framework, because it may ask for anything at any moment.

ALTER TABLE is part of the deploy

Mon, 24 Apr 2017 15:10:00 +0300

Forty million rows, three application instances behind a balancer, one migration that renames a column. The deploy script runs migrations first, then rolls the code instance by instance. For a few minutes old code was writing to a column that no longer existed. The error rate graph looked like a wall. I had approved that migration. It was one line.

Migration tools trained us to think a schema change is a code change. Write a file, run it, done. A schema change on a big table takes time, takes locks, and while it runs the previous version of the application is still serving traffic. Both versions must survive both schemas. That is the whole discipline, and it has a name: expand and contract.

Autowiring is coming to Symfony 3.3

Wed, 08 Mar 2017 15:25:00 +0300

Our services file repeats every class name three times. Once as the id, once as the class, and again in the arguments of every service that needs it. Add a constructor parameter, edit YAML, clear cache, repeat. Symfony 3.3 lands in May and promises to delete most of that file:

services:
    _defaults:
        autowire: true
        autoconfigure: true

    AppBundle\:
        resource: '../../src/AppBundle/*'

Autowiring and autoconfiguration on by default, the whole source folder registered in five lines. I have mixed feelings. Sorted, they look like this.

Cache stampede: the hot key problem

Tue, 07 Feb 2017 21:30:00 +0300

Cache hit rate 98 percent. The database still fell over. Both facts are true, and the second one does not care about the first.

What happened. One key holds the result of a heavy query for the main page. TTL one minute. At some second the key expires. In that second a few hundred requests all get a miss, and all of them go to recompute the same heavy query. Redis is fine. MySQL is not. This is a cache stampede, and hit rate will not warn you, because hit rate is an average. The stampede lives in the worst second.

Laravel Dusk and what browser tests are for

Thu, 26 Jan 2017 18:30:00 +0300

Laravel 5.4 came out on Tuesday, and the first thing I installed was Dusk. Browser tests without a Selenium server: it talks to ChromeDriver directly, the API is fluent, and a failed test leaves a screenshot behind.

My first test was the login path:

$this->browse(function ($browser) {
    $browser->visit('/login')
        ->type('email', 'user@example.com')
        ->type('password', 'secret')
        ->press('Log in')
        ->assertPathIs('/home');
});

It passed on the third run. The first run failed because the button on the test box had different text. The second failed because a JS animation was slower than the wait. This is the normal life of a browser test. It sees the real application, and the real application moves.

PHP 7.1: nullable is honest

Fri, 09 Dec 2016 20:35:00 +0300

// returns User or false, see wiki
public function findByEmail($email)

Every legacy codebase has this method. The comment is the type system. Half the callers check for false, some check for null because a sister method returns null, one caller checks nothing and works by luck.

PHP 7.1 came out last week. Nullable types, void, iterable, multi-catch. Small features, but together they continue the direction 7.0 started: less implicit agreement, more signature. The one I care about is ?Type:

Order status is a graph

Wed, 30 Nov 2016 22:10:00 +0300

if ($order->status == 'paid') in the shipping module. Another one in the refund module. A slightly different one in the admin panel. Then someone cancels a shipped order from an admin button, because the button just writes the string, and the warehouse learns about it a week later.

Symfony 3.2 is out this week and ships a Workflow component. Finally a framework says out loud what every order table has been whispering for years: status is a state machine, not a string column you assign wherever convenient.

The queue will run your job twice

Mon, 03 Oct 2016 15:20:00 +0300

The job called the payment provider. The call succeeded. Then the worker timed out before it marked the job done. The queue did what queues do: it retried. The provider did what it was asked: it charged again. The customer did what customers do and wrote an angry email.

Nobody made a mistake here. The queue promises at-least-once delivery, and “at least” is written in the contract. Network partitions, worker crashes, deploy restarts. Sooner or later every job runs twice, and the jobs that hurt are exactly the ones with external effects: payments, emails, webhooks, API calls.

Parallel query in PostgreSQL 9.6

Fri, 30 Sep 2016 22:10:00 +0300

38 seconds to 11. Same query, same forty million rows, no index added, no SQL changed. The only difference is PostgreSQL 9.6, released yesterday, and one setting.

The query is a typical report: count and sum over an events table, grouped by day, three months of data. On 9.5 the plan is one process grinding through the table. On 9.6 with parallelism enabled:

Finalize HashAggregate
  ->  Gather
        Workers Planned: 4
        ->  Partial HashAggregate
              ->  Parallel Seq Scan on events

Four workers scan their own chunks, aggregate partially, the leader merges. Just more hands. For years we optimized SQL as if the database had exactly one worker per query. That assumption expired yesterday.

Laravel 5.3: the app is not a website anymore

Fri, 26 Aug 2016 12:30:00 +0300

Mail::send() inside the checkout controller. A second copy inside the API controller. Then someone adds the chat webhook to one copy and forgets the other. Every project has this code. I have written it more than once.

Laravel 5.3 came out this week, and the two big pieces, Notifications and Passport, both point the same way. The application is no longer a thing that renders HTML. It is a core that talks to browsers, mobile clients and third parties, and HTML is one of the outputs.

Column order in a composite index

Mon, 04 Jul 2016 23:25:00 +0300

Fifty rows. Three indexes on the table, one per column. Still slow. The query is the standard one from any multi-tenant application:

SELECT * FROM orders
WHERE tenant_id = ? AND status = ?
ORDER BY created_at DESC
LIMIT 50;

“Add an index” is the typical reaction, and one index on each column is the typical result. The number of indexes is not the point. The order of columns inside one index is.

Doctrine and the 100 000 row import

Sun, 19 Jun 2016 20:35:00 +0300

Row sixty thousand. Allowed memory size exhausted. The import script on one project died there, and the code was the obvious loop: read a row, persist() an entity, next row, flush() at the end. On a hundred test rows it worked perfectly. It was my loop.

The reason is the Unit of Work. Doctrine keeps every managed entity in memory, plus a snapshot of its original data for change tracking. Persist a hundred thousand entities and you hold a hundred thousand objects twice. This is not a bug. It is the price of the ORM’s main feature, and on a normal web request the price is invisible because the request dies young.

Symfony by the piece

Tue, 10 May 2016 19:40:00 +0300

A folder of cron scripts, each one a .php file with hand-parsed $argv. That is what CLI looked like on our legacy Kohana application until last month. One line fixed it:

composer require symfony/console

Now each script is a small Command class with named options, --help for free, and exit codes that cron can actually check. The framework around it did not notice anything. Console does not care who serves your HTTP.

Docker instead of a setup README

Wed, 13 Apr 2016 21:50:00 +0300

Two days. That is how long the new colleague spent last month getting one project to run on his laptop. The README is two pages: PHP with a specific set of extensions, Nginx, MySQL, Redis, and a paragraph that starts with “on OS X it is a bit different”. Every laptop in the team is a slightly different snowflake.

So I finally tried Docker for local development. The compose file:

Freezing the Kohana layer

Fri, 04 Mar 2016 20:10:00 +0300

Kohana 3.3, on a project that earns money every day. The framework is effectively finished. The repository barely moves, the community left years ago. Nobody will approve a rewrite, and I have stopped asking. This is a normal situation and it deserves a better plan than “someday we migrate”.

The plan we settled on is a freeze.

Pin the exact framework version and vendor it. Not “3.3.*”, the exact commit. The build must be reproducible in five years, when the original download link is dead.

Middleware groups in Laravel 5.2

Mon, 08 Feb 2016 21:10:00 +0300

An IP whitelist check placed after auth. That was the whole bug. Every scanner bot on the internet was going through session start and a user lookup just to be told to go away. The fix was to move one line up in the Kernel.

Laravel 5.2 makes that line easier to see. Middleware groups, and the request pipeline is finally written down in one place:

protected $middlewareGroups = [
    'web' => [
        \App\Http\Middleware\EncryptCookies::class,
        \Illuminate\Session\Middleware\StartSession::class,
        \App\Http\Middleware\VerifyCsrfToken::class,
    ],
    'api' => [
        'throttle:60,1',
        'bindings',
    ],
];

Before 5.2, sessions and CSRF were global middleware. They ran for everything, including API routes that have no use for cookies. Now web and api are two separate pipelines, and you can read each one top to bottom.

Upsert in PostgreSQL 9.5

Tue, 12 Jan 2016 22:15:00 +0300

$row = $db->fetchOne('SELECT id FROM counters WHERE name = ?', array($name));
if ($row) {
    $db->execute('UPDATE counters SET value = value + 1 WHERE name = ?', array($name));
} else {
    $db->execute('INSERT INTO counters (name, value) VALUES (?, 1)', array($name));
}

Works on the laptop. In production two requests arrive in the same millisecond. Both SELECT, both see nothing, both INSERT. One dies with a duplicate key error. Or worse, there is no unique constraint, and now you have two rows and a bug report you cannot reproduce. The window between SELECT and INSERT is tiny, so it fires once a week, always for someone else.

PHP 7 measured on real code

Sat, 26 Dec 2015 21:35:00 +0300

One endpoint. The heaviest catalog page of one project, same data, same opcache settings, PHP 5.6 against 7.0. That is the only benchmark I trust. 7.0 has been out for three weeks, everyone has seen the hello-world numbers, and I do not believe hello-world numbers on principle.

Response time dropped about 40 percent. Memory per request, more than half. No code changes. I ran it again because I did not believe the first numbers either. The engine rewrite is real: smaller internal structures, cheaper function calls, and typical framework code is exactly that, thousands of small calls and arrays.

Symfony 3.0 is not a rewrite

Mon, 30 Nov 2015 22:05:00 +0300

Symfony 2.8 and 3.0, released today, both at once, on purpose. The pairing is the whole message. 2.8 is the last of the 2.x line and an LTS. 3.0 is roughly 2.8 with the deprecated code deleted. Same features, cleaner body.

This turns a scary major upgrade into two steps.

Step one, move to 2.8. Minor upgrade, low risk. Then make the deprecation log empty. The phpunit bridge prints every deprecated call your code and tests touch. Burn the list down during normal sprints, item by item, each fix small and releasable on its own.

JSON column is not a schema

Fri, 23 Oct 2015 22:40:00 +0300

ALTER TABLE users ADD profile JSON;

MySQL 5.7 went GA this week, and this line is now legal. Native JSON type: validation on insert, binary storage, functions to read paths. The question is not whether it works. It is what belongs in it.

Classic case: user profile with flexible metadata. Marketing wants a new field every other week. The old options were bad in familiar ways. Forty nullable columns and an ALTER for every idea. Or EAV, key-value rows, where every read is a self-join festival and nothing has a type.

Kohana on PHP 7: whose problem is it

Tue, 08 Sep 2015 10:45:00 +0300

The test suite went green on the PHP 7 release candidate on the second try. I expected worse. Then I opened modules/ and the mood changed.

The core mostly works. The community is patching compatibility in the 3.3 branch on GitHub right now, and the fixes are small: deprecated PHP 4 style constructors, changed engine behavior in dark corners. Whoever says the framework is dead is not reading the commit log. It is not dead. It is volunteers.

Redis cache and Redis queue are different databases

Fri, 21 Aug 2015 19:45:00 +0300

Two lines from redis.conf on one project:

maxmemory 2gb
maxmemory-policy allkeys-lru

Correct for a cache. Then sessions moved into the same instance, because Redis was already there. Then the job queue, same reason. One process, three tenants. This works until the day it does not.

Cache data is disposable by definition, and those two lines embrace that. Memory fills up, Redis evicts the coldest keys, the application rebuilds them on demand. Persistence is optional. After a restart a cold cache is an inconvenience, not an incident.

Run your tests on PHP 7 now

Wed, 08 Jul 2015 22:45:00 +0300

Three lines in .travis.yml. That is the whole cost of knowing in July what will break on PHP 7 in December.

php:
  - 5.6
  - nightly
matrix:
  allow_failures:
    - php: nightly

Alpha builds are out, release is planned for this autumn, and production is on 5.6 and will stay there for a while. The build stays green on 5.6. The nightly job fails quietly in the corner and produces a list of future problems while nobody is under pressure to read it.

Laravel 5.1 and the dependency ladder

Sun, 14 Jun 2015 19:10:00 +0300

PHP 5.5.9. That is the minimum for Laravel 5.1, which came out this week, the first Laravel with LTS: two years of bug fixes, three of security fixes. Two weeks ago I wrote the same about Symfony 2.7. Two frameworks arriving at the same idea in one summer is no coincidence. PHP applications got old enough to have a lifecycle.

Draw the ladder for your project once. OS at the bottom. Then the PHP version the OS ships, or the one you build. Then the framework. Then the packages on top. Every rung has its own end-of-life, and the rungs are coupled. If production sits on some old distro with 5.4, the 5.1 upgrade is suddenly an ops project. And above you, half the packages will require 5.1 in their next major, so staying behind slowly cuts you off from fixes.

Symfony 2.7: LTS is the feature

Sun, 31 May 2015 12:35:00 +0300

Three years of bug fixes. Four years of security fixes. Symfony 2.7 came out this weekend, and that is the line from the announcement I care about. The changelog has new things in it, but the important word is on the label: LTS.

For a pet project this is boring. For a product with paying users and a team, this is the actual feature.

A big application does not upgrade for fun. Every framework upgrade is testing time, regression risk, and a sprint that produces nothing visible. Business asks a fair question: what do we get. “Newer version” is not an answer. “We keep receiving security fixes until 2018 without touching anything” is.

Where Doctrine spends your milliseconds

Thu, 23 Apr 2015 10:45:00 +0300

$order->getCustomer()->getName() inside a foreach. Every time I open a slow Doctrine endpoint, I find some version of this line within ten minutes. The PHP is rarely slow. The shape of the data access is.

Same few places every time.

Lazy loading. The classic N+1. Doctrine proxies make it invisible: you touch the relation in a loop and every touch is a query. The profiler query count tells the truth. The code does not.

Three ways to get a dependency in Laravel

Mon, 09 Mar 2015 21:10:00 +0300

Same repository, three ways to get it in Laravel 5.

// 1. Constructor injection
public function __construct(OrderRepository $orders)
{
    $this->orders = $orders;
}

// 2. Facade
$order = Orders::find($id);

// 3. Service locator
$orders = App::make('App\Repositories\OrderRepository');

All three work. The container resolves everything either way. The difference is in what the class tells you about itself.

With constructor injection the dependencies are in the signature. You open the class, you read the constructor, you know what it needs. A test passes a mock and never touches the container.

Laravel 5 and the death of the models folder

Wed, 25 Feb 2015 22:20:00 +0300

app/models is gone. Laravel 5 came out at the start of the month, and of everything in the release the folder I notice most is the one that no longer exists.

The old skeleton had app/models and app/controllers. You knew where things go because the framework told you. Laravel 5 removes that. There is app/, it maps to the App namespace through PSR-4, and inside it you organize classes however you want.

Composer in a legacy Kohana project

Thu, 22 Jan 2015 20:05:00 +0300

require APPPATH.'../vendor/autoload.php';

One line in bootstrap.php. This month it became the whole strategy for a project we decided to keep on Kohana.

The framework is quiet, the project is alive. These two facts have to coexist somehow. The plan: nothing new gets written the Kohana way. New dependencies come through Composer, and Kohana’s autoloader and Composer’s autoloader live together fine.

First candidate was the HTTP client. We talk to two external APIs, and the code around Request_Client_External was not pretty. Guzzle is better in every way. But Guzzle does not spread through the codebase. There is an interface:

Host header is user input

Tue, 23 Dec 2014 19:20:00 +0300

$link = 'https://'.$_SERVER['HTTP_HOST'].'/reset/'.$token;

I wrote this line. More than once. It builds the link for a password reset email, and it looks like it reads something about the server. It does not. Host comes from the client, same as any other header.

Kohana 3.3.4 shipped this month with a fix around exactly this, and the topic is bigger than one framework.

The attack is short. Someone requests a password reset for your email address and puts his own domain into the Host header. The application trusts the header, builds the link, sends it. You receive a real email from a real site with a link to evil.example and your real token in the path. You click. He collects the token. Host header poisoning, and no framework protects you from it out of the box.

Symfony 2.6: dump() and boring upgrades

Thu, 18 Dec 2014 22:40:00 +0300

dump() instead of var_dump(). Symfony 2.6 came out at the end of November, and this is the visible gift: VarDumper. Collapsible output, clickable class names, works inside the toolbar. A small thing, used fifty times a day. Tools like this improve life more than big features do.

But the important story in 2.6 is not a feature. The release notes read like a preparation checklist for Symfony 3: deprecations, everywhere.

Symfony best practices: defaults, not laws

Sun, 14 Dec 2014 20:30:00 +0300

One AppBundle. That is the line in the official Symfony Best Practices book I read twice. For years the answer to “how many bundles” was “as many as you have features”, and here the framework’s own book says: one.

The book came out this autumn. Worth reading even if you are not on Symfony, because the interesting part is the change of tone.

The old Symfony way: everything is a bundle. Your application is bundles, reusable, configurable, with their own extensions and semantic configuration. Very flexible. And for a normal business application, mostly ceremony. You write a configuration class for code that will never leave this one project. I wrote such classes. Three of them. None was ever reused.

Debug toolbar in production is a gift to strangers

Tue, 09 Dec 2014 21:00:00 +0300

/_profiler on a live site. Type it after any Symfony domain and sometimes it answers. This autumn the story went around, and I will not retell it. The lesson is bigger than one framework.

Think what a profiler actually stores. Every SQL query with parameters. Cookies and session data. Routes, controller names, request headers. Sometimes config values. A full X-ray of your application, nicely formatted, with search.

Now count how many sites have /_profiler or a debugbar open to the world because someone deployed with app_dev.php, or left debug = true in the production config. Not a rare exotic mistake. I saw it. I made it once, on a staging server that stayed reachable from outside longer than anyone planned.

PHP 5.6: small syntax, big TLS surprise

Fri, 05 Dec 2014 19:45:00 +0300

A nightly cron failed silently for two days. That was our PHP 5.6 upgrade.

5.6 is out since the end of August, and we moved one production project this month. The syntax part is pleasant and minor. Variadics:

function log_all($level, ...$messages) { }

Argument unpacking with ...$args on the call side. Constant expressions in defaults. use function for importing functions. All nice. None of it changes your architecture. You will use variadics maybe twice a year and be happy both times.

Services instead of fat controllers

Tue, 02 Dec 2014 22:20:00 +0300

120 lines in one action. Order confirmation: load the order, charge the card, send the email, render the page. It ran fine for a year. Then a console script needed the same operation, and the action turned out to be glued to SMTP, to the database and to the HTTP request. All at once.

Symfony people talk about dependency injection so much it sounds like religion. It is one simple idea: a class receives its dependencies and does not create them.

Queues in Laravel 4.2: the user should not wait for your SMTP

Thu, 27 Nov 2014 20:50:00 +0300

1.8 seconds for one registration request. I put a timer around it on a project last month. 1.5 of those seconds was the welcome email going out over SMTP.

The user waits almost two seconds and looks at a spinner, for a handshake with a mail server he will never hear about.

Laravel 4.2 makes the fix one line:

Queue::push('SendWelcomeEmail', array('user_id' => $user->id));

The controller returns in 300 ms. A worker picks the job up and sends the email. php artisan queue:listen to start, beanstalkd or Redis behind it, the failed_jobs table for jobs that died.

Composer next to Kohana bootstrap

Sun, 23 Nov 2014 21:10:00 +0300

One line in bootstrap.php, above Kohana::init():

require DOCROOT.'vendor/autoload.php';

That is the whole integration. Kohana keeps its modules and its underscore autoloader. Composer brings Packagist. Two autoloaders in one process, and they do not collide: Composer packages use namespaces, Kohana classes have underscores, the lookups never meet.

{
    "require": {
        "guzzlehttp/guzzle": "~5.0"
    }
}

This setup runs in production on one of my projects for a few months. Zero problems from the autoloading side. The problems, when they came, were from me.

Kohana is not developed anymore. Now what

Wed, 19 Nov 2014 19:30:00 +0300

The commit log of kohana/kohana on GitHub. I opened it in the summer and counted: a few merges a month, then fewer, then nothing. This year the official development basically stopped. Our projects did not stop working on that day. But the question came to my inbox three times since, so here is the answer.

What to do with a working Kohana product depends on which of four situations you are in.

Cascading filesystem: the best idea in Kohana

Sun, 16 Nov 2014 22:05:00 +0300

Kohana::find_file(). One function, maybe forty lines. If I could keep only one thing from Kohana, this is the one.

It looks for every file in order: application/, then modules, then system/. First found wins. Config, views, classes, i18n, everything goes through the same lookup.

In practice: a module ships a config file, you put a file with the same name into application/config/, change one key. The module does not know. Nothing under modules/ is touched.

Kohana ORM and my first real N+1

Wed, 12 Nov 2014 20:15:00 +0300

Fifty one queries. That is what the SQL panel of the Kohana profiler showed for one catalog page. Fifty posts on the page.

Nothing changed in the code. The page took two seconds because there was more data than in spring, and the code was written for spring.

$post->author->name in a loop. One query for the list of posts, then one more for the author of every post, inside the foreach. The line looks innocent, and that is the problem: lazy loading hides the price. You write the relation, you go home early, and the bill comes in six months.

HMVC in Kohana: a request inside a request

Mon, 10 Nov 2014 21:40:00 +0300

Request::factory('widgets/stats') inside a controller. A colleague looked at this line last week and asked the question I hear most often about Kohana: you already have a request, why make another one?

I keep answering it at the desk. So I will write it down once. Short notes, mostly backend, mostly PHP. This is the first.

The answer is composition. Take a dashboard. A stats block, a recent orders block, a notifications block. Each one needs its own data and its own logic. You can put all of it into one action. It works. In a month it is 300 lines and nobody wants to open it.