Sql on Murat Useinov

Testing PostgreSQL 19 beta without guessing

Wed, 08 Jul 2026 21:45:00 +0300

A production-like dump, a separate PostgreSQL 19 beta instance, one evening plus machine time. That is the whole method, and it is the only thing a database beta is for: finding out where the new version changes the behavior of your application while the change is still a bug report and not your incident.

Restore the dump first. Data size matters. A plan that is fine on a thousand rows goes bad on fifty million.

MySQL 9.7 LTS is an operational choice

Fri, 10 Apr 2026 20:10:00 +0300

An SQL mode set in 2019 that nobody remembers why. An authentication plugin the driver happened to support. A charset and collation nobody chose. That is what a MySQL upgrade is made of. MySQL 9.7 LTS is out, so I am making the list again.

An LTS release of a database is a date more than a feature list. The date until which somebody else worries about patches, and the date after which the worry is yours. For a working PHP project the question is not “what is new”. It is “what did we assume”.

PostgreSQL 18, the upgrade notes

Fri, 26 Sep 2025 22:45:00 +0300

PostgreSQL 18 went GA yesterday. I ran the beta in July. Now it is not a preview, it is a ticket, and these are the notes from planning it.

The quiet feature that matters most for the upgrade itself: pg_upgrade now carries planner statistics over. Before, the minutes or hours after switching versions were the scary part. Fresh cluster, empty statistics, the planner guessing, production limping until ANALYZE finished. That cliff is mostly gone. This alone changes how a major upgrade feels at 2 a.m.

A weekend with PostgreSQL 18 beta

Wed, 02 Jul 2025 18:50:00 +0300

A staging copy of one database, restored over the weekend onto PostgreSQL 18 beta. GA is expected in autumn. Two things I wanted to touch: asynchronous I/O and the built-in uuidv7().

Start with uuidv7, because it is about a mistake many of us already made, me included. Random UUIDv4 primary keys scatter inserts across the whole index. Every insert lands on a random page, the working set is the entire index, buffers churn. UUIDv7 is time-ordered: new rows go to the same few pages, like a sequence, but still globally unique.

PostgreSQL 17 and why VACUUM is an API concern

Fri, 27 Sep 2024 15:20:00 +0300

A reporting transaction open for hours, and n_dead_tup on the hot table climbing all day while autovacuum ran non-stop and changed nothing.

That was one project. PostgreSQL 17 came out yesterday, and the headline work is exactly this plumbing: vacuum got a new memory structure for tracking dead tuples, far less memory and no longer capped the old way, plus better throughput for high-concurrency writes and bulk loads. Maintenance internals. Which is why backend people should care.

AI assistants do not know your production

Sat, 23 Dec 2023 10:40:00 +0300

A DTO with a dozen fields, a framework migration, a slow query incident. Three tasks from the last few months, all done with Copilot and ChatGPT open. Here is where the line runs for me.

Task one, boilerplate. The DTO, its validation rules, a test with fixtures. The assistant writes this faster than I type, and the result needs only a quick read. Same for converting an array-shaped legacy structure into typed classes. This is honest time saved, maybe an hour a day. The code is the kind where being generic is correct.

PostgreSQL 16, read through one workload

Sat, 16 Sep 2023 20:25:00 +0300

A nightly import, a few hundred million rows through COPY, then reporting queries in the morning. That is the only workload I care about, so that is how I read the PostgreSQL 16 release notes: with one pipeline in mind and everything else skipped.

Three items survived the filter.

COPY got faster, noticeably when several loads run at once. Our import is exactly this shape, parallel COPY streams into partitioned tables. I will take the free speed. But the old lesson stays: if your import still does row-by-row INSERT from PHP, no Postgres release will save you. Batch into COPY first, then talk about versions.

UUID primary keys and the B-tree bill

Thu, 03 Aug 2023 21:35:00 +0300

The same table, the same rows, about 1.5 times the disk. That was the measurement after someone made UUIDv4 the primary key in InnoDB, and that is before counting secondary indexes.

Every new API wants UUIDs. Distributed generation, no coordination, nothing leaks about row counts. All true. Then the database quietly starts paying.

InnoDB clusters the table by primary key. Auto-increment inserts always land in the rightmost page, warm and predictable. Random UUIDs land anywhere in the tree: page splits everywhere, half-empty pages, and the set of hot pages becomes the whole index. Secondary indexes store the primary key in every entry, so each of them inherits the full 16 bytes too. In PostgreSQL the heap does not cluster, so the pain is smaller, but index bloat and lost locality are still there.

Outbox: events that do not lie

Wed, 07 Jun 2023 20:40:00 +0300

An email confirming an order that does not exist. A colleague showed me this one last week, together with the mirror bug: an order that exists and never got its email.

The code was the obvious kind. Commit the Doctrine transaction, then dispatch to the AMQP transport. Two systems, two writes, no shared transaction. Publish after commit and the process can die between them, event lost. Publish inside the transaction and the broker gets an event for data that may still roll back. Both orders are wrong, and under real load this fires weekly.

MERGE arrives in PostgreSQL 15

Thu, 27 Oct 2022 20:20:00 +0300

A supplier price list, loaded into staging_prices, then reconciled against prices. Since PostgreSQL 15 came out two weeks ago, that is one statement:

MERGE INTO prices p
USING staging_prices s ON p.sku = s.sku
WHEN MATCHED AND s.price IS NULL THEN
    DELETE
WHEN MATCHED AND p.price <> s.price THEN
    UPDATE SET price = s.price, updated_at = now()
WHEN NOT MATCHED THEN
    INSERT (sku, price) VALUES (s.sku, s.price);

Before 15 this was three statements in a transaction, or a stored procedure, or a loop in PHP. Now it is one statement that says what it does. Update changed rows, delete withdrawn ones, insert new ones. People coming from Oracle and SQL Server waited a decade for this. The conditional WHEN MATCHED AND ... branches are the real value. ON CONFLICT cannot express “delete when the source says so” at all.

JSONB or a normal table

Sat, 16 Jul 2022 10:05:00 +0300

Audit snapshots, product attributes, and a document with two hot keys. Three cases from one Postgres project, one question for each: column or JSONB.

I answer it by looking at queries, not at data.

Audit snapshots. When an order changes, we store the whole previous state. Nobody queries inside it. It is read as a blob, whole, rarely, by a human during an incident. Perfect JSONB. Normalizing it would mean a dozen tables for data nobody joins.

Backfill in batches, not in one UPDATE

Thu, 17 Mar 2022 22:40:00 +0300

Sixty million rows in orders, a new nullable column total_cents from expand-and-contract, and a Friday afternoon. The question is how to fill it.

The naive answer is one statement. UPDATE orders SET total_cents = ROUND(total * 100). On a test database it works. On production it holds row locks on the whole table for the whole run, writes one giant chunk of WAL, and the replica falls minutes behind. On MySQL the binlog gets the same present. One project I worked on learned this on that Friday. Since then I backfill only in batches.

Stop building running totals in PHP

Tue, 06 Oct 2020 15:40:00 +0300

Two hundred thousand rows over the wire to compute forty numbers. That was our monthly report on MySQL 5.7: fetch all transactions for the period, loop in PHP, accumulate a running balance, compare each row with the previous one, rank customers by volume. Classic 5.7 shape, because the database could not say “previous row” or “rank within group”.

We are finally moving that project to MySQL 8. Two years after GA, which by database standards is reckless haste. The first win had nothing to do with performance. We deleted PHP.

A deadlock is the database doing its job

Sat, 02 May 2020 20:30:00 +0300

Error 1213 in the logs, and the chat lights up: the database is broken. It is not. Two transactions locked rows in opposite order, A waits for B, B waits for A, and InnoDB did the only sane thing: picked a victim and killed it. The cycle is gone. This is a feature.

The classic shape is a money transfer. One request moves funds from account 1 to account 2, another from 2 to 1, both update the first account and then the second. Opposite order, instant cycle under load. Tests never show it, because tests do not run two of these in the same millisecond.

Cursor pagination is part of your API contract

Wed, 08 Jan 2020 21:30:00 +0300

Someone inserts a row while the client walks the pages, and the whole window shifts. Page two shows an item the client already saw on page one. Or an item falls between pages and the client never sees it. For a feed this is annoying. For an export or a sync endpoint this is a data loss bug that nobody can reproduce.

I used to think keyset pagination is a performance trick. Deep OFFSET is slow, keyset is fast, end of story. Now I think the performance part is the boring half. Offset pagination over a changing dataset lies to the client. That is the interesting half.

PostgreSQL 12 inlines your CTE

Fri, 18 Oct 2019 21:45:00 +0300

grep -rn "WITH " src/ was the first thing I ran after upgrading to PostgreSQL 12 this month. I was looking for a trick that had stopped working.

For years WITH was an optimization fence. The planner materialized the CTE first and only then ran the outer query. Everyone used this both ways. As a bug: you wrap a subquery into a CTE for readability, the planner stops pushing conditions inside, a fast query becomes a scan of half a table. As a feature: you write a CTE on purpose, to pin evaluation order and stop the planner from being creative. Half of the CTE advice on the internet is really advice about the fence.

N+1 hides on your laptop

Fri, 05 Jul 2019 22:35:00 +0300

Ten rows in the dev database, on the same machine, 0.1 ms per query. Ten extra queries is one millisecond. The page feels instant, the code ships. Production has a thousand rows and the database one network hop away. Round trip is about a millisecond even in a good datacenter. A thousand queries is a second of pure network waiting. Not slow SQL. Each query is fast. The plural is slow.

Transactional DDL is not portable

Mon, 18 Mar 2019 22:45:00 +0300

Statement three of a migration fails. Column name typo. On PostgreSQL the first two ALTERs roll back with it, the schema returns to the exact state before the migration, you fix the typo and run again. On MySQL the first two ALTERs are already permanent.

Same up() method, same php artisan migrate, same green output. The migration tool gives one abstraction over two very different databases. The abstraction covers syntax. It does not cover what happens on failure.

PostgreSQL 11: partitions grow up, JIT arrives

Fri, 26 Oct 2018 19:10:00 +0300

jit = on, restart, run the API test suite. Same numbers as before. That was my first evening with PostgreSQL 11, released last week, and it was the correct result.

JIT is the loud feature of this release and the misunderstood one. Postgres can now compile expression evaluation into machine code through LLVM. People read “compilation” and expect their endpoints to get faster. They will not. A primary key lookup takes a fraction of a millisecond. There is nothing in it worth compiling, and the compilation itself costs more than the whole query. JIT is for the other kind of query: an aggregate chewing through millions of rows, where the same expression runs so many times that generating machine code for it pays back.

Covering index and the price of SELECT *

Sun, 15 Jul 2018 12:35:00 +0300

Using index in the Extra column of EXPLAIN. That is the cheapest read MySQL can do, and you often get it almost for free.

SELECT user_id, created_at
FROM orders
WHERE user_id = 42
ORDER BY created_at DESC
LIMIT 20;

With an index on (user_id, created_at) everything the query needs is in the index leaves. The table is never touched. A composite index finds rows fast, a covering index answers the query on its own. Postgres calls this Index Only Scan, with one condition: the visibility map must be fresh, so a table that vacuum never visits quietly falls back to heap fetches.

MySQL 8.0: finally, modern SQL

Tue, 24 Apr 2018 22:15:00 +0300

@row := @row + 1. I have a report in production that stands on that trick, and the trick was never guaranteed to work. Evaluation order of user variables in SELECT is undefined, it just happened to behave. MySQL 8.0 went GA last week, and for me the release is about SQL. Window functions and CTEs, the things Postgres people stopped noticing years ago, are here.

The classic task: top three orders per customer. In 5.7 you had a self-join nobody could read a month later, or the variable trick. Now:

Keyset pagination instead of OFFSET

Mon, 19 Jun 2017 10:20:00 +0300

LIMIT 50 OFFSET 500000. Page 10001 of an events table, from a paginator someone wrote in an afternoon. Postgres has no shortcut to row 500000. It walks the index through half a million entries, fetches them, throws them away, and returns fifty. Page one is fast. Page ten thousand is slow, and every page after it is slower. Run EXPLAIN ANALYZE on both: the plans are identical, the numbers are not.

Parallel query in PostgreSQL 9.6

Fri, 30 Sep 2016 22:10:00 +0300

38 seconds to 11. Same query, same forty million rows, no index added, no SQL changed. The only difference is PostgreSQL 9.6, released yesterday, and one setting.

The query is a typical report: count and sum over an events table, grouped by day, three months of data. On 9.5 the plan is one process grinding through the table. On 9.6 with parallelism enabled:

Finalize HashAggregate
  ->  Gather
        Workers Planned: 4
        ->  Partial HashAggregate
              ->  Parallel Seq Scan on events

Four workers scan their own chunks, aggregate partially, the leader merges. Just more hands. For years we optimized SQL as if the database had exactly one worker per query. That assumption expired yesterday.

Column order in a composite index

Mon, 04 Jul 2016 23:25:00 +0300

Fifty rows. Three indexes on the table, one per column. Still slow. The query is the standard one from any multi-tenant application:

SELECT * FROM orders
WHERE tenant_id = ? AND status = ?
ORDER BY created_at DESC
LIMIT 50;

“Add an index” is the typical reaction, and one index on each column is the typical result. The number of indexes is not the point. The order of columns inside one index is.

Upsert in PostgreSQL 9.5

Tue, 12 Jan 2016 22:15:00 +0300

$row = $db->fetchOne('SELECT id FROM counters WHERE name = ?', array($name));
if ($row) {
    $db->execute('UPDATE counters SET value = value + 1 WHERE name = ?', array($name));
} else {
    $db->execute('INSERT INTO counters (name, value) VALUES (?, 1)', array($name));
}

Works on the laptop. In production two requests arrive in the same millisecond. Both SELECT, both see nothing, both INSERT. One dies with a duplicate key error. Or worse, there is no unique constraint, and now you have two rows and a bug report you cannot reproduce. The window between SELECT and INSERT is tiny, so it fires once a week, always for someone else.

JSON column is not a schema

Fri, 23 Oct 2015 22:40:00 +0300

ALTER TABLE users ADD profile JSON;

MySQL 5.7 went GA this week, and this line is now legal. Native JSON type: validation on insert, binary storage, functions to read paths. The question is not whether it works. It is what belongs in it.

Classic case: user profile with flexible metadata. Marketing wants a new field every other week. The old options were bad in familiar ways. Forty nullable columns and an ALTER for every idea. Or EAV, key-value rows, where every read is a self-join festival and nothing has a type.

Kohana ORM and my first real N+1

Wed, 12 Nov 2014 20:15:00 +0300

Fifty one queries. That is what the SQL panel of the Kohana profiler showed for one catalog page. Fifty posts on the page.

Nothing changed in the code. The page took two seconds because there was more data than in spring, and the code was written for spring.

$post->author->name in a loop. One query for the list of posts, then one more for the author of every post, inside the foreach. The line looks innocent, and that is the problem: lazy loading hides the price. You write the relation, you go home early, and the bill comes in six months.