https://useinov.com/tags/ops/ Recent content in Ops on Murat Useinov Hugo en-us Wed, 07 Aug 2019 18:05:00 +0300 https://useinov.com/posts/2019/kubernetes-probes-php/ Wed, 07 Aug 2019 18:05:00 +0300 https://useinov.com/posts/2019/kubernetes-probes-php/ <p>Route <code>/health</code>, returns <code>ok</code>, everyone happy. That is the first version of every health endpoint on the project we are moving to Kubernetes. Then somebody pastes the same path into <code>livenessProbe</code> and <code>readinessProbe</code>, and a thirty second database hiccup becomes a long evening.</p> <p>The two probes ask different questions.</p> <p>Readiness asks: should this pod get traffic right now. Here it is correct to check dependencies. Database unreachable, cache cold, migrations still running: answer no. Kubernetes takes the pod out of the Service, traffic goes to the others, the pod returns when the world improves. Failing readiness is cheap and reversible. A polite &ldquo;not now&rdquo;.</p> https://useinov.com/posts/2019/redis-maxmemory/ Tue, 02 Apr 2019 15:10:00 +0300 https://useinov.com/posts/2019/redis-maxmemory/ <p>Users logged out at random. Not all, not always. Only sometimes, only in the afternoon. Afternoon is when traffic peaks. Traffic peaks fill the cache. The cache lived in the same Redis as the sessions, the instance hit <code>maxmemory</code>, and the eviction policy was <code>allkeys-lru</code>. Redis did exactly what we asked: threw away the least recently used keys, and some of them were sessions of people who went for lunch.</p>