https://useinov.com/tags/observability/ Recent content in Observability on Murat Useinov Hugo en-us Mon, 10 Mar 2025 22:30:00 +0300 https://useinov.com/posts/2025/observability-budget/ Mon, 10 Mar 2025 22:30:00 +0300 https://useinov.com/posts/2025/observability-budget/ <p>Last year we turned tracing on everywhere. This year the telemetry invoice is a line item a manager asks about by name. The dashboards did not get more useful. More data, same confusion at 2 a.m.</p> <p>Time to design instead of collect.</p> <p>The design starts from the questions I actually ask when the pager goes off. In practice there are four: is the latency of key endpoints normal, did the error rate jump, how far behind are the queues, is the database saturated. That is the core set. Metrics for these must be cheap, always on, with alerts. Everything else can be sampled traces I pull up on demand.</p> https://useinov.com/posts/2024/one-trace/ Thu, 08 Feb 2024 19:50:00 +0300 https://useinov.com/posts/2024/one-trace/ <p>A request creates a job. The job calls an external API and writes to the database. Something in this chain is slow, and the logs tell four disconnected stories.</p> <p>Grepping a request id across services is our folklore. OpenTelemetry finally makes the boring standard version practical in PHP.</p> <p>The core is small. One trace id for the whole causal chain, a span id for every operation, and one rule: pass the context along. Over HTTP it is a single header:</p>