sampling

Tracing is the only telemetry that answers where the time went across services. The mechanism is context propagation, and the reason teams fail at it is that one uninstrumented hop breaks the chain.

The instinct is to delete old logs. That is backwards: the waste lives in what you ingest, not how long you keep it. This answer walks the levers in the order that actually pays.

Choose what evidence must survive, then price the collection path. A tail policy can only inspect traces that reach it; low SDK sampling changes the guarantee before the collector sees a span.

Sampling keeps proportions and loses rare individuals; filtering keeps chosen classes complete and loses whatever you misclassified. Decide retention at write time, keep counters always, rehydrate the rest.

Agent per node, gateway per cluster, and one decision that shapes everything else: tail sampling needs every span of a trace in one place, which is why the topology has a tier that looks redundant until you need it.