vault / exp / metrics-layer.md

Metric semantic layer

Experimentation 56 words 2 outgoing 2 backlinks

A metric definition includes more than a formula: its population, event, time window and owner matter too. The platform I describe stored those definitions as metadata and used a BigQuery query engine to compute them. Reusing one definition helps only when teams can find it and know who maintains it.

Read the full example: related post.

LINKS IN THIS NOTE

[[ A/B testing ]] Random assignment creates comparable groups; a shared metric defines what is compared. A p-value describes how surprising the data would be under a specified null hypothesis. It is not the probability that the hypothesis is true. Decide the measurement and... [[ Self-service tooling ]] Self-service is not merely putting a form over a backend. Experiment creators need definitions they can understand and reuse without copying SQL. The metric-layer post treats discoverability and ownership as part of the platform: a catalogue is useful only when...