vault / exp / self-service.md

Self-service tooling

Experimentation 56 words 2 outgoing 1 backlinks

Self-service is not merely putting a form over a backend. Experiment creators need definitions they can understand and reuse without copying SQL. The metric-layer post treats discoverability and ownership as part of the platform: a catalogue is useful only when the shared path is easier than making another private definition.

Read the full example: related post.

LINKS IN THIS NOTE

[[ Metric semantic layer ]] A metric definition includes more than a formula: its population, event, time window and owner matter too. The platform I describe stored those definitions as metadata and used a BigQuery query engine to compute them. Reusing one definition helps only... [[ A/B testing ]] Random assignment creates comparable groups; a shared metric defines what is compared. A p-value describes how surprising the data would be under a specified null hypothesis. It is not the probability that the hypothesis is true. Decide the measurement and...