vault / data / bigquery.md

BigQuery

Data & exploration 67 words 1 outgoing 1 backlinks

Where most experiment analysis in my posts actually ran: storage separated from compute, SQL over terabytes without owning a cluster. Costs follow bytes scanned, so the gotchas are all about reading less: partition by the date you filter on, cluster by the keys you group on, select only the columns you need, and dry-run a query to see its bytes before it bills you.

Reference: BigQuery docs.

LINKS IN THIS NOTE

[[ Metric semantic layer ]] A metric is more than a formula: population, event, time window and owner. Store definitions as metadata that people and programs can both read, and let a query engine compile them - the post's platform did this on BigQuery with...