vault / exp / bandits.md

Multi-armed bandits

Experimentation 55 words 2 outgoing 3 backlinks

A bandit shifts allocation while learning which option performs better. That trades some fixed-test simplicity for less traffic spent on weak options. In the coupon example, Thompson sampling updates a traffic split in a batch job; request handling uses that split rather than sampling a posterior on every request.

Read the full example: related post.

LINKS IN THIS NOTE

[[ Thompson sampling ]] Sample one possible reward rate from each option's posterior, then choose the option with the highest sampled rate. Uncertainty supplies exploration: an option with little evidence can still win a draw. In the production pattern described in the post, repeated... [[ A/B testing ]] Random assignment creates comparable groups; a shared metric defines what is compared. A p-value describes how surprising the data would be under a specified null hypothesis. It is not the probability that the hypothesis is true. Decide the measurement and...