vault / exp / thompson.md

Thompson sampling

Experimentation 55 words 1 outgoing 1 backlinks

Sample one possible reward rate from each option’s posterior, then choose the option with the highest sampled rate. Uncertainty supplies exploration: an option with little evidence can still win a draw. In the production pattern described in the post, repeated draws become allocation weights written by a batch job.

Read the full example: related post.

LINKS IN THIS NOTE

[[ Multi-armed bandits ]] A bandit shifts allocation while learning which option performs better. That trades some fixed-test simplicity for less traffic spent on weak options. In the coupon example, Thompson sampling updates a traffic split in a batch job; request handling uses that...