Fit your own scaling law on a laptop

A fixed compute budget forces a trade: a bigger model sees fewer tokens. So the experiment is an IsoFLOP sweep, the method behind Chinchilla's second approach: at each budget C, train several sizes N for D = C / 6N tokens, and look for the bottom of the loss curve. One budget gives you the best size for that budget; three budgets give you how the best size moves, which is the scaling law. The decision that matters most is choosing sizes and budgets so every curve has a minimum inside the sizes tried, and refusing to fit a budget where it doesn't.