Good evening from Greece! While I’m catching up on the Australian Grand Prix, I wanted to share this great thread that analogizes post-training optimizations in LLMs to in-season car improvements in Formula 1:

Consider F1, most of the teams show up to the beginning of the year with a new chassis and engine. Then, they spend all year on aerodynamics and systems changes (a minor over simplification), and can dramatically improve the performance of the car. The best F1 teams improve way more during a season.

The best post-training teams extract a ton of performance in a very short time frame. The set of techniques is everything after the end of most of pretraining. It includes “mid-training” like Annealing / high-quality end of pre-training, instruction tuning, RLVR, preference-tuning, etc.

Read the full thread here.

via Nathan Lambert on Bluesky