概要
公開日: 2026-05-12
翻訳生成に失敗したため、原文をそのまま保存しています。
原文
May 12, 2026 Research What Parameter Golf taught us Lessons from 1,000+ participants, 2,000+ submissions, and an open machine learning challenge shaped by coding agents. Loading… Share We launched Parameter Golf to engage and support the machine learning research community in exploring a new, tightly constrained machine learning problem. We wanted the challenge to be interesting enough to reward real technical creativity, while remaining conceptually simple and easy to verify. Participants had to minimize held-out loss on a fixed FineWeb dataset while staying within a 16 MB artifact limit, including both model weights and training code, and a 10-minute training budget on 8×H100s. We provided a baseline, dataset, and evaluation scripts so participants could fork the repo, improve the model, and submit their results through GitHub. Over the course of eight weeks, we received more than 2,000 submissions from over 1,000 participants. We were impressed by the technical breadth, creativity, and rule-bending across the submissions, from careful optimizer tuning and quantization work to new modeling ideas and test-time training. One of the most exciting parts of the challenge was seeing how widely participants used AI coding agents. Agents helped lower the cost of experimentation, made it easier for more people to participate, and changed the pace of the competition. They also created new challenges for submission review, attribution, and scoring. The challenge also became a meaningful talent discovery surface for us. That was one of our goals for Parameter Golf, and it was a useful signal that open-ended technical challenges can reveal exceptional machine learning taste and persistence. In this post, we highlight some of the submissions we found surprising and interesting, and share what we learned from running a coding contest in the age of powerful AI agents. Technical impressions Record track We judged and independently reproduced each submission on the record-track leaderboard, and verified that each submission was record-breaking at the time it was submitted. Several themes stood out. Training optimization Some of the strongest results came from careful tuning of existing components. Submission Contributor Technique Why it mattered #60 @notapplica Combined prior wins from #50 , #42 , and likely #39 , then made a deeper model work with Muon weight decay, spectral embedding initialization, residual-mix scheduling, and compiled evaluation. A strong example of disciplined leaderboard work: identifying which existing improvements matter and combining them cleanly. Quantization Several submissions pushed hard on compression and export. Submission Contributor Technique Why it mattered #414 @signalrush Used GPTQ-lite to quantize weights after training. The first leaderboard submission to successfully use GPTQ-lite, leading to better evaluation. #1060 @dexhunter Built on #634 by @raahilshah to successfully use full Hessian GPTQ. Extended earlier quantization work into a stronger compression path. Test-time and evaluation strategies Some submissions pushed the boundary between model improvement and evaluation strategy. These approaches were valid under the rules, but they required careful review from us as organizers. Submission Contributor Technique Why it mattered #77 @samacqua Used score-first, per-document LoRA test-time training: score first, adapt only on already-scored chunks, and reset at document boundaries. Pushed the boundary between model improvement and evaluation strategy while staying reviewable under the rules. #1019 @abaybektursun Used self-generated GPTQ calibration: generate calibration text from the trained model, then build GPTQ Hessians from those activations. A creative calibration strategy that required careful review from organizers. New modeling and data ideas A few submissions introduced modeling or data ideas that were especially creative. Submission Contributor Technique Why it mattered #1729 @romeerp Introduc