Jerry Tworek Listed Autoresearch's Open Problems. Then 18 Teams Started Building.

The build-day recap from the AGI House Auto-Research Summit & Build Session, July 18th in Hillsborough. For the ideas, read the flagship memo and our interview with Jerry Tworek; for the technical landscape going in, the pre-event research brief.

Our brief before the event made one claim: the bottleneck in autoresearch isn't how many experiments you can run — models execute tirelessly — it's idea diversity, knowing what's worth trying. Then Jerry Tworek's keynote put up an open-problems slide with diversity/exploration listed first, ahead of better models and test-time training. Eighteen teams spent the afternoon building against exactly that.

The winners, mapped to the open problems

🏆 Vibes Lab for Auto Research (Yaroslav Volovich) — injects human research judgment into scientific autoresearch loops. The most direct attack on the day's central problem: the loop can execute, but taste about which hypotheses deserve compute is still scarce, and Vibes Lab treats that judgment as a first-class input rather than a bug.

🏆 AutoDesign (Ondřej Romancov) — a self-improving design harness. Rather than optimizing a solution, it hill-climbs the scaffold itself — the pattern Introspection calls the outer loop, applied to design work.

🏆 BetaChip (Harish Santhanalakshmi Ganesan) — a real-time RL agent for optimized chip design. Squarely in the lineage of the "famous loops" from Tworek's keynote: a verifiable domain where the score is honest and the loop can run hot.

🏆 autoprime (Dennis Noll) — a search for humanity's "most valuable number." The fun one, and quietly the purest exploration experiment of the four: an open-ended objective that forces the question of what the loop should even want.

Congratulations as well to our finalist teams — Andy Wang, Subhan Poudel, Femi Ositade, Michael Zuo, Jeffrey Zhou, Julian Quick, Saeejith Nair, Artem Shitov, Dwijen Patel, Katie Kim, Pierre Bouquet, Hugo Couvreur, and Subrata Goswami. Demos from all participating teams are coming to the AGI House platform.

The stack the co-hosts brought

Three of our co-hosts are building what amounts to a shared architecture at different layers — worth reading as one stack:

• Compute layer — Daytona. Sandboxes for serving and training agents, on bare metal rather than a hyperscaler. The insight worth keeping: in a large RL run cycling thousands of sandboxes, spin-up latency is wasted cluster compute. Sandbox infrastructure isn't plumbing; it's the loop's clock speed.
• Learning layer — NeoSigma (Gauri Gupta, Ritvik Kapila). Agents that improve from experience, with rewards drawn from production signal — user feedback, telemetry, business outcomes — because "the hardest failures appear in production, not on public benchmarks." They've open-sourced an autoresearch harness with self-improvement experiments on tau-bench.
• Orchestration layer — Introspection (Roland Gavrilescu, ex-xAI). A dual-loop design: the inner loop turns signals into work; the outer loop distills what was learned back into the agent's recipe. Traces become patterns, patterns become judges, judges grade experiments, experiments promote new versions to prod — all built on Pi, which they pitch as "the Linux of harnesses." Their standing challenge to builders is a good one-liner for the whole vertical-agent frontier: can you encode taste — judgment that no unit test can grade — in a recipe that agents improve over time?

Three companies, one architecture: deploy → signal → judge → improve → redeploy. The flagship memo unpacks why that convergence matters.

Kilian's challenge to the room

Kilian Lieret (Meta FAIR) built SWE-bench and SWE-agent — and came to argue that his own paradigm is aging out. SWE-bench is now as old as HumanEval was when SWE-bench launched, and the standard responses — harder tasks, stricter grading — saturate fast. The jump he wants is from task-oriented benchmarks (fix this bug; unit tests grade it) to goal-oriented ones (make money, cure a disease), where scoring itself is the unsolved problem.

His existence proof is CodeClash: a PvP benchmark where models write codebases that compete against each other in arenas — trading simulators, poker, resource games — over multiple rounds, with agents free to maintain memories and build tools between rounds. Unsaturable by construction, and a direct test of self-directed improvement. The humbling result: relatively simple human solutions beat roughly 90% of model submissions, even from very good models. Out of distribution, "models just kind of flop around."

That gap — between task competence and goal competence — is where the next benchmark, and possibly the next company, lives. His full argument is in his research spotlight.

Thank you, and what's next

Thanks to our keynotes Jerry Tworek, Pushmeet Kohli, Ivan Nardini, and Kilian Lieret; to our co-hosts at Daytona (Muhammad Annas Hashmi), NeoSigma (Gauri Gupta, Ritvik Kapila), and Introspection (Roland Gavrilescu); and to Jiachen (Amber) Liu, Jinjing Liang, and Serge Didenko Vasylechko for making the day run.

Recordings — including Tworek's keynote, his full interview, and Lieret's spotlight — plus team demos are going up on the AGI House platform. If you're building or researching autoresearch loops — especially the verifier side — we want to hear from you before the next one.

Read the rest of the series: the flagship memo on the day's big ideas and debates, the Jerry Tworek deep-dive, and the pre-event research brief.