Open Source Contributions
Benchmarks, evaluation, and RL infrastructure I contribute to, grouped by project.
Agent-Native Research Artifact. I maintain an open format for publishing research so that agents can read, verify and build on it.
Evaluation research. I work on detecting reward hacking in agent benchmarks, and co-authored Open-Endedness Bench and the auto-research paper.
RL evaluation infrastructure. Rollout snapshotting, branching and replay for RL training runs.
BenchGuard. A corpus of real agent trajectories that game benchmark verifiers, used to build and test cheating detectors. I contributed 39 verified cases and reviewed others’ submissions.
Terminal-Bench-Science. Perturb-seq and drug-combination environments, each with a deterministic statistical oracle.
Harbor adapter. I ported all 33 ContractBench tasks into Harbor and checked each one against its oracle.
Persona-simulation tasks. A DeepTutor tutoring-chat task and an Open Library book-choice web task, each run with a batch of personas.
AgentPersona-Bench. Persona-adherence tasks across survey, chat, web and desktop-app settings, judged from what the agent actually produces.
A hardened replica of the Healthline website for web agents: 67 articles and 20 WebVoyager tasks.
Three paper-reproduction tasks for Agents’ Last Exam (Othello-GPT probing, DanceTrack tracking and ContractBench), and a self-improvement task for RSI-Exam built on ContractBench.
Robust data-parallel load balancing. Made sequence-length balancing robust to batch sizes that do not divide evenly across ranks, so RL training no longer aborts when invalid rollouts are filtered out.
Bounded-uneven partitioning. Extended the balancer to keep per-rank token load even while bounding the sample-count skew between ranks to one.
Reward pipeline. Aligned the reward_extra_info keys across a batch, so samples that report different extra reward fields can still be aggregated in diffusion-model RL, with tests.