Jicheng (Jeremy) Wang

Training and evaluating LLM agents rest on the same thing: a signal that says whether the agent did the task. The question I keep returning to is:

How can we train LLM agents on feedback we can verify, and know that their gains hold beyond the benchmark?

A flaw in how we measure an agent becomes a flaw in what it learns, so I treat evaluation and training as one problem: a score is worth trusting only if the evidence behind it can be checked, and a model is worth training only on signals that pass that test, during training and, as I am exploring now, at test time.

I was fortunate to be advised by Prof. Hao Chen (IEEE Fellow) in the Security Lab at UC Davis. My work there began with a narrower question: can a model learn from what its environment reports, such as the execution of its own code, rather than from labels? Once the model acts as an agent, the bottleneck moves to the signal itself: an agent can reach the right answer while ignoring what its tools told it, and a score can be gamed. The questions that followed are the ones I still work on: how can an evaluation check the evidence behind a result, not just the result? And once that evidence exists, how much more can it do than grade: can it repair an action that has been hijacked, choose among programs by executing them, and supervise a model on its own rollouts? As an undergraduate, I worked on natural language understanding with Prof. Min Peng at the Wuhan University AI Lab. I currently work at AWS SageMaker AI, where I build evaluation for LLM agents that fine-tune and deploy models.

I am always glad to talk about research and explore collaborations. Feel free to reach out by email or schedule a meeting.