Publications

Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, Alane Suhr (2024). Autonomous Evaluation and Refinement of Digital Agents. COLM 2024.

PDF Cite Code Twitter

Xingyao Wang, Jiayi Pan, Graham Neubig (2024). OpenHands: An Open Platform for AI Software Developers as Generalist Agents. Preprint.

PDF Cite Code Twitter

Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, Sergey Levine (2024). Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning. NIPS 2024.

Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, Aviral Kumar (2024). DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning. Preprint.

PDF Cite Code Twitter

Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, Aviral Kumar (2024). ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL. ICML 2024.

PDF Cite Code Project

Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, Joyce Chai (2023). Inversion-Free Image Editing with Natural Language. CVPR 2024.

PDF Cite Code Project

Yichi Zhang, Jiayi Pan, Yuchen Zhou, Rui Pan, Joyce Chai (2023). Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?. In EMNLP 2023.

PDF Cite Code Project Press

Ziqiao Ma, Jiayi Pan, Joyce Chai (2023). World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models. In ACL 2023.

Yichi Zhang, Jianing Yang, Keunwoo Yu, Yinpei Dai, Shane Storks, Yuwei Bao, Jiayi Pan, Nikhil Devraj, Ziqiao Ma, Joyce Chai (2023). SEAGULL: An Embodied Agent for Instruction Following through Situated Dialog. In Alexa Prize SimBot Challenge Proceedings.

Jiayi Pan, Glen Chou, Dmitry Berenson (2022). Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification. International Conference on Robotics and Automation (ICRA) 2023.

PDF Cite Code Project

Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keunwoo Peter Yu, Yuwei Bao, Joyce Chai (2022). DANLI: Deliberative Agent for Following Natural Language Instructions. In EMNLP 2022.