1

Autonomous Evaluation and Refinement of Digital Agents

Jiayi Pan, Yichi Zhang, Nickolas Tomlin, Yifei Zhou, Sergey Levine, Alane Suhr. COLM 2024 / ⭐️ MAR Workshop @ CVPR 2024 Best Paper. …

Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, Alane Suhr
PDF Cite Code Twitter
Autonomous Evaluation and Refinement of Digital Agents
ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL

Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, Aviral Kumar. ICML 2024.

We present ArCHer, a new framework of multi-turn RL …
Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, Aviral Kumar
PDF Cite Code Project
ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
Inversion-Free Image Editing with Natural Language

Sihan Xu*, Yidong Huang*, Jiayi Pan, Ziqiao Ma, Joyce Chai. CVPR 2024.

We present an inversion-free editing (InfEdit) method which …
Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, Joyce Chai
PDF Cite Code Project
Inversion-Free Image Editing with Natural Language
Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?

Yichi Zhang, Jiayi Pan, Yuchen Zhou, Rui Pan, Joyce Chai. EMNLP 2023.

Do Vision-Language Models, an emergent human-computer …
Yichi Zhang, Jiayi Pan, Yuchen Zhou, Rui Pan, Joyce Chai
PDF Cite Code Project Scientific American
Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models

Ziqiao Ma*, Jiayi Pan*, Joyce Chai. ⭐️ ACL 2023 Outstanding Paper.

We study grounding and bootstrapping in open-world language …
Ziqiao Ma, Jiayi Pan, Joyce Chai
PDF Cite Code
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models
Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification

Jiayi Pan, Glen Chou, Dmitry Berenson. ICRA 2023.

We present a learning-based approach to translate from natural language commands …
Jiayi Pan, Glen Chou, Dmitry Berenson
PDF Cite Code Project
Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification
DANLI: Deliberative Agent for Following Natural Language Instructions

Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keunwoo Peter Yu, Yuwei Bao, Joyce Chai. EMNLP 2022, …

Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keunwoo Peter Yu, Yuwei Bao, Joyce Chai
PDF Cite Code
DANLI: Deliberative Agent for Following Natural Language Instructions