Ruiyi Zhang

Research Scientist

San Jose

Ruiyi Zhang is a Research Scientist at Adobe Research. His research interests include (interactive) machine learning, reinforcement learning (RL) and natural language processing (NLP), especially the intersection of them. Before that, Ruiyi obtained a Ph.D. advised from the Department of Computer Science, Duke University. His thesis is about Uncertainty Estimation in Deep Reinforcement Learning and his advisor is Professor Lawrence Carin.

 

Please refer to his personal webpage for more information.

 

Publications

Click, Type, Repeat: A Comprehensive Survey on GUI Agents

Nguyen, Dang., Chen, Jian., Wang, Yu., Wu, Gang., Park, Namyong., Hu, Zhengmian., Lyu, Hanjia., Wu, Junda., Aponte, Ryan., Xia, Yu., Li, Xintong., Shi, Jing., Chen, Hongjie., Lai, Viet., Xie, Zhouhang., Kim, Sungchul., Zhang, Ruiyi., Yu, Tong., Tanjim, Mehrab., Ahmed, Nesreen., Mathur, Puneet., Yoon, David., Yao, Lina., Kveton, Branislav., Kil, Jihyung., Nguyen, Thien., Bui, Trung., Zhou, Tianyi., Rossi, Ryan., Dernoncourt, Franck. (Aug. 1, 2025)

ACL 2025 Findings

From Selection to Generation: A Survey of LLM-based Active Learning

Xia, Yu., Mukherjee, Subhojyoti., Xie, Zhouhang., Wu, Junda., Li, Xintong., Aponte, Ryan., Lyu, Hanjia., Barrow, Joe., Chen, Hongjie., Dernoncourt, Franck., Kveton, Branislav., Yu, Tong., Zhang, Ruiyi., Gu, Jiuxiang., Ahmed, Nesreen., Wang, Yu., Chen, Xiang., Deilamsalehy, Hanieh., Kim, Sungchul., Hu, Zhengmian., Zhao, Yue., Lipka, Nedim., Yoon, David., Huang, Ting-Hao., Wang, Zichao., Mathur, Puneet., Pal, Soumyabrata., Mukherjee, Koyel., Zhang, Zhehao., Park, Namyong., Nguyen, Thien., Luo, Jiebo., Rossi, Ryan., McAuley, Julian. (Aug. 1, 2025)

ACL 2025

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes

Gallegos, Isabel., Aponte, Ryan., Rossi, Ryan., Barrow, Joe., Tanjim, Mehrab., Yu, Tong., Deilamsalehy, Hanieh., Zhang, Ruiyi., Kim, Sungchul., Dernoncourt, Franck., Lipka, Nedim., Owens, Deonna., Gu, Jiuxiang. (May. 4, 2025)

NAACL 2025

Personalizing Data Delivery: Investigating User Characteristics and Enhancing LLM Predictions

Luera, Reuben., Rossi, Ryan., Dernoncourt, Franck., Siu, Alexa., Kim, Sungchul., Yu, Tong., Zhang, Ruiyi., Chen, Xiang., Lipka, Nedim., Zhang, Zhehao., Kim, Seon., Lee, Tak. (May. 2, 2025)

WWW 2025

SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding

Chen, Jian., Zhang, Ruiyi., Zhou, Yufan., Yu, Tong., Dernoncourt, Franck., Gu, Jiuxiang., Rossi, Ryan., Chen, Changyou., Sun, Tong. (Apr. 28, 2025)

ICLR 2025

Token-level adversarial prompt detection based on perplexity measures and contextual information

Hu, Zhengmian., Wu, Gang., Mitra, Saayan., Zhang, Ruiyi., Sun, Tong., Huang, Heng., Swaminathan, Vishy. (Mar. 5, 2025)

ICLR Workshop on Building Trust in Language Models and Applications

GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration

Fan, Yue., Zhao, Handong., Zhang, Ruiyi., Shen, Yu., Wang, Xin., Wu, Gang. (Jan. 23, 2025)

arXiv

KaPQA: Knowledge-Augmented Product Question-Answering

Eppalapally, Swetha., Dangi, Daksh., Bhat, Chaithra., Gupta, Ankita., Zhang, Ruiyi., Bagga, Karishma., Yoon, David., Lipka, Nedim., Rossi, Ryan., Dernoncourt, Franck. (Aug. 16, 2024)

ACL 2024 Workshop on Knowledge Augmented Methods for NLP

AutoDAN: interpretable gradient-based adversarial attacks on large language models

Zhu, Sicheng., Zhang, Ruiyi., An, Bang., Wu, Gang., Barrow, Joe., Wang, Zichao., Huang, Furong., Nenkova, Ani., Sun, Tong. (Aug. 1, 2024)

COLM 2024

Customization Assistant for Text-to-image Generation

Zhou, Yufan., Zhang, Ruiyi., Gu, Jiuxiang., Sun, Tong. (Jun. 17, 2024)

CVPR 2024

TRINS: Towards Multimodal Language Models that Can Read

Zhang, Ruiyi., Zhang, Yanzhe., Chen, Jian., Zhou, Yufan., Gu, Jiuxiang., Chen, Changyou., Sun, Tong. (Jun. 17, 2024)

CVPR 2024

Bias and Fairness in Large Language Models: A Survey

Gallegos, Isabel., Rossi, Ryan., Barrow, Joe., Tanjim, Mehrab., Kim, Sungchul., Dernoncourt, Franck., Yu, Tong., Zhang, Ruiyi., Ahmed, Nesreen. (Jun. 11, 2024)

Computational Linguistics 2024

Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer in Prompt Tuning

Xie, Kaige., Yu, Tong., Wang, Haoliang., Wu, Junda., Zhao, Handong., Zhang, Ruiyi., Mahadik, Kanak., Nenkova, Ani., Riedl, Mark. (Mar. 17, 2024)

The Conference of the European Chapter of the Association for Computational Linguistics (EACL)

InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding

Wu, Junda., Yu, Tong., Wang, Rui., Song, Zhao., Zhang, Ruiyi., Zhao, Handong., Lu, Chaochao., Li, Shuai., Henao, Ricardo. (Dec. 10, 2023)

NeurIPS

VaQuitA: Enhancing alignment in llm-assisted video understanding

Wang, Yizhou., Zhang, Ruiyi., Wang, Haoliang., Bhattacharya, Uttaran., Fu, Yun., Wu, Gang. (Dec. 4, 2023)

arXiv

Learning Navigational Visual Representations with Semantic Map Supervision

Hong, Yicong., Zhou, Yang., Zhang, Ruiyi., Dernoncourt, Franck., Bui, Trung., Gould, Stephen., Tan, Hao. (Oct. 6, 2023)

ICCV 2023

TiGAN: Text-Based Interactive Image Generation and Manipulation

Zhou, Yufan., Zhang, Ruiyi., Gu, Jiuxiang., Tensmeyer, Chris., Yu, Tong., Chen, Changyou., Xu, Jinhui., Sun, Tong. (Jun. 28, 2022)

Proceedings of the AAAI Conference on Artificial Intelligence

Towards Language-Free Training for Text-to-Image Generation

Zhou, Yufan., Zhang, Ruiyi., Chen, Changyou., Li, Chunyuan., Tensmeyer, Chris., Yu, Tong., Gu, Jiuxiang., Xu, Jinhui., Sun, Tong. (Jun. 19, 2022)

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition