Yunxin Li

Ph.D. Student

About Me

I'm a Ph.D. Student from Harbin Institute of Technology, Shenzhen and advised by Prof. Baotian Hu. I obtained a Bachelor of Science degree from Harbin Institute of Technology, and a Master of Engineering degree from Harbin Institute of Technology, Shenzhen. Long-term cooperation with Dr. Lin Ma, Meituan, Beijing; Prof. Wenhan Luo, HKUST; Dr. Longyue Wang, Alibaba, Group; Yuxiang Wu , University College London.

Research Internship: ByteDance Doubao (Seed) Team

Past Research Interns:

Tencent PCG and advised by Dr. Wenhan Luo
Tencent AILab and advised by Dr. Longyue Wang

The long-term goal of my research is to help humans with more capable artificial intelligence. Dream of building an intelligent metaverse and interesting research directions including:

Multimodal Collaborative Reasoning
Video Understanding and Generation
Multimodal Agent
Embodied Intelligence

I am actively seeking cooperators who share my interest in applying LLMs to Mathematics, Complex Logical Problems, and Multimodal Scenarios, with the goal of exploring these areas together.

Selected Publications

Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation

Yunxin Li, Haoyuan Shi, Baotian Hu, Longyue Wang, Jiashun Zhu, Jinyi Xu, Zhen Zhao, Min Zhang PDF Code

SIGGRAPH Asia, 2024.

VideoVista: A Versatile Benchmark for Video Understanding and Reasoning

Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, Min Zhang PDF Web Code

arXive, 2024.

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts

Yunxin Li, Shenyuan Jiang, Baotian Hu, Longyue Wang, Wanqi Zhong, Wenhan Luo, Lin Ma, Min Zhang PDF Web Code

arXive, 2024.

Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment

Yunxin Li, Xinyu Chen, Baotian Hu, Haoyuan Shi, Min Zhang PDF Code

ACL 2024 Main Conference.

VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context

Yunxin Li, Baotian Hu, Haoyuan Shi, Wei Wang, Longyue Wang, Min Zhang PDF Code

ICML, 2024.

LMEye: An Interactive Perception Network for Large Language Models

Yunxin Li, Baotian Hu, Xinyu Chen, Lin Ma, Yong Xu, Min Zhang PDF Code

IEEE Transactions on Multimedia (TMM), 2024.

A Multimodal In-Context Tuning Approach for E-Commerce Product Description Generation

Yunxin Li, Baotian Hu, Wenhan Luo, Lin Ma, Yuxin Ding, Min Zhang PDF Code

LREC-COLING, 2024.

Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs

Yunxin Li, Baotian Hu, Wei Wang, Xiaochun Cao, Min Zhang PDF

arXive, 2023.

A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answering

Yunxin Li*, Longyue Wang*, Baotian Hu, Xinyu Chen, Wanqi Zhong, Chenyang Lyu, Min Zhang PDF Code

Technical Paper, 2023.

Training Multimedia Event Extraction With Generated Images and Captions

Zilin Du, Yunxin Li, Xu Guo, Yidan Sun, Boyang Li PDF Code

ACM on Multimedia (ACM MM), 2023.

A Neural Divide-and-Conquer Reasoning Framework for Image Retrieval from Linguistically Complex Text

Yunxin Li , Baotian Hu, Yuxin Ding, Lin Ma, Min Zhang PDF Code

ACL 2023 Main Conference.

A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues

Yunxin Li , Baotian Hu, Xinyu Chen, Yuxin Ding, Lin Ma, Min Zhang PDF Code

ACL 2023 Main Conference.

Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations

Qian Yang*, Yunxin Li* , Baotian Hu, Lin Ma, Yuxing Ding, Min Zhang PDF Code

ACM on Multimedia (ACM MM), 2022.

Medical Dialogue Response Generation with Pivotal Information Recalling

Yu Zhao*, Yunxin Li*, Yuxiang Wu, Baotian Hu, Qingcai Chen, Xiaolong Wang, Yuxin Ding, Min Zhang PDF Code

SIGKDD, 2022.

Fast and Robust Online Handwritten Chinese Character Recognition with Deep Spatial & Contextual Information Fusion Network

Yunxin Li , Qian Yang, Qingcai Chen, Baotian Hu, Xiaolong Wang, Yuxin Ding, Lin Ma PDF

IEEE Transactions on Multimedia (TMM), 2022.