Chao Yang

杨超, Research Scientist, Shanghai AI Lab.

yangchao.png

I am Research Scientist at Shanghai AI Lab (上海人工智能实验室), leading a Fundamental Large Model Safety & Decision Intelligence research group.

Recently, I have completed my postdoctoral fellowship under the guidance of Professor Yu Qiao , where my research focused on the security aspects of large-scale models. My work delved into the vulnerabilities and defense mechanisms associated with AI systems, particularly in the context of large language models and their applications.

Previously, I received my Ph.D. in Department of Computer Science and Technology at Tsinghua University in 2022, advised by Prof. Fuchun Sun and Prof. Huaping Liu.

My research interest includes Large Language Model Safety, Multi-modal Large Model, and Robotic Embodied Intelligence for Trustworthy AGI. Some of my current research keywords can be found below:

  • Agentic System: Harness Agent, Agentic RL, and Agent Safety.
  • Multimodal LLM: Modality Fusion, Multimodal alignment, VQA.
  • Embodied Robotics: Robotic Manipulation, Reinforcement Learning, Imitation Learning.

For Academic Cooperation, please feel free to email me at yangchao[at] pjlab [dot] org [dot] cn. For other matters, please contact me at yangchao9264 [at] 126 [dot] com or yangchaoemigmo [at] gmail [dot] com.

news

Apr 30, 2026 [ICML 2026 Spotlight] We are delighted that our paper “Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback” has been accepted to ICML 2026 as a Spotlight. Critique-GRPO integrates natural-language critiques with numerical rewards in online reinforcement learning, allowing a model to learn from both initial responses and critique-guided refinements. :sparkles:
Apr 30, 2026 [ICML 2026] “How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs” has been accepted to ICML 2026. The paper introduces FlowTracer, which models attention as a directed acyclic graph, traces answer-targeted information flow, and uses the resulting token credits to focus reinforcement-learning updates on consequential reasoning steps. :sparkles:
Apr 30, 2026 [ICML 2026] Our work “REFLECTOR: Internalizing Step-wise Reflection against Indirect Jailbreak” has been accepted to ICML 2026. REFLECTOR learns structured reflection through teacher-guided fine-tuning, then uses reinforcement learning to internalize autonomous, trajectory-level reflection against multi-step indirect attacks. :sparkles:
Apr 04, 2026 [Findings of ACL 2026] Our work on adaptive prompt optimization, “Adaptive Prompt Optimization for Open-Ended Tasks: Uncertainty Preference as a Secondary Signal,” has been accepted to Findings of ACL 2026. It uses semantic entropy to estimate task uncertainty and steer prompt selection toward higher-entropy candidates for creative tasks and lower-entropy candidates for conservative ones. :sparkles:
Jan 25, 2026 [ICLR 2026] We are pleased to share that our benchmark paper “MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers” has been accepted to ICLR 2026. Built on real MCP servers, the benchmark supports realistic multi-turn safety evaluation across attacks from the server, host, and user sides, including cross-server workflows. :sparkles:
Jan 25, 2026 [ICLR 2026] “Native Reasoning Models: Training Language Models to Reason on Unverifiable Data” has been accepted to ICLR 2026. The paper presents Native Reasoning Training, treating reasoning traces as latent variables so language models can learn from standard question-answer pairs without expert-written reasoning demonstrations or external verifiers. :sparkles:
Nov 08, 2025 [AAAI 2026] Our paper “SHADOW: Dynamic-Aware Credit Assignment Against Long-Horizon Tasks” has been accepted to AAAI 2026. It improves credit assignment for long-horizon language-model agents through dynamics-aware state grouping and a local dynamic advantage estimator. :sparkles:
Sep 18, 2025 [NeurIPS 2025] Our paper “VLMs Can Aggregate Scattered Training Patches” has been accepted to NeurIPS 2025. It identifies visual stitching, through which models can integrate benign-looking image patches scattered across training samples, exposing a way harmful content can bypass data moderation. :sparkles:
Jul 15, 2025 🎉 Big Project Release! We introduce SafeWork-R1, a cutting-edge multimodal reasoning model that demonstrates the coevolution of capabilities and safety. SafeWork-R1 :rocket: :sparkles:
May 16, 2025 [Findings of ACL 2025] Our paper “Adversarial Preference Learning for Robust LLM Alignment” has been accepted to Findings of ACL 2025. It introduces an iterative adversarial training framework that discovers input-specific attacks and uses closed-loop feedback to improve alignment robustness. :sparkles:
May 02, 2025 [ICML 2025] Our paper “Emergent Response Planning in LLMs” has been accepted to ICML 2025. Through probing, it studies how language-model hidden representations encode structural, content, and behavioral attributes of outputs beyond the next token. :sparkles:
May 02, 2025 [ICML 2025] Our paper “C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation” has been accepted to ICML 2025. It uses three lightweight agents and tree-structured rollout credit assignment to coordinate retrieval decisions, query generation, and document selection without modifying the retriever or language model. :sparkles:
Dec 08, 2024 We proposal a new law, AI 45°-Law toward trustworthy AGI! Arxiv Link :sparkles:
Sep 26, 2024 [NeurIPS 2024] Our paper “Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models” has been accepted to NeurIPS 2024. It aligns a frozen large language model at inference time by searching with the log-probability difference between smaller tuned and untuned models. :sparkles:
Sep 23, 2024 [Findings of EMNLP 2024] Our paper “Inference-Time Language Model Alignment via Integrated Value Guidance” has been accepted to Findings of EMNLP 2024. It guides decoding with implicit and explicit value functions at the token and chunk levels, enabling alignment without fine-tuning the target language model. :sparkles:
Jul 04, 2024 [ECCV 2024] Our paper “MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models” has been accepted to ECCV 2024. It provides a safety benchmark for evaluating multimodal language models against image-based manipulations across a broad set of harmful scenarios. :sparkles:
May 16, 2024 [ACL 2024 Outstanding Paper Award · Oral] Our paper “Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!” has been accepted to ACL 2024 as an oral presentation and received an Outstanding Paper Award. It presents a training-free attack that contrasts aligned and pretrained output distributions to expose a vulnerability in safety alignment. :sparkles:
May 16, 2024 [ACL 2024 Oral] Our paper “SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning” has been accepted to ACL 2024 as an oral presentation. It introduces a structure-based return and fine-grained rewards to capture dependencies among steps in structured reasoning and explanation. :sparkles:
May 16, 2024 [Findings of ACL 2024] Our paper “Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization” has been accepted to Findings of ACL 2024. It introduces MODPO, an RL-free extension of direct preference optimization that combines multiple alignment objectives with configurable weights. :sparkles:
May 02, 2024 [ICML 2024] Our paper “RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis” has been accepted to ICML 2024. It uses tree-structured multimodal code generation to decompose instructions into object-centric manipulation units and synthesize robot behaviors across platforms. :sparkles:
Apr 20, 2024 [IJCAI 2024 Survey Track] Our paper “Safety of Multimodal Large Language Models on Images and Text” has been accepted to the IJCAI 2024 Survey Track. It surveys evaluation datasets and metrics, attack techniques, and defenses for multimodal language-model safety across image and text inputs. :sparkles:
Mar 13, 2024 [NAACL 2024] Our paper “Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey” has been accepted to NAACL 2024. This survey organizes research on conversational LLM safety around attacks, defenses, and evaluation methods. :sparkles:
Feb 27, 2024 [CVPR 2024] Our paper “LLaMA-Excitor: General Instruction Tuning via Indirect Feature Interaction” has been accepted to CVPR 2024. It adds lightweight Excitor blocks as a bypass in self-attention, adapting attention to instructions while preserving pretrained capabilities across language-only and multimodal tuning. :sparkles:
Feb 27, 2024 [CVPR 2024] Our paper “VideoDistill: Language-aware Vision Distillation for Video Question Answering” has been accepted to CVPR 2024. It introduces a language-aware VideoQA framework with question-guided sparse frame sampling and visual refinement to focus answer generation on relevant visual evidence. :sparkles:
Dec 09, 2023 [AAAI 2024] Our paper “Critic-Guided Decision Transformer for Offline Reinforcement Learning” has been accepted to AAAI 2024. It combines a learned critic with Decision Transformer trajectory modeling to align target returns with the expected returns of actions in offline reinforcement learning. :sparkles:

selected publications

  1. CVPR2024
    VideoDistill: Language-aware Vision Distillation for Video Question Answering
    Bo Zou*, Chao Yang*, Yu Qiao, and 2 more authors
    arXiv preprint arXiv:2404.00973, 2024
    PDF
  2. CVPR2024
    LLaMA-Excitor: General Instruction Tuning via Indirect Feature Interaction
    Bo Zou*, Chao Yang*, Yu Qiao, and 2 more authors
    2024
    PDF
  3. ACL2024 Oral
    Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!
    Zhanhui Zhou, Jie Liu, Zhichen Dong, and 4 more authors
    arXiv preprint arXiv:2402.12343, 2024
  4. NAACL2024
    Attacks, defenses and evaluations for llm conversation safety: A survey
    Zhichen Dong, Zhanhui Zhou, Chao Yang+, and 2 more authors
    arXiv preprint arXiv:2402.09283, 2024
  5. AAAI2024
    Critic-guided decision transformer for offline reinforcement learning
    Yuanfu Wang, Chao Yang, Ying Wen, and 2 more authors
    arXiv preprint arXiv:2312.13716, 2023
  6. ECCV2024
    MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
    Xin Liu, Yichen Zhu, Jindong Gu, and 3 more authors
    arXiv preprint arXiv:2311.17600, 2024