Shuang Qiu I am an Assistant Professor in the Department of Systems Engineering and Department of Computer Science (secondary) at City University of Hong Kong since 2025. Prior to CityU, I did research in the Department of Mathematics at the Hong Kong University of Science and Technology as a research assistant professor working with Prof. Tong Zhang (张潼) and in the Booth School of Business at the University of Chicago as a postdoc researcher working with Prof. Mladen Kolar and Prof. Zhaoran Wang. I received my Ph.D. degree in Computer Science and Engineering from the University of Michigan, Ann Arbor advised by Prof. Jieping Ye (叶杰平).

[Interest]        [Publication]        [Grant]        [Service]

Openings

I am actively seeking self-motivated students with strong mathematical or programming skills for the following positions:
  • Postdoc & Research Assistant
    Multiple Postdoc and long-term / short-term RA positions available now

  • Ph.D. Students (Fall 2027)

    🌟 Available! Regular PhD track / Self-funded PhD / Self-funded MPhil
    Exceptionally qualified Ph.D. applicants will be encouraged to apply for the Hong Kong PhD Fellowship Scheme (HKPFS).

    🌟 Available! We offer a special PhD application track for applicants with a bachelor’s degree from C9 League, UCAS, and QS/THE Top 20 universities, featuring an expedited offer process and a much higher scholarship. Welcome eligible applicants to get in touch with me as soon as possible!
    (C9 League: THU, PKU, SJTU, FDU, ZJU, NJU, USTC, XJTU, HIT)
    • CityUHK Presidential Research Award (CPRA): THU, PKU, QS/THE Top 10; Annual stipend comparable to HKPFS
    • Top Universities Studentship Scheme (TUSS): C9 League, UCAS, QS/THE Top 20
    🌟 Available! We offer a joint PhD program for Ph.D. students from UCAS, FDU, ZJU, NJU, USTC, XJTU, and HIT. Welcome eligible applicants to get in touch with me as soon as possible!

If you have a background in or are passionate about the following topics:
  • Embodied AI & World Model
  • Agentic System & Multimodal LLM
  • AI for Engineering & Industry
  • Reinforcement Learning Algorithm & Theory
please do not hesitate to contact me (shuanqiu [AT] cityu [DOT] edu [DOT] hk) with your CV and transcript.


(Oct 25, 2026) To 2027 PhD/MPhil Applicants: Thank you for your interest in our research group! I have received your applications and I will start reviewing all of them very soon, including the earlier inquiries. Given the high volume, I regret that I'm unable to offer interviews to all applicants. Over the next few weeks, I will reach out to you if there is a strong match. Thank you for your understanding and patience!

News

09/2026 📝 I will be serving as Area Chair of ICLR 2027

07/2026 📝 I will be serving as Senior PC of AAAI 2027

06/2026 📅 The flagship distributed AI conference DAI 2026 will be held in Hong Kong, China, Nov 29 - Dec 02, 2026. Call for Contributions
I am pleased to serve as the Local Organizing Co-Chair. Welcome to join us at DAI 2026 !

04/2026 🎉 Congratulations to my incoming PhD student on receiving the prestigious and highly competitive Hong Kong PhD Fellowship Scheme

10/2025 🎉 Great honor to be a recipient of the Best Area Chair Award for the Chinese Conference on Pattern Recognition and Computer Vision, the largest and most comprehensive conference on pattern recognition and computer vision in China

09/2025 📃 We release our world model work LatticeWorld for building an industry-grade 3D world with multimodal LLMs. Check out this demo!

09/2025 📝 I will be serving as Area Chair of ICLR 2026

08/2025 🎉 Great honor to receive research funding from renowned VC MiraclePlus (奇绩创坛) to support our research on Agentic AI and LLMs.

Research Interest
My current research focuses on sequential decision-making and its applications across a range of domains. Specifically, my research interests span both theoretical and applied work in reinforcement learning, game theory, large language models and agentic systems, embodied intelligence, nonconvex optimization, and AI for engineering and industry.

Selected Recent Publication [Full List]

Algorithms & Appl.[LLM Reasoning] Process Reward Modeling with Reliability- and Conservatism-Aware Bayesian Belief
Zhongjian Qiao, Zhixiang Zhou, Jiafei Lyu, Peisong Wang, Qiaosheng Zhang, Siyang Gao, Shuang Qiu#
(Preprint)
[PDF]
Theory & Analysis[Safe Multi-Objective RL] Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence
Chenxi Li, Yao Shu, Siyang Gao, Shuang Qiu#, Guangxin Jiang#
(Preprint)
[PDF]
Algorithms & Appl.[LLM Agents] EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
Qijin Zhang*, Hanzhen Wang*, Yuehan Zhao*, Haoyu Dai, Yao Shu, Zhongxiang Dai, Shuang Qiu#
(Preprint)
[PDF]
Theory & Analysis[RL] Fast Rates for Offline Q-Learning with KL Regularization
Yize Hao, Guangxin Jiang, Shuang Qiu#
(Preprint)
[PDF]
Theory & Analysis[Multi-objective Bandit] Few-for-Many Best-Arm Identification in Multi-Objective Bandits
Bo Xue, Yilu Liu, Haodong Jing, Hongzong Li, Xi Lin, Shuang Qiu#
(Preprint)
[PDF]
Algorithms & Appl.[LLM Reasoning] Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
Yuanteng Chen, Zhilei Liu, Peisong Wang, Yuantian Shao, Chuangyi Li, Weining Wang, Shuang Qiu, Gang Li, Jing Liu, Jian Cheng
(Preprint)
[PDF]
Algorithms & Appl.[LLM Reasoning] DemoPSD: Disagreement-Modulated Policy Self-Distillation
Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu#, Linqi Song#
(Preprint)
[PDF]
Algorithms & Appl.[LLM Reasoning] ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Models
Wenhao Liu*, Hao Shi*, Yunhe Li*, Weizhi Fei, Xiangyuan Wang, Mengzhe Ruan, Hanxu Hou, Peisong Wang, Linqi Song, Shuang Qiu#
(Preprint)
[PDF]
Algorithms & Appl.[Robotics] AssemLM: Spatial Reasoning Multimodal Large Language Models for Robotic Assembly
Zhi Jing, Jinbin Qiao, Ouyang Lu, Jicong Ao, Shuang Qiu, Huazhe Xu, Yu-Gang Jiang, Chenjia Bai
Conference on Robot Learning (CoRL), 2026
[PDF]  [Project]
Algorithms & Appl.[LLM Reasoning] Learning to Reason with Insight for Informal Theorem Proving
Yunhe Li, Hao Shi, Bowen Deng, Wei Wang, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Siyang Gao, Chao Wang, Shuang Qiu#, Linqi Song#
(Preprint)
[PDF]
Algorithms & Appl.[RL] Unifying Value Alignment and Assignment in Cross-Domain Offline Reinforcement Learning with Heterogeneous Datasets
Zhongjian Qiao, Jiafei Lyu, Chenjia Bai, Peisong Wang, Siyang Gao, Shuang Qiu#
International Conference on Machine Learning (ICML), 2026
[PDF]
Algorithms & Appl.[LLM] T-POP: Test-Time Personalization with Online Preference Feedback
Zikun Qu, Min Zhang, Mingze Kong, Xiang Li, Zhiwei Shang, Zhiyong Wang, Yikun Ban, Shuang Qiu, Yao Shu, Zhongxiang Dai
International Conference on Machine Learning (ICML), 2026
[PDF]
Algorithms & Appl.[LLM] Certain Head, Uncertain Tail: Expert-Sample for Test-Time Scaling in Fine-Grained MoE
Yuanteng Chen, Peisong Wang, Nanxin Zeng, Yuantian Shao, Shuang Qiu, Gang Li, Jing Liu, Jian Cheng
International Conference on Machine Learning (ICML), 2026
[PDF]
Algorithms & Appl.[LLM Reasoning] Self-Reflective Generation at Test Time
Jian Mu, Qixin Zhang, Zhiyong Wang, Menglin Yang, Shuang Qiu, Chengwei Qin, Zhongxiang Dai, Yao Shu
Annual Meeting of the Association for Computational Linguistics (ACL main), 2026
[PDF]
Algorithms & Appl.[RL] Cross-Domain Offline Policy Adaptation with Dynamics-and Value-Aligned Data Filtering
Zhongjian Qiao, Rui Yang, Jiafei Lyu, Chenjia Bai, Xiu Li, Siyang Gao, Shuang Qiu#
(Preprint)
[PDF]  [Code]  [Chinese Media Coverage]
Algorithms & Appl.[LLM Reasoning] Deep Dense Exploration for LLM Reinforcement Learning via Pivot-Driven Resampling
Yiran Guo, Zhongjian Qiao, Yingqi Xie, Jie Liu, Dan Ye, Ruiqing Zhang, Shuang Qiu#, Lijie Xu#
(Preprint)
[PDF]
Algorithms & Appl.[Generative Model] SparVAR: Exploring Sparsity in Visual AutoRegressive Modeling for Training-Free Acceleration
Zekun Li, Ning Wang, Tongxin Bai, Changwang Mei, Peisong Wang, Shuang Qiu, Jian Cheng
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026
[PDF]
Algorithms & Appl.[RL] Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shifts
Zhongjian Qiao, Rui Yang, Jiafei Lyu, Xiu Li, Zhongxiang Dai, Zhuoran Yang, Siyang Gao, Shuang Qiu#
International Conference on Learning Representations (ICLR), 2026
[PDF]  [Code]
Algorithms & Appl.[RL] Model-based Offline RL via Robust Value-Aware Model Learning with Implicitly Differentiable Adaptive Weighting
Zhongjian Qiao, Jiafei Lyu, Boxiang Lyu, Yao Shu, Siyang Gao, Shuang Qiu#
International Conference on Learning Representations (ICLR), 2026
[PDF]  [Code]  [Chinese Media Coverage]
Algorithms & Appl.[World Model] LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
Yinglin Duan*, Zhengxia Zou*, Tongwei Gu*, Wei Jia, Zhan Zhao, Luyi Xu, Xinzhu Liu, Hao Jiang, Kang Chen#, Shuang Qiu#
(Preprint)
[PDF]  [Demo Video]  [Chinese Media Coverage]
LatticeWorld overview
Algorithms & Appl.[LLM Reasoning] Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models
Yiran Guo, Lijie Xu, Jie Liu, Dan Ye, Shuang Qiu
Advances in Neural Information Processing Systems (NeurIPS), 2025
[PDF]  [Code]  [Chinese Media Coverage | Highlighted as Lead Story]
Algorithms & Appl.[World Model] Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective
Yang Zhang, Xinran Li, Jianing Ye, Shuang Qiu, Delin Qu, Chongjie Zhang, Xiu Li, Chenjia Bai
Advances in Neural Information Processing Systems (NeurIPS), 2025
[PDF]
Theory & Analysis[Multi-Objective RL] Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learning
Shuang Qiu, Dake Zhang, Rui Yang, Boxiang Lyu, Tong Zhang
(Preprint)
[PDF]
Algorithms & Appl.[Robust LLM] ROPO: Robust Preference Optimization for Large Language Models
Xize Liang*, Chao Chen*, Shuang Qiu*, Jie Wang#, Yue Wu, Zhihang Fu, Zhihao Shi, Feng Wu, Jieping Ye
International Conference on Machine Learning (ICML), 2025
[PDF]
Algorithms & Appl.[RL] On the Value of Myopic Behavior in Policy Reuse
Chenjia Bai, Kang Xu, Shuang Qiu, Haoran He, Bin Zhao, Zhen Wang, Wei Li, Xuelong Li
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2025
[PDF]
Algorithms & Appl.[LLM] Online Preference Alignment for Language Models via Count-based Exploration
Chenjia Bai, Yang Zhang, Shuang Qiu, Qiaosheng Zhang, Kang Xu, Xuelong Li
International Conference on Learning Representations (ICLR Spotlight), 2025
[PDF]  [Code]  [Chinese Media Coverage]
Algorithms & Appl.[Robust RL] Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling
Jiawei Xu, Rui Yang, Shuang Qiu, Feng Luo, Meng Fang, Baoxiang Wang, Lei Han
International Conference on Learning Representations (ICLR), 2025
[PDF]
Algorithms & Appl.[RL & Diffusion] Forward KL Regularized Preference Optimization for Aligning Diffusion Policies
Zhao Shan, Chenyou Fan, Shuang Qiu, Jiyuan Shi, Chenjia Bai
AAAI Conference on Artificial Intelligence (AAAI), 2025
[PDF]
Theory & Analysis[RL & Econ] Learning Dynamic Mechanisms in Unknown Environments: A Reinforcement Learning Approach
Shuang Qiu*, Boxiang Lyu*, Qinglin Meng*, Zhaoran Wang, Zhuoran Yang, Michael I. Jordan
Journal of Machine Learning Research (JMLR), 2024
[PDF]
Algorithms & Appl.[Multi-Objective LLM] Rewards-in-Context: Multi-Objective Alignment of Foundation Models with Dynamic Preference Adjustment
Rui Yang*, Xiaoman Pan*, Feng Luo*, Shuang Qiu*, Han Zhong, Dong Yu, Jianshu Chen
International Conference on Machine Learning (ICML), 2024
[PDF]  [Code]
Theory & Analysis[Risk-Sensitive RL] Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
Dake Zhang, Boxiang Lyu, Shuang Qiu#, Mladen Kolar, Tong Zhang
International Conference on Machine Learning (ICML Spotlight), 2024
[PDF]
Algorithms & Appl.[Multi-Objective LLM] Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
Haoxiang Wang, Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, Tong Zhang
Annual Meeting of the Association for Computational Linguistics (ACL main), 2024
[PDF]  [Code]
Theory & Analysis[RL & Game] Posterior Sampling for Competitive RL: Function Approximation and Partial Observation
Shuang Qiu*, Ziyu Dai*, Han Zhong, Zhaoran Wang, Zhuoran Yang, Tong Zhang
Advances in Neural Information Processing Systems (NeurIPS), 2023
[PDF]
Theory & Analysis[RL] Optimistic Exploration with Learned Features Provably Solves Markov Decision Processes with Neural Dynamics
Sirui Zheng, Lingxiao Wang, Shuang Qiu, Zhuoran Yang, Csaba Szepesvari, Zhaoran Wang
International Conference on Learning Representations (ICLR), 2023
[PDF]
Theory & Analysis[Optimization] Gradient-Variation Bound for Online Convex Optimization with Constraints
Shuang Qiu, Xiaohan Wei, Mladen Kolar
AAAI Conference on Artificial Intelligence (AAAI), 2023
[PDF]
Theory & Analysis[RL & Game] Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning
Shuang Qiu, Lingxiao Wang, Chenjia Bai, Zhuoran Yang, Zhaoran Wang
International Conference on Machine Learning (ICML), 2022
[PDF]  [Code]
Algorithms & Appl.[Optimization] In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle
Lijie Xu, Shuang Qiu, Binhang Yuan, Jiawei Jiang, Cedric Renggli, Shaoduo Gan, Kaan Kara, Guoliang Li, Ji Liu, Wentao Wu, Jieping Ye, Ce Zhang
International Conference on Management of Data (SIGMOD), 2022
[PDF]   [Extended Version]
Theory & Analysis[RL & Game] On Reward-Free RL with Kernel and Neural Function Approximations: Single-Agent MDP and Markov Game
Shuang Qiu, Jieping Ye, Zhaoran Wang, Zhuoran Yang
International Conference on Machine Learning (ICML), 2021
[PDF]
Theory & Analysis[RL & Game] Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions
Shuang Qiu, Xiaohan Wei, Jieping Ye, Zhaoran Wang, Zhuoran Yang
International Conference on Machine Learning (ICML), 2021
[PDF]
Theory & Analysis[RL] On Finite-Time Convergence of Actor-Critic Algorithm
Shuang Qiu , Zhuoran Yang, Jieping Ye, Zhaoran Wang
IEEE Journal on Selected Areas in Information Theory (JSAIT), 2021
[PDF]
Algorithms & Appl.[Image Rendering] Stylized Neural Painting
Zhengxia Zou, Tianyang Shi, Shuang Qiu, Yi Yuan, Zhenwei Shi
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[PDF]  [Code]  [Project]
- Featured Applications
REMINI, an AI photo enhancer with 50M+ users worldwide
你我当年, an AI photo editor ranked No. 16 ("photos") in XiaoMI AppStore
RunwayML, a web-based video editing software
- Media Coverage
动 动手，一起为 春天 中国 “添彩” - 送您一支AI画笔，为祖国春天涂抹万千风情
有 了这支矢量神 经风 格画 笔，无需GAN也可生成精美绘画
Automatic Image-to-Painting Translation Method Generates Vivid Paintings in Controllable Styles
Stylized Neural Painter: An Image-To-Painting Translation Method That Generates Vivid And Realistic Painting Artworks With Controllable Styles
Algorithms & Appl.[Graph Embedding] Pine: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions
Shupeng Gui, Xiangliang Zhang, Pan Zhong, Shuang Qiu, Mingrui Wu, Jieping Ye, Zhengdao Wang, Ji Liu
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021
[PDF]
Theory & Analysis[Safe RL] Upper Confidence Primal-Dual Reinforcement Learning for CMDP with Adversarial Loss
Shuang Qiu, Xiaohan Wei, Zhuoran Yang, Jieping Ye, Zhaoran Wang
Advances in Neural Information Processing Systems (NeurIPS), 2020
[PDF]
Theory & Analysis[Compressed Sensing] Robust One-Bit Recovery via ReLU Generative Networks: Near-Optimal Statistical Rate and Global Landscape Analysis
Shuang Qiu*, Xiaohan Wei*, Zhuoran Yang
International Conference on Machine Learning (ICML), 2020
[PDF]


Grant
  • 2024-2027, Hong Kong RGC General Research Fund (GRF). PI: Shuang Qiu, Co-I: Dong Xia
  • 2022-2025, Hong Kong RGC General Research Fund (GRF). PI: Kani Chen,   Co-I: Shuang Qiu

Academic Service

Conference
  • International Conference on Distributed Artificial Intelligence (DAI), 2026. Local Organizing Co-Chair
  • International Conference on Learning Representations (ICLR), 2026-2027. Area Chair
  • AAAI Conference on Artificial Intelligence (AAAI), 2023-2027. Senior PC Member
  • International Conference on Multimedia and Expo (ICME), 2026. Area Chair
  • Chinese Conference on Pattern Recognition and Computer Vision (PRCV), 2025-2026. Area Chair
    (Best Area Chair Award (2025). 6/140+)
  • Innovation Forum on Intelligence Computing hosted by Zhejiang Lab and Science/AAAS, 2024-2025. Session Chair
  • International Conference on Learning Representations (ICLR), 2023-2025. Reviewer
  • International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. Reviewer
  • Advances in Neural Information Processing Systems (NeurIPS), 2018-2025. Reviewer
  • International Conference on Machine Learning (ICML), 2021-2025. Reviewer
  • AAAI Conference on Artificial Intelligence (AAAI), 2021. Program Committee Member
  • International Joint Conferences on Artificial Intelligence (IJCAI), 2021-2024. Program Committee Member
  • SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2020, 2023. Reviewer
Journal Reviewer
  • Journal of the American Statistical Association (JASA)
  • Journal of Machine Learning Research (JMLR)
  • IEEE Transactions on Information Theory (TIT)
  • Computer Science Review
  • IEEE Transactions on Knowledge and Data Engineering (TKDE)
  • IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
  • Springer Machine Learning (ML).
  • IEEE Transactions on Signal Processing (TSP)