Shuang Qiu I am an Assistant Professor in the Department of Systems Engineering and Department of Computer Science (secondary) at City University of Hong Kong. Prior to CityU, I did research in the Department of Mathematics at the Hong Kong University of Science and Technology as a research assistant professor working with Prof. Tong Zhang and in the Booth School of Business at the University of Chicago as a postdoc researcher  working with Prof. Mladen Kolar and Prof. Zhaoran Wang. I received my Ph.D. degree in Computer Science and Engineering from the University of Michigan, Ann Arbor advised by Prof. Jieping Ye.

[Interest]        [Publication]        [Grant]        [Service]

Openings

I am actively seeking self-motivated students with strong mathematical or programming skills for the following positions:
  • **Postdoc & Research Assistant** Multiple Postdoc and long-term / short-term / remote RA positions available now
    • (Multimodal) Large Language Model Reasoning
    • Reinforcement Learning
    • Embodied AI
  • Ph.D. Students (Fall 2025 - Application Deadline: December 1, 2024)
    • Exceptionally qualified Ph.D. applicants will be encouraged to apply for the Hong Kong PhD Fellowship Scheme (HKPFS).
If you have backgrounds in or are passionate about both applied and theoretical aspects of the above topics, please do not hesitate to contact me (shuanqiu [AT] cityu [DOT] edu [DOT] hk) with your CV and/or transcript.


Research Interest
My recent research interests generally focus on sequential decision-making problems and their applications in Embodied AI. Specifically, my research interest include both applications and theories of reinforcement learning, game theory, nonconvex optimization, large language models (LLMs), robotics, generative models (e.g. diffusion models), and econometrics.

Selected Recent Publication [Full List]
[LLM Reasoning] Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models
Yiran Guo, Lijie Xu, Jie Liu, Dan Ye, Shuang Qiu
(Preprint)
[PDF]  [Code]  [Media Coverage | Lead Story]


[Multi-Objective RL] Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learning
Shuang Qiu, Dake Zhang, Rui Yang, Boxiang Lyu, Tong Zhang
(Preprint)
[PDF]


[Robust LLM] ROPO: Robust Preference Optimization for Large Language Models
Xize Liang*, Chao Chen*, Shuang Qiu*, Jie Wang#, Yue Wu, Zhihang Fu, Zhihao Shi, Feng Wu, Jieping Ye
International Conference on Machine Learning (ICML), 2025
[PDF]


[RL] On the Value of Myopic Behavior in Policy Reuse
Chenjia Bai, Kang Xu, Shuang Qiu, Haoran He, Bin Zhao, Zhen Wang, Wei Li, Xuelong Li
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2025
[PDF]


[LLM] Online Preference Alignment for Language Models via Count-based Exploration
Chenjia Bai, Yang Zhang, Shuang Qiu, Qiaosheng Zhang, Kang Xu, Xuelong Li
International Conference on Learning Representations (ICLR Spotlight), 2025
[PDF]  [Code]  [Media Coverage]


[Robust RL] Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling
Jiawei Xu, Rui Yang, Shuang Qiu, Feng Luo, Meng Fang, Baoxiang Wang, Lei Han
International Conference on Learning Representations (ICLR), 2025
[PDF]


[RL & Diffusion] Forward KL Regularized Preference Optimization for Aligning Diffusion Policies
Zhao Shan, Chenyou Fan, Shuang Qiu, Jiyuan Shi, Chenjia Bai
AAAI Conference on Artificial Intelligence (AAAI), 2025
[PDF]


[RL & Econ] Learning Dynamic Mechanisms in Unknown Environments: A Reinforcement Learning Approach
Shuang Qiu*, Boxiang Lyu*, Qinglin Meng*, Zhaoran Wang, Zhuoran Yang, Michael I. Jordan
Journal of Machine Learning Research (JMLR), 2024
[PDF]


[Multi-Objective LLM] Rewards-in-Context: Multi-Objective Alignment of Foundation Models with Dynamic Preference Adjustment
Rui Yang*, Xiaoman Pan*, Feng Luo*, Shuang Qiu*, Han Zhong, Dong Yu, Jianshu Chen
International Conference on Machine Learning (ICML), 2024
[PDF]  [Code]


[Risk-Sensitive RL] Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
Dake Zhang, Boxiang Lyu, Shuang Qiu#, Mladen Kolar, Tong Zhang
International Conference on Machine Learning (ICML Spotlight), 2024
[PDF]


[Multi-Objective LLM] Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
Haoxiang Wang, Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, Tong Zhang
Annual Meeting of the Association for Computational Linguistics (ACL main), 2024
[PDF]  [Code]


[RL & Game] Posterior Sampling for Competitive RL: Function Approximation and Partial Observation
Shuang Qiu*, Ziyu Dai*, Han Zhong, Zhaoran Wang, Zhuoran Yang, Tong Zhang
Advances in Neural Information Processing Systems (NeurIPS), 2023
[PDF]


[RL] Optimistic Exploration with Learned Features Provably Solves Markov Decision Processes with Neural Dynamics
Sirui Zheng, Lingxiao Wang, Shuang Qiu, Zhuoran Yang, Csaba Szepesvari, Zhaoran Wang
International Conference on Learning Representations (ICLR), 2023
[PDF]


[Optimization] Gradient-Variation Bound for Online Convex Optimization with Constraints
Shuang Qiu, Xiaohan Wei, Mladen Kolar
AAAI Conference on Artificial Intelligence (AAAI), 2023
[PDF]


[RL & Game] Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning
Shuang Qiu, Lingxiao Wang, Chenjia Bai, Zhuoran Yang, Zhaoran Wang
International Conference on Machine Learning (ICML), 2022
[PDF]  [Code]


[Optimization] In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle
Lijie Xu, Shuang Qiu, Binhang Yuan, Jiawei Jiang, Cedric Renggli, Shaoduo Gan, Kaan Kara, Guoliang Li, Ji Liu, Wentao Wu,
Jieping Ye, Ce Zhang
International Conference on Management of Data (SIGMOD), 2022
[PDF]   [Extended Version]


[RL & Game] On Reward-Free RL with Kernel and Neural Function Approximations: Single-Agent MDP and Markov Game
Shuang Qiu, Jieping Ye, Zhaoran Wang, Zhuoran Yang
International Conference on Machine Learning (ICML), 2021
[PDF]


[RL & Game] Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions
Shuang Qiu, Xiaohan Wei, Jieping Ye, Zhaoran Wang, Zhuoran Yang
International Conference on Machine Learning (ICML), 2021
[PDF]


[Image Rendering] Stylized Neural Painting
Zhengxia Zou, Tianyang Shi, Shuang Qiu, Yi Yuan, Zhenwei Shi
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[PDF]  [Code]  [Project]
- Featured Applications
REMINI, an AI photo enhancer with 50M+ users worldwide
你我当年, an AI photo editor ranked No. 16 ("photos") in XiaoMI AppStore
RunwayML, a web-based video editing software
- Media Coverage
动 动手，一起为 春天 中国 “添彩” - 送您一支AI画笔，为祖国春天涂抹万千风情
有 了这支矢量神 经风 格画 笔，无需GAN也可生成精美绘画
Automatic Image-to-Painting Translation Method Generates Vivid Paintings in Controllable Styles
Stylized Neural Painter: An Image-To-Painting Translation Method That Generates Vivid And Realistic Painting Artworks With Controllable Styles


[Safe RL] Upper Confidence Primal-Dual Reinforcement Learning for CMDP with Adversarial Loss
Shuang Qiu, Xiaohan Wei, Zhuoran Yang, Jieping Ye, Zhaoran Wang
Advances in Neural Information Processing Systems (NeurIPS), 2020
[PDF]
[Compressed Sensing] Robust One-Bit Recovery via ReLU Generative Networks: Near-Optimal Statistical Rate and Global Landscape Analysis
Shuang Qiu*, Xiaohan Wei*, Zhuoran Yang
International Conference on Machine Learning (ICML), 2020
[PDF]


Grant
  • 2024-2027, Hong Kong RGC General Research Fund (GRF). PI: Shuang Qiu, Co-I: Dong Xia

  • 2022-2025, Hong Kong RGC General Research Fund (GRF). PI: Kani Chen,   Co-I: Shuang Qiu

Academic Service

Conference
  • AAAI Conference on Artificial Intelligence (AAAI), 2023-2025. Senior PC Member

  • International Conference on Learning Representations (ICLR), 2023-2025. Reviewer

  • International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. Reviewer

  • Advances in Neural Information Processing Systems (NeurIPS), 2018-2025. Reviewer

  • International Conference on Machine Learning (ICML), 2021-2025. Reviewer

  • AAAI Conference on Artificial Intelligence (AAAI), 2021. Program Committee Member

  • International Joint Conferences on Artificial Intelligence (IJCAI), 2021-2024. Program Committee Member

  • SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2020, 2023. Reviewer
Journal
  • Journal of the American Statistical Association (JASA). Reviewer

  • Journal of Machine Learning Research (JMLR). Reviewer

  • IEEE Transactions on Information Theory (TIT). Reviewer

  • IEEE Transactions on Knowledge and Data Engineering (TKDE). Reviewer

  • IEEE Transactions on Neural Networks and Learning Systems (TNNLS). Reviewer

  • Springer Machine Learning (ML). Reviewer

  • IEEE Transactions on Signal Processing (TSP). Reviewer