CV

Google Scholar

I am a Senior Researcher in the Robot Learning Group of Microsoft Research, Redmond, and I am affiliated with the MSR Reinforcement Learning Group. I am a practical theoretician who is interested in developing foundations for designing principled algorithms that can tackle real-world challenges. My research studies structural properties in sequential decision making problems, especially in robotics, and aims to improve the learning efficiency of autonomous agents. My recent works focus on reinforcement learning and imitation learning from offline data. Previously, I worked on online learning, Gaussian processes, and integrated motion planning and control.

I received PhD in Robotics in 2020 from Georgia Tech, where I was advised by Byron Boots at Institute for Robotics and Intelligent Machines. During my PhD study, I interned at Microsoft Research AI, Redmond, in Summer 2019, working with Alekh Agarwal and Andrey Kolobov; at Nvidia Research, Seattle, in Summer 2018, working with Nathan Ratliff and Dieter Fox.

Before Georgia Tech, I received from National Taiwan University (NTU) my M.S. in Mechanical Engineering in 2013, and double degrees of B.S. in Mechanical Engineering and B.S. in Electrical Engineering in 2011. During that period, I was advised by Han-Pang Huang, who directs NTU Robotics Laboratory, and my research included learning dynamical systems, force/impedance control, kernel methods, and approximation theory – with applications ranging from manipulator, grasping, exoskeleton, brain-computer interface, to humanoid.

I was fortunately awarded with Outstanding Paper Award, Runner-Up (ICML 2022), Best Paper Award (OptRL Workshop @ NeurIPS 2019), Best Student Paper & Best Systems Paper, Finalist (RSS 2019), Best Paper (AISTATS 2018), Best Systems Paper, Finalist (RSS 2018), NVIDIA Graduate Fellowship, and Google PhD Fellowship (declined).

Preprints

2023

  • Improving Offline RL by Blending Heuristics  
    arXiv preprint arXiv:2306.00321, 2023

    S. Geng, A. Pacchiano, A. Kolobov, and C.-A. Cheng


Journal/Conference Publications

2023

  • Survival Instinct in Offline Reinforcement Learning  
    Selected for Spotlight Presentation (<3%)
    Conference on Neural Information Processing Systems, 2023

    A. Li, D. Misra, A. Kolobov, and C.-A. Cheng

  • Adversarial Model for Offline Reinforcement Learning  
    Conference on Neural Information Processing Systems, 2023

    M. Bhardwaj, T. Xie, B. Boots, N. Jiang, and C.-A. Cheng

  • PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining  
    Conference on Robot Learning, 2023

    G. Thomas, C.-A. Cheng, R. Loynd, V. Vineet, M. Jalobeanu, and A. Kolobov

  • Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control  
    Conference on Robot Learning, 2023

    V. Myers, A. He, K. Fang, H. Walke, P. Hansen-Estruch, C.-A. Cheng, M. Jalobeanu, A. Kolobov, A. Dragan, and S. Levine

  • MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations    
    International Conference on Machine Learning, 2023

    A. Li, B. Boots, and C.-A. Cheng

  • Provable Reset-free Reinforcement Learning by No-Regret Reduction  
    International Conference on Machine Learning, 2023

    H.-A. Nguyen and C.-A. Cheng

  • Hindsight Learning for MDPs with Exogenous Inputs  
    International Conference on Machine Learning, 2023

    S. R. Sinclair, F. Frujeri, C.-A. Cheng, L. Marshall, H. Barbalho, J. Li, J. Neville, I. Menache, and A. Swaminathan

  • Provably Efficient Lifelong Reinforcement Learning with Linear Function Representation  
    International Conference on Learning Representations, 2023

    S. Amani, L. F. Yang, and C.-A. Cheng

2022

  • MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control    
    Neural Information Processing Systems Datasets and Benchmarks Track, 2022

    N. Wagener, A. Kolobov, F. V. Frujeri, R. Loynd, C.-A. Cheng, and M. Hausknecht

  • Adversarially trained actor critic for offline reinforcement learning      
    Outstanding Paper Award, Runner-Up
    International Conference on Machine Learning, 2022

    C.-A. Cheng*, T. Xie*, N. Jiang, and A. Agarwal

2021

  • Bellman-consistent pessimism for offline reinforcement learning  
    Selected for Oral Presentation (<1%)
    Advances in Neural Information Processing Systems, 2021

    T. Xie, C.-A. Cheng, N. Jiang, P. Mineiro, and A. Agarwal

  • Heuristic-guided reinforcement learning  
    Advances in Neural Information Processing Systems, 2021

    C.-A. Cheng, A. Kolobov, and A. Swaminathan

  • Safe reinforcement learning using advantage-based intervention  
    International Conference on Machine Learning, 2021

    N. C. Wagener, B. Boots, and C.-A. Cheng

  • Cautiously optimistic policy optimization and exploration with linear function approximation  
    Conference on Learning Theory, 2021

    A. Zanette, C.-A. Cheng, and A. Agarwal

  • RMP²: A structured composable policy class for robot learning  
    Robotics: Science and Systems, 2021

    A. Li*, C.-A. Cheng*, M. A. Rana, M. Xie, K. Van Wyk, N. Ratliff, and B. Boots

  • Explaining fast improvement in online imitation learning  
    Uncertainty in Artificial Intelligence, 2021

    X. Yan, B. Boots, and C.-A. Cheng

  • RMPflow: A geometric framework for generation of multitask motion policies  
    IEEE Transactions on Automation Science and Engineering, 2021

    C.-A. Cheng, M. Mukadam, J. Issac, S. Birchfield, D. Fox, B. Boots, and N. Ratliff

2020

  • Policy improvement via imitation of multiple oracles  
    Selected for Spotlight Presentation (<3%)
    Advances in Neural Information Processing Systems, 2020

    C.-A. Cheng, A. Kolobov, and A. Agarwal

  • Intra order-preserving functions for calibration of multi-class neural networks  
    Advances in Neural Information Processing Systems, 2020

    A. Rahimi*, A. Shaban*, C.-A. Cheng*, B. Boots, and R. Hartley

  • A reduction from reinforcement learning to no-regret online learning  
    International Conference on Artificial Intelligence and Statistics, 2020

    C.-A. Cheng, R. T. Combes, B. Boots, and G. Gordon

  • Online learning with continuous variations: Dynamic regret and reductions  
    International Conference on Artificial Intelligence and Statistics, 2020

    C.-A. Cheng*, J. Lee*, K. Goldberg, and B. Boots

  • Extending Riemmanian Motion Policies to a Class of Underactuated Wheeled-Inverted-Pendulum Robots  
    2020 IEEE International Conference on Robotics and Automation (ICRA), 2020

    B. Wingo, C.-A. Cheng, M. Murtaza, M. Zafar, and S. Hutchinson

  • Trajectory-wise control variates for variance reduction in policy gradient methods  
    Conference on Robot Learning, 2020

    C.-A. Cheng, X. Yan, and B. Boots

  • Riemannian motion policy fusion through learnable lyapunov function reshaping  
    Conference on robot learning, 2020

    M. Mukadam, C.-A. Cheng, D. Fox, B. Boots, and N. Ratliff

  • Imitation learning for agile autonomous driving  
    The International Journal of Robotics Research, 2020

    Y. Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. A. Theodorou, and B. Boots

2019

  • Stable, concurrent controller composition for multi-objective robotic tasks  
    2019 IEEE 58th Conference on Decision and Control (CDC), 2019

    A. Li, C.-A. Cheng, B. Boots, and M. Egerstedt

  • A singularity handling algorithm based on operational space control for six-degree-of-freedom anthropomorphic manipulators  
    International Journal of Advanced Robotic Systems, 2019

    Z.-H. Kang, C.-A. Cheng, and H.-P. Huang

  • An Online Learning Approach to Model Predictive Control  
    Best Student Paper Award & Best Systems Paper Award, Finalist
    Robotics: Science and Systems, 2019

    N. Wagener*, C.-A. Cheng*, J. Sacks, and B. Boots

  • Predictor-corrector policy optimization  
    Selected for Long Talk (<5%)
    International Conference on Machine Learning, 2019

    C.-A. Cheng, X. Yan, N. Ratliff, and B. Boots

  • Accelerating imitation learning with predictive models  
    The 22nd International Conference on Artificial Intelligence and Statistics, 2019

    C.-A. Cheng, X. Yan, E. Theodorou, and B. Boots

  • Truncated back-propagation for bilevel optimization  
    The 22nd International Conference on Artificial Intelligence and Statistics, 2019

    A. Shaban*, C.-A. Cheng*, N. Hatch, and B. Boots

2018

  • RMPflow: A computational graph for automatic motion policy generation  
    International Workshop on the Algorithmic Foundations of Robotics, 2018

    C.-A. Cheng, M. Mukadam, J. Issac, S. Birchfield, D. Fox, B. Boots, and N. Ratliff

  • Orthogonally Decoupled Variational Gaussian Processes  
    Conference on Neural Information Processing Systems, 2018

    H. Salimbeni*, C.-A. Cheng*, B. Boots, and M. Deisenroth

  • Fast Policy Learning using Imitation and Reinforcement  
    Selected for Plenary Presentation (<8%)
    Conference on Uncertainty in Artificial Intelligence, 2018

    C.-A. Cheng, X. Yan, N. Wagener, and B. Boots

  • Agile Off-Road Autonomous Driving Using End-to-End Deep Imitation Learning  
    Best Systems Paper Award, Finalist
    Robotics: Science and Systems, 2018

    Y. Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. Theodorou, and B. Boots

  • Convergence of Value Aggregation for Imitation Learning  
    Best Paper Award
    International Conference on Artificial Intelligence and Statistics, 2018

    C.-A. Cheng and B. Boots

  • Optical sensing and control methods for soft pneumatically actuated robotic manipulators  
    2018 IEEE International Conference on Robotics and Automation (ICRA), 2018

    J. L. Molnar, C.-A. Cheng, L. O. Tiziani, B. Boots, and F. L. Hammond

2016-2017

  • Variational Inference for Gaussian Process Models with Linear Complexity  
    Advances in Neural Information Processing Systems, 2017

    C.-A. Cheng and B. Boots

  • Approximately Optimal Continuous-Time Motion Planning and Control via Probabilistic Inference  
    IEEE International Conference on Robotics and Automation, 2017

    M. Mukadam, C.-A. Cheng, X. Yan, and B. Boots

  • Incremental Variational Sparse Gaussian Process Regression  
    Advances in Neural Information Processing Systems, 2016

    C.-A. Cheng and B. Boots

  • Learn the Lagrangian: A Vector-Valued RKHS Approach to Identifying Lagrangian Systems  
    IEEE Transactions on Cybernetics, 2016

    C.-A. Cheng and H.-P. Huang

  • Virtual Impedance Control for Safe Human-Robot Interaction  
    Journal of Intelligent & Robotic Systems, 2016

    S.-Y. Lo, C.-A. Cheng, and H.-P. Huang

  • Learning the inverse dynamics of robotic manipulators in structured reproducing kernel Hilbert space  
    IEEE Transactions on Cybernetics, 2016

    C.-A. Cheng, H.-P. Huang, H.-K. Hsu, W.-Z. Lai, and C.-C. Cheng

2010-2015

  • Humanoid robot push-recovery strategy based on CMP criterion and angular momentum regulation  
    IEEE International Conference on Advanced Intelligent Mechatronics, 2015

    C.-H. Chang, H.-P. Huang, H.-K. Hsu, and C.-A. Cheng

  • Efficient grasp synthesis and control strategy for robot hand-arm system  
    IEEE International Conference on Automation Science and Engineering, 2015

    M.-B. Huang, H.-P. Huang, C.-C. Cheng, and C.-A. Cheng

  • Development of a P300 BCI and design of an elastic mechanism for a rehabilitation robot  
    International Journal of Automation and Smart Technology, 2015

    H.-P. Huang, Y.-H. Liu, W.-Z. Lin, Z.-H. Kang, C.-A. Cheng, and T.-H. Huang

  • Identification of the inverse dynamics of robot manipulators with the structured kernel  
    International Automatic Control Conference, 2013

    C.-A. Cheng, H.-P. Huang, H.-K. Hsu, W.-Z. Lai, C.-C. Cheng, and Y.-C. Li

  • Self-learning assistive exoskeleton with sliding mode admittance control  
    IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013

    T.-H. Huang, C.-A. Cheng, and H.-P. Huang

  • Bayesian human intention estimator for exoskeleton system  
    IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2013

    C.-A. Cheng, T.-H. Huang, and H.-P. Huang

  • Design of a new hybrid control and knee orthosis for human walking and rehabilitation  
    IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012

    T.-H. Huang, H.-P. Huang, C.-A. Cheng, J.-Y. Kuan, P.-T. Lee, and S.-Y. Huang

  • Development of a Brain-machine Interface for Motor Imagination Task  
    International Conference on Automation Technology, 2012

    H.-P. Huang, Y.-H. Liu, T.-H. Huang, Z.-H. Kang, W.-Z. Lin, W. Ching-Ping, and C.-A. Cheng

  • Novel feature of the EEG based motor imagery BCI system: Degree of imagery  
    International Conference on System Science and Engineering, 2011

    Y.-H. Liu, C.-A. Cheng, and H.-P. Huang

  • Motor Imagery Recognition for Brain-computer Interfaces using Hilbert-Huang Transform and Effective Event-related-desynchronization Features  
    CSME National Conference, 2010

    C.-A. Cheng, Y.-H. Liu, and H.-P. Huang


Workshop Papers

2023

  • Survival Instinct in Offline Reinforcement Learning and Implicit Human Bias in Data  
    Oral Presentation
    Interactive Learning with Implicit Human Feedback Workshop at ICML 2023, 2023

    A. Li, D. Misra, A. Kolobov, and C.-A. Cheng

  • Provable Reset-free Reinforcement Learning by No-Regret Reduction  
    AAAI 2023 Reinforcement Learning Ready for Production Workshop, 2023

    H.-A. Nguyen and C.-A. Cheng

2022

  • ARMOR: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data  
    NeurIPS 2022 Offline RL Workshop, 2022

    T. Xie, M. Bhardwaj, N. Jiang, and C.-A. Cheng

  • HEETR: Pretraining for Robotic Manipulation on Heteromodal Data  
    CoRL 2022 Workshop on Pre-training Robot Learning, 2022

    G. Thomas, A. Kolobov, C.-A. Cheng, V. Vineet, and M. Jalobeanu

2020

  • RMP²: A Differentiable Policy Class for Robotic Systems with Control-Theoretic Guarantees  
    NeurIPS 2020 3rd Robot Learning Workshop, 2020

    A. Li*, C.-A. Cheng*, M. A. Rana, N. Ratliff, and B. Boots

2019

  • Continuous Online Learning and New Insights to Online Imitation Learning  
    Best Paper Award
    NeurIPS 2019 Optimization Foundations of Reinforcement Learning Workshop, 2019

    J. Lee*, C.-A. Cheng*, K. Goldberg, and B. Boots

  • Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Method  
    NeurIPS 2019 Optimization Foundations of Reinforcement Learning Workshop, 2019

    C.-A. Cheng*, X. Yan*, and B. Boots

2018

  • Predictor-Corrector Policy Optimization  
    Deep Reinforcement Learning Workshop NeurIPS, 2018

    C.-A. Cheng, X. Yan, N. Ratliff, and B. Boots

2017

  • Learning Deep Neural Network Control Policies for Agile Off-Road Autonomous Driving  
    The NIPS Deep Reinforcement Learning Symposium, 2017

    Y. Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. Theodorou, and B. Boots.

  • Convergence of Value Aggregation for Imitation Learning  
    The NIPS Deep Reinforcement Learning Symposium, 2017

    C.-A. Cheng and B. Boots

2016

  • Incremental Variational Sparse Gaussian Process Regression  
    NIPS Workshop on Adaptive and Scalable Nonparametric Methods in Machine Learning, 2016

    C.-A. Cheng and B. Boots


Theses

2020

  • Efficient and Principled Robot Learning: Theory and Algorithms  
    Georgia Institute of Technology, 2020

    C.-A. Cheng

2013

  • Robot Dynamics Learning and Human-Robot Interaction  
    National Taiwan University, 2013

    C.-A. Cheng