Lixin YANG (杨理欣)

Research Assistant Professor
Shanghai Jiao Tong University, School of Artificial Intelligence
上海交通大学人工智能学院 助理研究员, 硕士生导师
Member of Machine Vision and Intelligence Group (MVIG) at SJTU

Email: siriusyang at sjtu dot edu dot cn
Office: Bldg. SAI, No. 1954 Huashan Rd., Xuhui Dist., Shanghai, 200230, China

About.  I’m a Research Assistant Professor in Shanghai Jiao Tong University (SJTU), affiliated with the School of Artificial Intelligence (SAI), where I joined in September 2024. I obtained Ph.D. degree in Computer Science from SJTU in 2023, advised by Prof. Cewu Lu at the Machine Vision and Intelligence Group and M.S. degree in Mechanical Engineering, SJTU. My research interests include 3D Vision and Robotics. Currently, I am focusing on modeling and imitating the hand manipulating objects, including 3D hand | object pose | shape estimation, grasp | motion generation, imitation learning, dexterous manipulation.

Join Us.  I am looking for Master Student at SJTU SAI and self-motivated research interns. Contact me if you are interested in the above topics.
诚意科研研究实习生(带薪), 我们一起做有意思的科研。

profile photo

News
  • [2026.01] Two papers have been accepted to ICRA 2026 🇦🇹
  • [2025.07] Generalizable Multi-view Hand Reconstruction POEMv2 is accepted by TPAMI 2025
  • [2025.06] Dense Policy is accepted by ICCV 2025 🇺🇸
  • [2025.05] Motion-before-Action is accepted by RA-L.
  • [2025.01] HybrIK-X is accepted by TPAMI.
  • [2024.12] Invited Talk at ROSCon China Workshop, thanks Yao Mu for hosting.
  • [2024.09] One paper on articulated object image manipulation got accepted by NeurIPS 2024 🇨🇦.
  • [2024.07] One paper: SemGrasp is accepted by ECCV 2024 🇮🇹.
  • [2024.02] One paper: OakInk2 is accepted by CVPR 2024 🇺🇸.
  • [2024.02] The Contact Potential Field is accepted by TPAMI.
  • [2023.12] One paper: FAVOR is accepted by AAAI 2024 🇨🇦.
  • [2023.10] One paper: Color-NeuS is accepted by 3DV 2024 🇨🇭.
  • [2023.08] I defend my doctoral thesis and earn my Ph.D!
  • [2023.08] I am honored to be an invited speaker at the HANDS workshop at ICCV 2023.
  • [2023.07] One paper: CHORD is accepted by ICCV 2023 🇫🇷.
  • [2023.06] Invited Talk at 智东西公开课 | 研讨会: 3D手部重建及具身智能交互. 视频 (中文).
  • [2023.02] One paper: POEM is accepted by CVPR 2023.
  • [2022.10] I have taken the wonderful journey of marriage alongside my cherished wife.
  • [2022.10] Invited Talk at International Digital Economy Academy (IDEA), Thanks Ailing Zeng for hosting.
  • [2022.09] One paper: DART got accepted by NeurIPS 2022 - Datasets and Benchmarks Track.
  • [2022.07] Invited Talk at 智东西公开课 | AI新青年讲座: 基于图像的手物交互重建与虚拟人手生成. 视频 (中文).
  • [2022.04] Invited Talk at MPI-IS Perceiving Systems. Thanks Yuliang Xiu for hosting. (info).
  • [2022.03] Two paper were accepted by CVPR 2022: one Oral, one poster.
  • [2021.07] One paper got accepted by ICCV 2021.
Publications

*=equal contribution, #=corresponding author

No publications under the selected topic.
LIDEA: Human-to-Robot Imitation Learning via Implicit Feature Distillation and Explicit Geometry Alignment
Yifu Xu*, Bokai Lin*, Xinyu Zhan, Hongjie Fang, Yong-Lu Li, Cewu Lu, Lixin Yang#
arXiv, 2026 project arXiv

A human-to-robot learning that bridges the embodiment gap with transitive 2D feature distillation and explicit 3D geometry alignment.

LaMP: Learning Vision-Language-Action Policy with 3D Scene Flow as Latent Motion Prior
Xinkai Wang, Chenyi Wang, Yifu Xu, Mingzhe Ye, Fucheng Zhang, Jialin Tian, Xinyu Zhan, Lifeng Zhu, Cewu Lu, Lixin Yang#
arXiv, 2026 project arXiv

A dual-expert Vision-Language-Action framework that injects one-step partially denoised 3D scene flow as a latent motion prior.

TrajBooster: Boosting Humanoid Whole-Body Manipulation via Trajectory-Centric Learning
Jiacheng Liu*, Pengxiang Ding*, Qihang Zhou, Yuxuan Wu, Da Huang, Zimian Peng, Wei Xiao, Weinan Zhang, Lixin Yang#, Cewu Lu#, Donglin Wang#,
ICRA, 2026 project arXiv

A cross-embodiment framework that transfers wheeled-humanoid data to bipedal VLA models via morphology-agnostic 6D end-effector trajectories and a heuristic-enhanced online DAgger controller.

Multi-view Hand Reconstruction with a Point-Embedded Transformer
Lixin YangLicheng Zhong,  Pengxiang Zhu,  Xinyu Zhan,  Junxiao Kong,  Jian XuCewu Lu#
TPAMI, 2025 paper arXiv code

POEM-v2 a generalizable multi-view 3D hand reconstruction model trained on large-scale multi-view datasets. It enables accurate, flexible, and occlusion-robust hand mesh recovery across arbitrary multi-view setups.

AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons
Hongjie Fang*,  Chenxi Wang*,  Yiming Wang*, Jingjing Chen*, Shangning Xia, Jun Lv, Zihao He, Xiyan Yi, Yunhan Guo, Xinyu Zhan, Lixin Yang, Weiming Wang, Cewu Lu#, Hao-Shu Fang#
CoRL, 2025 (Oral Presentation) project arXiv

A low-cost exoskeleton system for large-scale in-the-wild demonstration collection. It transforms the collected in-the-wild demonstrations into pseudo-robot demonstrations.

Dense Policy: Bidirectional Autoregressive Learning of Actions
ICCV, 2025 project arXiv

A bidirectional robotic autoregressive policy, which infers trajectories by gradually expanding actions from sparse keyframes, demonstrated exceeding diffusion policies.

Motion Before Action: Diffusing Object Motion as Manipulation Condition
RA-L, 2025, ICRA, 2026 project arXiv code

A two cascaded diffusion processes for object motion generation and robot action generation under object motion guidance.

SemGrasp: Semantic Grasp Generation via Language Aligned Discretization
ECCV, 2024 (Oral Presentation) project arXiv

A MLLM-based method that infuses language instructions into grasp generation; & A new language-pose dataset, CapGrasp, featuring detailed caption of grasping poses.

OakInk2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion
Xinyu Zhan*, Lixin Yang*, Yifei Zhao, Kangrui Mao, Hanlin Xu, Zenan Lin, Kailin Li, Cewu Lu#
CVPR, 2024 project arXiv

A 4D motion dataset focusing on bimanual object manipulation tasks involved in complex daily activities with three-tiered task abstraction: Object Affordance, Primitive Task, and Complex Task.

FAVOR: Full-Body AR-Driven Virtual Object Rearrangement Guided by Instruction Text
Kailin Li*, Lixin Yang*, Zenan Lin, Jian Xu, Xinyu Zhan, Yifei Zhao, Pengxiang Zhu, Wenxiong Kang, Kejian Wu, Cewu Lu#
AAAI, 2024 project arXiv code data

A full-body motion dataset and pipeline for text-guided desktop object rearrangement with avatars.

Color-NeuS: Reconstructing Neural Implicit Surfaces with Color
Licheng Zhong*,  Lixin Yang*,  Kailin LiHaoyu ZhenMei HanCewu Lu#
3DV, 2024 project arXiv code data

Reconstructs colored neural implicit surfaces by decoupling geometry from view-dependent shading for accurate colored mesh extraction.

CHORD: Category-level in-Hand Object Reconstruction via Shape Deformation
ICCV, 2023 project arXiv code tool

A category-level in-hand object reconstruction method with shape deformation priors and a synthetic COMIC dataset for diverse interactions.

POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo
CVPR, 2023 arXiv code

A Transformer-based multi-view hand reconstruction method that uses point embeddings to fuse sparse camera observations into accurate hand meshes.

DART: Articulated Hand Model with Diverse Accessories and Rich Textures
Daiheng Gao*,   Yuliang Xiu*,  Kailin Li*,  Lixin Yang*, Feng Wang, Peng Zhang, Bang Zhang, Cewu Lu, Ping Tan
NeurIPS, 2022 - Datasets and Benchmarks Track project arXiv code video

A richly textured MANO-derived articulated hand model with accessories, appearance variation, and benchmark-ready assets.

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction
Lixin Yang*, Kailin Li*, Xinyu Zhan*, Fei Wu, Anran Xu, Liu Liu, Cewu Lu#
CVPR, 2022 project paper arXiv code

A large-scale hand-object interaction knowledge base with affordance annotations and a model for transferring grasps across objects.

ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis
Lixin Yang*, Kailin Li*, Xinyu Zhan, Jun Lv, Wenqiang Xu, Jiefeng Li, Cewu Lu#
CVPR, 2022 (Oral Presentation) paper arXiv code

An online exploration and synthesis framework that improves articulated hand-object pose estimation and generates dexterous grasps for novel objects.

Learning a Contact Potential Field to Model the Hand-Object Interaction
Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Junming Zhang, Jiefeng Li, Cewu Lu#
TPAMI, 2024 paper

Extends CPF into a unified spring-mass-style contact field that couples attraction, repulsion, and geometry constraints for robust hand-object reconstruction.

CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction
Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, Cewu Lu#

Models hand-object contact as a spring-mass-like potential field, enabling geometry-aware optimization for physically plausible interaction reconstruction.

HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery
Jiefeng Li, Siyuan Bian, Chao Xu, Zhicun Chen, Lixin Yang, Cewu Lu#
TPAMI, 2025 arXiv

Extends HybrIK to efficient whole-body mesh recovery by jointly modeling body, hands, and face.

HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation
Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, Cewu Lu#
CVPR, 2021 project paper supp arXiv code

A hybrid analytical-neural inverse kinematics method for fast and accurate 3D human pose and shape recovery.

HandTailor: Towards High-Precision Monocular 3D Hand Recovery
Jun Lv, Wenqiang Xu, Lixin Yang, Sucheng Qian, Chongzhao Mao, Cewu Lu#
BMVC, 2021 paper arXiv code
BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks
Lixin Yang, Jiasen Li, Wenqiang Xu, Yiqun Diao, Cewu Lu#
BMVC, 2020 paper arXiv code
Other Collaborations
GaPT-DAR: Category-level Garments Pose Tracking via Integrated 2D Deformation and 3D Reconstruction
Li Zhang, Mingliang Xu, Jianan Wang, Qiaojun Yu, Lixin Yang, Yong-Lu Li, Cewu Lu, Rujing Wang, Liu Liu#
CVPR, 2025 project paper
General Articulated Objects Manipulation in Real Images via Part-Aware Diffusion Process
Zhou Fang, Yong-Lu Li#, Lixin Yang, Cewu Lu#
NeurIPS, 2024 paper


Talks
  • Human-Robot Data Companion: Pipeline and Representation.
    [2025.09] SII TechFest workshop: Embodied AI Reasoning and Scaling. Thank Panpan Cai for hosting.

  • Tutorial: Multimodal Embodied Perception and Action Prediction
    [2025.07]USTC Summer School 2025. Advances in Computer Graphics. Thank Yumeng Liu for hosting.

  • Empowering Object Manipulation with Human Knowledge.
    [2024.12] ROSCon China Workshop. Thank Yao Mu for hosting.
    [2024.12] PKU-Agibot Lab Thank Hongwei Fan for hosting.

  • Paving the Way for Understanding Human Interactions with Objects: The OakInk2 Dataset.
    [2023.08] ICCV 2023 HANDS Workshop, Thank Linlin Yang for hosting.

  • 3D手部重建及具身智能交互.
    [2023.09] ByteDance PICO. Thank Chao Wen for hosting.
    [2023.06] 智东西公开课研讨会. Co-speakers: Xinyu Chen and Daiheng Gao.

  • Inferring Human Hand-Object Interaction through Visual Input
    [2022.10] International Digital Economy Academy , Thank Ailing Zeng for hosting.

  • 基于图像的手物交互重建与虚拟人手生成
    [2022.07] 智东西公开课: AI新青年讲座

  • Leverage Kinematic and Contact constraints for understanding hand-object interaction
    [2022.04] MPI-IS Perceiving Systems. Thank Yuliang Xiu for hosting.


  • Services
  • Conference reviewer:
    • Robotics: Science and Systems (RSS)
    • IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
    • IEEE International Conference on Computer Vision (ICCV)
    • European Conference on Computer Vision (ECCV)
    • ACM SIGGRAPH
    • Conference on Neural Information Processing Systems (NeurIPS)
    • International Conference on Learning Representations (ICLR)
    • International Conference on Machine Learning (ICML)
    • Association for the Advancement of Artificial Intelligence (AAAI)
    • Conference of the European Association for Computer Graphics (Eurographics)
  • Journal reviewer:
    • IEEE Transaction on Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
    • International Journal of Computer Vision (IJCV)
    • Pattern Recognition (PR)
  • Other Academic service:
    • Understanding HANDS in Action Workshop Organizer (ECCV 2024, ICCV 2025, ECCV 2026)


  • Fundings and Awards
  • Challenge of Multimodal Robot Learning, internUtopia and Real-world Track 1: Manipulation, Third Place Award, IROS 2025
    2025.10 | Team RoboSoul; Xinkai Wang, Chenyi Wang, Fucheng Zhang, Xinyu Zhan, Bokai Lin, Yifu Xu; supervised by Lixin Yang
  • 国家自然科学基金青年科学基金项目(C类)2026.01 - 2028.12,国家自然科学基金委员会,主持
  • 上海交通大学Explore X基金,2025.01-2027.12,上海交通大学,主持
  • 上海市启明星扬帆计划,2024.12-2027.11,上海市科学技术委员会,主持
  • 上海市科委“科技创新行动计划”项目,2023.12-2025.11,上海市科学技术委员会,课题负责人
  • 2023年吴文俊人工智能科学技术奖,自然科学奖一等奖,2023,第四完成人(第一学生完成人)


  • website template