Lixin YANG (杨理欣)

Research Assistant Professor
Shanghai Jiao Tong University, School of Artificial Intelligence
上海交通大学人工智能学院 助理研究员, 硕士生导师
Member of Machine Vision and Intelligence Group (MVIG) at SJTU

Email: siriusyang at sjtu dot edu dot cn
Office: Bldg. SAI, No. 1954 Huashan Rd., Xuhui Dist., Shanghai, 200230, China

About.  I’m a Research Assistant Professor in Shanghai Jiao Tong University (SJTU), affiliated with the School of Artificial Intelligence (SAI), where I joined in September 2024. I obtained Ph.D. degree in Computer Science from SJTU in 2023, advised by Prof. Cewu Lu at the Machine Vision and Intelligence Group and M.S. degree in Mechanical Engineering, SJTU. My research interests include 3D Vision and Robotics. Currently, I am focusing on modeling and imitating the hand manipulating objects, including 3D hand | object pose | shape estimation, grasp | motion generation, imitation learning, dexterous manipulation.

Join Us.  I am looking for Master Student at SJTU SAI and self-motivated research interns. Contact me if you are interested in the above topics.
诚意科研研究实习生(带薪), 我们一起做有意思的科研。

profile photo

News
  • [2026.06] LaMP and ChronoFlow have been accepted to ECCV 2026 🇸🇪
  • [2026.06] LIDEA has been accepted to IROS 2026 🇺🇸
  • [2026.01] TrajBooster & Motion-before-Action will be presented at ICRA 2026 🇦🇹
  • [2025.07] Generalizable Multi-view Hand Reconstruction POEMv2 is accepted by TPAMI 2025
  • [2025.06] Dense Policy is accepted by ICCV 2025 🇺🇸
  • [2025.05] Motion-before-Action is accepted by RA-L.
  • [2025.01] HybrIK-X is accepted by TPAMI.
  • [2024.12] Invited Talk at ROSCon China Workshop, thanks Yao Mu for hosting.
  • [2024.09] One paper on articulated object image manipulation got accepted by NeurIPS 2024 🇨🇦.
  • [2024.07] One paper: SemGrasp is accepted by ECCV 2024 🇮🇹.
  • [2024.02] One paper: OakInk2 is accepted by CVPR 2024 🇺🇸.
  • [2024.02] The Contact Potential Field is accepted by TPAMI.
  • [2023.12] One paper: FAVOR is accepted by AAAI 2024 🇨🇦.
  • [2023.10] One paper: Color-NeuS is accepted by 3DV 2024 🇨🇭.
  • [2023.08] I defend my doctoral thesis and earn my Ph.D!
  • [2023.08] I am honored to be an invited speaker at the HANDS workshop at ICCV 2023.
  • [2023.07] One paper: CHORD is accepted by ICCV 2023 🇫🇷.
  • [2023.06] Invited Talk at 智东西公开课 | 研讨会: 3D手部重建及具身智能交互. 视频 (中文).
  • [2023.02] One paper: POEM is accepted by CVPR 2023.
  • [2022.10] I have taken the wonderful journey of marriage alongside my cherished wife.
  • [2022.10] Invited Talk at International Digital Economy Academy (IDEA), Thanks Ailing Zeng for hosting.
  • [2022.09] One paper: DART got accepted by NeurIPS 2022 - Datasets and Benchmarks Track.
  • [2022.07] Invited Talk at 智东西公开课 | AI新青年讲座: 基于图像的手物交互重建与虚拟人手生成. 视频 (中文).
  • [2022.04] Invited Talk at MPI-IS Perceiving Systems. Thanks Yuliang Xiu for hosting. (info).
  • [2022.03] Two paper were accepted by CVPR 2022: one Oral, one poster.
  • [2021.07] One paper got accepted by ICCV 2021.
Publications

*=equal contribution, #=corresponding author

No publications under the selected topic.
LaMP: Learning Vision-Language-Action Policy with 3D Scene Flow as Latent Motion Prior
Xinkai Wang, Chenyi Wang, Yifu Xu, Mingzhe Ye, Fucheng Zhang, Jialin Tian, Xinyu Zhan, Lifeng Zhu, Cewu Lu, Lixin Yang#
ECCV, 2026 project arXiv

A dual-expert Vision-Language-Action framework that injects one-step partially denoised 3D scene flow as a latent motion prior.

ChronoFlow-Policy: Unifying Past-Current-Future Interaction Flow in Visuomotor Policy Learning
Bokai Lin*, Yifu Xu*, Xinyu Zhan, Hongjie Fang, Jialin Tian, Fu-Cheng Zhang, Yong-Lu Li, Cewu Lu, Lixin Yang#
ECCV, 2026

A diffusion-based visuomotor policy that co-trains action generation with unified past-current-future object-gripper keypoint flows for robust long-horizon manipulation.

LIDEA: Human-to-Robot Imitation Learning via Implicit Feature Distillation and Explicit Geometry Alignment
Yifu Xu*, Bokai Lin*, Xinyu Zhan, Hongjie Fang, Yong-Lu Li, Cewu Lu, Lixin Yang#
IROS, 2026 project arXiv

A human-to-robot learning that bridges the embodiment gap with transitive 2D feature distillation and explicit 3D geometry alignment.

TrajBooster: Boosting Humanoid Whole-Body Manipulation via Trajectory-Centric Learning
Jiacheng Liu*, Pengxiang Ding*, Qihang Zhou, Yuxuan Wu, Da Huang, Zimian Peng, Wei Xiao, Weinan Zhang, Lixin Yang#, Cewu Lu#, Donglin Wang#,
ICRA, 2026 project arXiv

A cross-embodiment framework that transfers wheeled-humanoid data to bipedal VLA models via morphology-agnostic 6D end-effector trajectories and a heuristic-enhanced online DAgger controller.

Motion Before Action: Diffusing Object Motion as Manipulation Condition
RA-L, 2025, ICRA, 2026 project arXiv code

A two cascaded diffusion processes for object motion generation and robot action generation under object motion guidance.

Dense Policy: Bidirectional Autoregressive Learning of Actions
ICCV, 2025 project arXiv

A bidirectional robotic autoregressive policy, which infers trajectories by gradually expanding actions from sparse keyframes, demonstrated exceeding diffusion policies.

AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons
Hongjie Fang*,  Chenxi Wang*,  Yiming Wang*, Jingjing Chen*, Shangning Xia, Jun Lv, Zihao He, Xiyan Yi, Yunhan Guo, Xinyu Zhan, Lixin Yang, Weiming Wang, Cewu Lu#, Hao-Shu Fang#
CoRL, 2025 (Oral Presentation) project arXiv

A low-cost exoskeleton system that captures human in-the-wild manipulation and directly converts the wearer's actions into scalable pseudo-robot demonstrations.

Multi-view Hand Reconstruction with a Point-Embedded Transformer
Lixin YangLicheng Zhong,  Pengxiang Zhu,  Xinyu Zhan,  Junxiao Kong,  Jian XuCewu Lu#
TPAMI, 2025 paper arXiv code

POEM-v2 a generalizable multi-view 3D hand reconstruction model trained on large-scale multi-view datasets. It enables accurate, flexible, and occlusion-robust hand mesh recovery across arbitrary multi-view setups.

HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery
Jiefeng Li, Siyuan Bian, Chao Xu, Zhicun Chen, Lixin Yang, Cewu Lu#
TPAMI, 2025 arXiv

Extends HybrIK to efficient whole-body mesh recovery by jointly modeling body, hands, and face.

SemGrasp: Semantic Grasp Generation via Language Aligned Discretization
ECCV, 2024 (Oral Presentation) project arXiv

A MLLM-based method that infuses language instructions into grasp generation; & A new language-pose dataset, CapGrasp, featuring detailed caption of grasping poses.

OakInk2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion
Xinyu Zhan*, Lixin Yang*, Yifei Zhao, Kangrui Mao, Hanlin Xu, Zenan Lin, Kailin Li, Cewu Lu#
CVPR, 2024 project arXiv

A 4D motion dataset focusing on bimanual object manipulation tasks involved in complex daily activities with three-tiered task abstraction: Object Affordance, Primitive Task, and Complex Task.

Learning a Contact Potential Field to Model the Hand-Object Interaction
Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Junming Zhang, Jiefeng Li, Cewu Lu#
TPAMI, 2024 paper

Extends CPF into a unified spring-mass-style contact field that couples attraction, repulsion, and geometry constraints for robust hand-object reconstruction.

Color-NeuS: Reconstructing Neural Implicit Surfaces with Color
Licheng Zhong*,  Lixin Yang*,  Kailin LiHaoyu ZhenMei HanCewu Lu#
3DV, 2024 project arXiv code data

Reconstructs colored neural implicit surfaces by decoupling geometry from view-dependent shading for accurate colored mesh extraction.

FAVOR: Full-Body AR-Driven Virtual Object Rearrangement Guided by Instruction Text
Kailin Li*, Lixin Yang*, Zenan Lin, Jian Xu, Xinyu Zhan, Yifei Zhao, Pengxiang Zhu, Wenxiong Kang, Kejian Wu, Cewu Lu#
AAAI, 2024 project arXiv code data

A full-body motion dataset and pipeline for text-guided desktop object rearrangement with avatars.

CHORD: Category-level in-Hand Object Reconstruction via Shape Deformation
ICCV, 2023 project arXiv code tool

A category-level in-hand object reconstruction method with shape deformation priors and a synthetic COMIC dataset for diverse interactions.

POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo
CVPR, 2023 arXiv code

A Transformer-based multi-view hand reconstruction method that uses point embeddings to fuse sparse camera observations into accurate hand meshes.

DART: Articulated Hand Model with Diverse Accessories and Rich Textures
Daiheng Gao*,   Yuliang Xiu*,  Kailin Li*,  Lixin Yang*, Feng Wang, Peng Zhang, Bang Zhang, Cewu Lu, Ping Tan
NeurIPS, 2022 - Datasets and Benchmarks Track project arXiv code video

A richly textured MANO-derived articulated hand model with accessories, appearance variation, and benchmark-ready assets.

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction
Lixin Yang*, Kailin Li*, Xinyu Zhan*, Fei Wu, Anran Xu, Liu Liu, Cewu Lu#
CVPR, 2022 project paper arXiv code

A large-scale hand-object interaction knowledge base with affordance annotations and a model for transferring grasps across objects.

ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis
Lixin Yang*, Kailin Li*, Xinyu Zhan, Jun Lv, Wenqiang Xu, Jiefeng Li, Cewu Lu#
CVPR, 2022 (Oral Presentation) paper arXiv code

An online exploration and synthesis framework that improves articulated hand-object pose estimation and generates dexterous grasps for novel objects.

HandTailor: Towards High-Precision Monocular 3D Hand Recovery
Jun Lv, Wenqiang Xu, Lixin Yang, Sucheng Qian, Chongzhao Mao, Cewu Lu#
BMVC, 2021 paper arXiv code
CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction
Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, Cewu Lu#

Models hand-object contact as a spring-mass-like potential field, enabling geometry-aware optimization for physically plausible interaction reconstruction.

HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation
Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, Cewu Lu#
CVPR, 2021 project paper supp arXiv code

A hybrid analytical-neural inverse kinematics method for fast and accurate 3D human pose and shape recovery.

BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks
Lixin Yang, Jiasen Li, Wenqiang Xu, Yiqun Diao, Cewu Lu#
BMVC, 2020 paper arXiv code
Other Collaborations
GaPT-DAR: Category-level Garments Pose Tracking via Integrated 2D Deformation and 3D Reconstruction
Li Zhang, Mingliang Xu, Jianan Wang, Qiaojun Yu, Lixin Yang, Yong-Lu Li, Cewu Lu, Rujing Wang, Liu Liu#
CVPR, 2025 project paper
General Articulated Objects Manipulation in Real Images via Part-Aware Diffusion Process
Zhou Fang, Yong-Lu Li#, Lixin Yang, Cewu Lu#
NeurIPS, 2024 paper


Talks
Sep 2025
Human-Robot Data Companion: Pipeline and Representation.
SII TechFest workshop: Embodied AI Reasoning and Scaling.
Jul 2025
Tutorial: Multimodal Embodied Perception and Action Prediction
Dec 2024
Empowering Object Manipulation with Human Knowledge.
Aug 2023
Paving the Way for Understanding Human Interactions with Objects: The OakInk2 Dataset.
ICCV 2023 HANDS Workshop.
Sep 2023
Jun 2023
3D手部重建及具身智能交互.
Oct 2022
Inferring Human Hand-Object Interaction through Visual Input
Jul 2022
基于图像的手物交互重建与虚拟人手生成
Apr 2022
Leverage Kinematic and Contact constraints for understanding hand-object interaction
Awards
2025.10
IROS 2025, Challenge of Multimodal Robot Learning,
internUtopia and Real-world Track 1: Manipulation, Third Place Award
Team RoboSoul
Xinkai Wang, Chenyi Wang, Fucheng Zhang, Xinyu Zhan, Bokai Lin, Yifu Xu, Lixin Yang*
2023
吴文俊人工智能科学技术奖,自然科学奖一等奖
行为视觉理解
卢策吾、马利庄、欧阳万里、杨理欣、方浩树
Fundings
2026 - 2028
国家自然科学基金青年科学基金项目(C类)
国家自然科学基金委员会,主持
2025 - 2027
上海交通大学 Explore X 基金
上海交通大学,主持
2024 - 2027
上海市启明星扬帆计划
上海市科学技术委员会,主持
2023 - 2025
上海市科委“科技创新行动计划”项目
上海市科学技术委员会,课题负责人
Services
Reviewer
Conferences
RSS, CVPR, ICCV, ECCV, NeurIPS, ICLR, ICML, AAAI, SIGGRAPH, Eurographics
Journals
TPAMI, IJCV, PR
Academic service
ECCV 2024, ICCV 2025, ECCV 2026


website template