Lixin YANG (杨理欣)

Bio.  I’m a Research Assistant Professor in Shanghai Jiao Tong University (SJTU), School of Artificial Intelligence (SAI). Since 2019, I have been part of the Machine Vision and Intelligence Group under the supervision of Prof. Cewu Lu, where I obtained my Ph.D. in 2023. Prior to that, I received my M.S degree at the Intelligent Robot Lab in SJTU. My research interests include 3D Vision and Robotics. Currently, I am focusing on modeling and imitating the hand manipulating objects, including 3D hand | object pose | shape estimation, grasp | motion generation, imitation learning, dexterous manipulation.

Join Us.  I am looking for Master Student at SJTU SAI and self-motivated research interns. Contact me if you are interested in the above topics.

诚意招收研究实习生(带薪), 我们一起做有意思的科研。

Email  /  Google Scholar  /  GitHub  /  Twitter

*=equal contribution, #=corresponding author

a bidirectionally expanded learning approach that enhances auto-regressive policies for robotic manipulation. It employs a lightweight encoder-only architecture to iteratively unfold the action sequence from an initial single frame into the target sequence in a coarse-to-fine manner with logarithmic-time inference.

AirExo-2, a low-cost exoskeleton system for large-scale in-the-wild demonstration collection. It transforms the collected in-the-wild demonstrations into pseudo-robot demonstrations. RISE-2, a generalizable imitation policy that integrates 2D and 3D perceptions.

a novel category-level garment pose tracking framework that transforms 3D point-wise learning into 2D warping deformation learning using 3D-2D projection and 2D-3D reconstruction.

A two cascaded diffusion processes for object motion generation and robot action generation under object motion guidance.

Part-Aware Diffusion Model for manipulating articulated objects in real images. It uses Abstract 3D Models for efficient object representation and dynamic feature maps to transfer appearance while generating novel views or parts.

A MLLM-based method that infuses language instructions into grasp generation; & A new language-pose dataset, CapGrasp, featuring detailed caption of grasping poses.

A 4D motion dataset focusing on bimanual object manipulation tasks involved in complex daily activities; & A three-tiered task abstraction: Object Affordance, Primitive Task, and Complex Task, to systematically organize manipulation tasks.

A full-body human motion dataset that captures text-guided desktop object rearrangement through MoCap and AR glasses; & A pipeline for generating avatar's motion of object rearrangement driven by text instruction.

A NeuS-based reconstruction method for both object surface and texture. Can be used to reconstruct object with complex geometry and texture given a hand-held object video.

A single-view hand-held object reconstruction method that exploits the categorical shape prior to reconstruct the shape of intra-class objects; & A new synthetic dataset, COMIC, that contains the category-level collection of objects with diverse shape, materials, interacting poses, and viewing directions.

A multi-view hand mesh recovery (HMR) method with Transformer. It leverages the "power of points", including Basis Points Set, point's positional encoding and point-Transformer, to unify and merge information from sparsely arranged cameras.

A MANO-derived hand model that contains 325 exquisite hand-crafted texture maps (vary in appearance and cover different kinds of blemishes, make-ups, and accessories).

A dataset that focuses on human grasp based on object's affordance. It contains two knowledge base: 1) Object affordance knowledge (Oak) and 2) Interaction knowledge (Ink).
A new model: Tink, for transferring interaction pose from one object to another.

An online data syhthesis tool for articulated hand(-object) pose estimation. An grasping systhesis method that can generate dexterous hand grasping poses for arbitrary object.

A novel contact representation (CPF) that used to imporve physical hand-object interaction. A hybrid learning-fitting framework (MIHO) that aligns the top-down pose estimation with bottom-up contact modeling.

A novel contact representation (CPF) that used to imporve physical hand-object interaction. A hybrid learning-fitting framework (MIHO) that aligns the top-down pose estimation with bottom-up contact modeling.

A hybrid inverse kinematics method for 3D body mesh recovery, combining 3D keypoint estimation and body mesh recovery. HybrIK-X extends this to model hands and faces, offering fast, accurate whole-body pose estimation.