I am a Sr. Research Scientist at NVIDIA Seattle Robotics Lab. I received my Doctoral degree in Electronic Engineering from the Chinese University of Hong Kong in 2018 (advisor: Prof. Xiaogang WANG, co-advisor: Prof. Wanli OUYANG). Previously, I worked as a visiting student at Robotics Institute, Carnegie Mellon University (10/2017-4/2018) with Prof. Abhinav Gupta.
Research interests: computer vision, machine learning and their applications to robotics.

Google Scholar

GitHub

Linkedin

News

• One paper is accepted to CVPR 2023. (April 1st, 2023)
• Two papers are accepted to CoRL 2022 and IROS 2022/R-AL. (Oct, 2022)
• Two papers are accepted to ICRA 2022. (May, 2022)
• Our 6D grasping (GA-DDPG) paper is accepted to CoRL 2021.
• Our H2R handover paper won the 2021 IEEE ICRA Best Paper Award on HRI. (June 2, 2021)
• Our DexYCB hand grasping object benchmark is accepted to CVPR 2021. (March 3, 2021)
• Our reactive handovers paper is accepted to ICRA 2021. (Feb 28, 2021)
• Two papers are accepted to IROS 2020. (July 1, 2020)
• One paper is accepted to ICRA 2020. (Jan 21, 2020)
• One paper is accepted to ICLR 2019. (Jan 7, 2019)
• I join the NVIDIA Seattle Robotics Lab as a Research Scientist. (Jan 2, 2019)
• One paper is accepted to CVPR 2018. (Feb 28, 2018)
• Our team won the 2nd places of the PoseTrack Challenge 2017 (Technical report | Leaderboard: BUTDS and BUTD2). (Oct 23, 2017)
• I am currently a visiting scholar at CMU working with Prof. Abhinav Gupta (from Oct 15 2017 to April 30 2018).
• Two papers are accepted to ICCV 2017. (July 17, 2017)
• The code for our CVPR 2017 paper has been made publicly available now. (April 5, 2017)
• One paper is accepted to CVPR 2017. (March 18, 2017)
• I have fullfilled the candidacy requirement and will be advanced to Ph.D. (post-candidacy) stage. (June 3, 2016)

Publication

Conferences & Preprints

* equal contribution

• FoundationPose: Unified 6d pose estimation and tracking of novel objects
  Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield
  Computer Vision and Pattern Recognition (CVPR), Seattle, Washington, 2024. (Highlight, AC 2.8%)

  arXiv | Project Page | Code | Dataset

• SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers
  Sammy Christen*, Lan Feng*, Wei Yang, Yu-Wei Chao, Otmar Hilliges, Jie Song
  International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 2024.
  

  arXiv | Project Page

• AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
  Yuzhe Qin, Wei Yang, Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang, Yu-Wei Chao, Dieter Fox
  Robotics: Science and Systems (RSS), Daegu, Republic of Korea, 2023.

  arXiv | Project Page | Code: Web visualizer | Retargeting

• Learning Human-to-Robot Handovers from Point Clouds
  Sammy Christen, Wei Yang, Claudia Pérez-D'Arpino, Otmar Hilliges, Dieter Fox, Yu-Wei Chao
  Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, 2023. (Highlight, AC 2.5%)

  arXiv | Project Page | Video

• Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation
  Qiuyu Chen, Karl Van Wyk, Yu-Wei Chao, Wei Yang, Arsalan Mousavian, Abhishek Gupta, Dieter Fox
  Conference on Robot Learning (CoRL), Auckland, NZ, 2022.

  arXiv | Project Page | OpenReview

• Learning Perceptual Concepts by Bootstrapping from Human Queries
  Andreea Bobu, Chris Paxton, Wei Yang, Balakumar Sundaralingam, Yu-Wei Chao, Maya Cakmak, Dieter Fox
  International Conference on Intelligent Robots and Systems (IROS), Kyoto, 2022
  IEEE Robotics and Automation Letters (RA-L), 2022
  International Conference on Robotics and Automation (ICRA), Scaling Robot Learning Workshop, Philadelphia (PA), USA, 2022. Spotlight
  

  arXiv | Code | Project Page

• Model Predictive Control for Fluid Human-to-Robot Handovers
  Wei Yang*, Balakumar Sundaralingam*, Chris Paxton*, Iretiayo Akinola, Yu-Wei Chao, Maya Cakmak, Dieter Fox
  International Conference on Robotics and Automation (ICRA), Philadelphia (PA), USA, 2022.
  

  arXiv | Project Page

• HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
  Yu-Wei Chao, Chris Paxton, Yu Xiang, Wei Yang, Balakumar Sundaralingam, Tao Chen, Adithyavairavan Murali, Maya Cakmak, Dieter Fox
  International Conference on Robotics and Automation (ICRA), Philadelphia (PA), USA, 2022.
  

  arXiv | Project Page

• Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds
  Lirui Wang, Yu Xiang, Wei Yang, Arsalan Mousavian and Dieter Fox
  Conference on Robot Learning (CoRL), London, UK, 2021.

  arXiv | Project Page | Code | OpenReview

• DexYCB: A Benchmark for Capturing Hand Grasping of Objects
  Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S. Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, Jan Kautz, Dieter Fox
  Computer Vision and Pattern Recognition (CVPR), Virtual, 2021.

  arXiv | Project Page | Code | Video

• Reactive Human-to-Robot Handovers of Arbitrary Objects
  Wei Yang, Chris Paxton, Arsalan Mousavian, Yu-Wei Chao, Maya Cakmak, Dieter Fox
  International Conference on Robotics and Automation (ICRA), Xi'an, China, 2021.
  

  arXiv | Project Page | Short video (3 min) | Long video (12 min) | NVIDIA blog

  🏆 Best Paper Award on Human-Robot Interaction (HRI)
  
• Human Grasp Classification for Reactive Human-to-Robot Handovers
  Wei Yang* , Chris Paxton*, Maya Cakmak and Dieter Fox
  International Conference on Intelligent Robots and Systems (IROS), On-Demand, 2020

  arXiv | Short video (1 min) | Long video (15 min)

  Press coverage: NVIDIA | VentureBeat | AIM | The Robot Report | Process Online

• Collaborative Interaction Models for Optimized Human-Robot Teamwork
  Adam Fishman, Chris Paxton, Wei Yang, Nathan Ratliff, Byron Boots, Dieter Fox
  International Conference on Intelligent Robots and Systems (IROS), On-Demand, 2020

  arXiv | Project Page | Video

• DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System
  Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff and Dieter Fox
  International Conference on Robotics and Automation (ICRA), Paris, France, 2020.

  arXiv | Project Page

• Visual Semantic Navigation using Scene Priors
  Wei Yang, Xiaolong Wang, Ali Farhadi, Abhinav Gupta, Roozbeh Mottaghi
  International Conference on Learning Representations (ICLR), New Orleans, Louisiana, 2019.

  arXiv | Video | Code ^‡

  (^‡ Pytorch re-implementation in a CVPR'19 paper. Our method is indicated as Scene Priors.)
  
• 3D Human Pose Estimation in the Wild by Adversarial Learning
  Wei Yang, Wanli Ouyang, Xiaolong Wang, Jimmy Ren, Hongsheng Li, Xiaogang Wang
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, Utah, 2018.

  arXiv | Video

• Learning Feature Pyramids for Human Pose Estimation
  Wei Yang, Shuang Li, Wanli Ouyang, Hongsheng Li, Xiaogang Wang
  International Conference on Computer Vision (ICCV), Venice, Italy, 2017 (AC 28.9%).
  arXiv | Code
• Identity-Aware Textual-Visual Matching with Latent Co-attention
  Shuang Li, Tong Xiao, Hongsheng Li, Wei Yang, Xiaogang Wang
  International Conference on Computer Vision (ICCV), Venice, Italy, 2017 (AC 28.9%).
  arXiv
• Towards Multi-Person Pose Tracking: Bottom-up and Top-down Methods
  Sheng Jin, Xujie Ma, Zhipeng Han, Yue Wu, Wei Yang, Wentao Liu, Chen Qian, Wanli Ouyang
  International Conference on Computer Vision (ICCV) PoseTrack Workshop, Venice, Italy, 2017.
  PDF | Leaderboard (BUTDS and BUTD2)
  
• Multi-Context Attention for Human Pose Estimation
  Xiao Chu*, Wei Yang*, Wanli Ouyang, Cheng Ma, Alan L. Yuille, Xiaogang Wang
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, 2017 (AC 29.6%).
  PDF | Code
• End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation
  Wei Yang, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Nevada, 2016 (Oral, AC 3.9%).
  PDF | Project
• Multi-task Recurrent Neural Network for Immediacy Prediction
  Xiao Chu, Wanli Ouyang, Wei Yang and Xiaogang Wang
  in Proceedings of IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015 (Oral, AC 3.3%).
  PDF | Project | Dataset
• Clothing Co-Parsing by Joint Image Segmentation and Labeling
  Wei Yang, Ping Luo, and Liang Lin
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, 2014 (AC 29.9%).
  PDF | Dataset
• Data-Driven Scene Understanding by Adaptive Exemplar Retrieval
  Xionghao Liu, Wei Yang, Ya Li, Liang Lin, and Jian-Huang Lai,
  Proc. of IEEE International Conference on Multimedia and Expo (ICME), Chengdu, China, 2014 (AC 29.6%).
  arXiv
• Learning Contour-Fragment-based Shape Model with And-Or Tree Representation
  Liang Lin, Xiaolong Wang, Wei Yang, and Jian-Huang Lai
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, 2012 (AC 24.1%).
  PDF
• Interactive CT image segmentation with online discriminative learning
  Wei Yang, Xiaolong Wang, Liang Lin, Chengying Gao
  Proc. of IEEE International Conference on Image Processing (ICIP), Brussels, Belguim, 2011 (AC 40.6%).
  PDF | Project | Dataset

Journal Papers

• Progressively diffused networks for semantic visual parsing.
  Ruimao Zhang, Wei Yang, Zhanglin Peng, Pengxu Wei, Xiaogang Wang, and Liang Lin.
  Pattern Recognition (PR), 2019.
  PDF | Arxiv
• Clothes Co-Parsing via Joint Image Segmentation and Labeling with Application to Clothing Retrieval.
  Xiaodan Liang, Liang Lin, Wei Yang, Ping Luo, Junshi Huang, and Shuicheng Yan.
  IEEE Transactions on Multimedia (T-MM), 2016.
  PDF
• Inference With Collaborative Model for Interactive Tumor Segmentation in Medical Image Sequences.
  Liang Lin, Wei Yang, Chenglong Li, Jin Tang, Xiaochun Cao.
  IEEE Transactions on Cybernetics (T-Cybernetics), 2015.
  PDF | Project | Dataset
• Data-Driven Scene Understanding with Adaptively Retrieved Exemplars.
  Xionghao Liu, Wei Yang, Liang Lin, Qing Wang, Zhaoquan Cai, Jian-Huang Lai.
  IEEE Multimedia, 2015.
  Project | PDF | Code
• Discriminatively Trained And-Or Graph Models for Object Shape Detection.
  Liang Lin, Xiaolong Wang, Wei Yang, and JianHuang Lai.
  IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 37(5): 959-972, 2015.
  Project | PDF | Code | Dataset

Experiences

• Research Scientist

  NVIDIA Research, Seattle, WA, USA
  Jan 2019 - present

• 

  Visiting Scholar

  Carnegie Mellon University, Pittsburgh, PA, USA
  November 2017 - April 2018

• 

  Software Engineer (intern)

  Tencent, Shenzhen, China
  July 2010 - September 2010

Professional Activities

I serviced as a reviewer for the following conferences and journals:

• Computer Vision and Pattern Recognition (CVPR), 2018-2021
• European Conference on Computer Vision (ECCV), 2018, 2020
• International Conference on Computer Vision (ICCV), 2017, 2019, 2021
• Asian Conference on Computer Vision (ACCV), 2018
• IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018
• International Joint Conference on Artificial Intelligence (IJCAI), 2017
• IEEE Transactions on Circuits and Systems for Video Technology (TPAMI)
• IEEE Transactions on Multimedia (TMM)
• IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)
• IEEE Transaction on Cybernetics (TCYB)
• IEEE Transactions on Artificial Intelligence (TAI)
• International Journal of Computer Vision (IJCV)
• Elsevier Journal of Neurocomputing (NEUCOM)
• Elsevier Journal of Pattern Recognition (PR)
• Elsevier Journal of Computer Vision and Image Understanding (CVIU)
• IET Image Processing

Teaching

Teaching assistant at CUHK for the following courses:

• 2017, Spring. Introduction to Deep learning (ELEG 5491).
• 2016, Fall. Complex Analysis and Differential Equations (ENGG 2420A).
• 2016, Spring. Probability and Statistics for Engineers (ENGG 2430D).
• 2015, Fall. Complex Analysis and Differential Equations for Engieers (ENGG 2420A).
• 2015, Summer. Solidworks.
• 2014, Fall. Digital Circuits and Systems (ELEG2201).

Selected Awards

• 2021 IEEE ICRA Best Paper Award on Human-Robot Interaction, 2021
• PoseTrack Challenge 2017, 2nd place, 2017.
• Tutor with Commendation, The Chinese University of Hong Kong, 2016/17.
• Green Walkers Award, The Chinese University of Hong Kong, July 2017.
• Scholarships
  • National Scholarship, 2012.
  • The Third Prize Scholarship, 2010.
  • The Second Prize Scholarship, 2008-2009.
• Amway University IT Project Competition, Silver Medal, 2011.
• Computer Programming Competition of Sun Yat-sen University, Third prize, 2009.

Talks

• Vision-Based Human-to-Robot Object Handovers(中文), TechBeat, 2021
• Human Pose Estimation with Deep Learning, SIAT, Shenzhen, China, 2018
• Human Pose Estimation with Deep Learning, VALSE, virtual, 2018

Education

CUHK
(2014 - 2018)

SYSU
(2007 - 2014)

Contact

• Email: platero.yang (at) gmail.com
• Address: 6th Floor, 4545 Roosevelt Way NE, Seattle, WA 98105

(last update: April 2023)