Zhiyao He

Zhiyao He

Robot Learning · Embodied AI

Building robots that perceive, reason, and act in the real world, from simulation to real hardware.

Seeking a PhD in robot learning · 2027 start
Download CV GitHub LinkedIn Email Blog
Zhiyao He, portrait

About

I am a robotics and AI engineer working on embodied AI. I build robot pipelines that connect perception, language models, planning and control, taking systems from simulation in Gazebo, MuJoCo and Isaac Gym to real hardware.

My most recent system is voice-controlled navigation and manipulation on the Unitree Go2-W quadruped with a D1 arm. Alongside that I build offline, on-device voice agents for service robots. My goal is to do a PhD in robot learning, especially on how robots can learn to perceive, reason, and act reliably in real-world environments.

Based in the Hengqin Island & Macau area. M.S. from Hohai University. TOEFL 100/120, CEFR C1.

Selected Projects

Perception + Manipulation · my work

Voice-Controlled Navigation & Manipulation on Unitree Go2-W

Go2-W quadruped + D1 arm · team project with Macau Millennium College · ROS 2

A quadruped that hears a spoken request, navigates to a target, and grasps it. I owned the perception and manipulation halves of the system.

Perception
Fine-tuned YOLO11 by transfer learning to detect a specific bottle; a wrist-mounted depth camera and 2D-to-3D transform locate the target in the world frame.
Manipulation
A grasping pipeline on MoveIt 2: KDL inverse kinematics, OMPL / RRTConnect planning, FCL collision checking, planning a collision-free path and executing the grasp.
Integration
Teammates built SLAM navigation and the ASR / LLM / TTS pipeline; we defined the interfaces and integrated the full system, tested in MuJoCo and Gazebo and on the real robot.
ROS 2YOLO11RealSenseMoveIt 2KDLOMPL / RRTConnectFCLMuJoCoGazebo
Real robot · nav + grasp + deliver
Offline on-device agent · my work

Offline Voice RAG Assistant on AgiBot Lingxi X2

In-store retail assistant · industry project for CTM (Macau Telecom) · runs fully offline on Jetson

Customers ask about products, plans and pricing by voice; the robot answers with no network dependency, so it stays reliable during live events.

Pipeline
A bridge node streams the X2's microphone audio to a local FastAPI backend running local ASR (FunASR / Paraformer-zh), a local LLM (Qwen2.5-7B via Ollama), and pgvector RAG over a PostgreSQL product database.
Output
Answers are spoken through the X2's built-in offline TTS. An online variant integrates with third-party business systems to act on live backend data.
JetsonFastAPIFunASRQwen2.5-7BOllamaPostgreSQLpgvectorRAGoffline TTS
Real robot · voice assistant
RL locomotion + bring-up · my work

RL Locomotion & Robot Bring-up on the AKI Humanoid

Company education product · team reproduction of Unitree's open-source Qmini

Bringing a reproduced humanoid to life and teaching it to walk in simulation.

Bring-up
Brought up and debugged the control system and power system on the reproduced robot.
Learning
Ran reinforcement-learning locomotion training in Isaac Gym; set up Gazebo, MuJoCo and Isaac Gym environments for AKI and the Unitree G1 / R1 humanoids.
Isaac GymReinforcement LearningMuJoCoGazeboUnitree G1 / R1control systems
Simulation

Research Interests

01
Robot Learning & Reinforcement Learning
Learning locomotion and manipulation policies in simulation and transferring them to hardware.
02
Vision-Language-Action & World Action Models
Mapping language and perception to robot actions, and learning world models that predict how actions change the environment.
03
Manipulation & Grasping
Reliable grasping and motion planning under real-world perception noise.
04
Simulation, SLAM & Navigation
Building the sim-to-real and navigation stack that manipulation depends on.

Publications & Patent

My published work is from an earlier machine-learning and hydrology research period; it reflects a full research pipeline from data to peer review.

2019
Runoff Prediction Method Based on Adaptive Elman Neural Network
Chenming Li, Lei Zhu, Zhiyao He, Hongmin Gao, Yao Yang, Dan Yao, Xiaoyu Qu
Water Paper
2017
Annual Average Runoff Ensemble Forecast for Jinping I-Stage Hydropower Station Based on Elman Neural Network
Zhiyao He, Mingxiang Yang, Hao Wang, et al.
Water Resources and Power Paper
2020
Medium- and Long-term Runoff Forecasting Method Based on Mutual Information, Kernel PCA, and Elman Neural Network
Zhiyao He · Chinese National Invention Patent, granted 2020
Patent CN107463993B Google Patents

Experience

Mar 2026 – Present

AI Engineer

WellWin Technology, Hengqin Island & Macau, China
  • Built the perception and grasping pipelines for the Go2-W voice manipulation system.
  • Built offline and online voice agents for the AgiBot Lingxi X2 on Jetson.
  • Set up simulation and ran RL locomotion training for AKI and Unitree G1 / R1 humanoids.
Jul 2019 – Feb 2026

Independent Study & Research

Nanjing, China
Completed a full CS degree at University of the People (GPA 4.00); second journal paper published and invention patent granted during this period. Admitted to the M.Sc. Data Science at Trinity College Dublin (2025), declined for funding reasons.
Jul 2018 – Jun 2019

Software Engineer

Huawei Technologies · Nanjing R&D Institute, Nanjing, China
Intelligent security systems team (face, vehicle and other recognition products): system development, deployment and feature validation; cross-team debugging and system-level testing.
Jun 2016 – Jun 2017

Research Intern

China Institute of Water Resources and Hydropower Research, Beijing, China
Built a complete runoff-prediction pipeline: data collection and cleaning, feature engineering, model development, tuning, training and evaluation. Produced two journal papers and one patent.

Education

2015–2018

M.S., Signal & Information Processing

Hohai University, Nanjing, China
Outstanding Graduate Student (research-based degree). Thesis on machine learning for runoff forecasting.
2011–2014

B.Eng., Electrical Engineering & Automation

Anhui Agricultural University, Hefei, China
CGPA 3.65 / 4.00. Finished the four-year program in three years, the only student in the cohort to do so.
2021–2024

A.S., Computer Science

University of the People, USA
CGPA 4.00 / 4.00. President's List, 12 terms.

Technical Skills

Programming & systems
PythonC++LinuxDockerGit
Robotics
ROS / ROS 2MoveIt 2Nav2GazeboMuJoCoIsaac GymSLAMNavigationPerception
AI & robot learning
PyTorchMachine LearningComputer VisionReinforcement LearningLLMsRAGVLA
Robot platforms
Unitree G1 / B2 / Go2 / Go2-WAgiBot Lingxi X2

Honors & Awards

Contact

Open to PhD positions in robot learning.
Happy to talk about embodied intelligence and physical AI.
zhiyaohe@outlook.com GitHub CV