Zhiyao He

Zhiyao He

Robot Learning · Embodied AI

Building robots that perceive, reason, and act in the real world, from simulation to real hardware.

Seeking a PhD in robot learning · 2027 start
Download CV GitHub LinkedIn Email Blog
Zhiyao He, portrait

About

I am a robotics and AI engineer working on embodied AI. I build robot pipelines that connect perception, language models, planning and control, taking systems from simulation in Gazebo, MuJoCo and Isaac Gym to real hardware.

My most recent system is voice-controlled navigation and manipulation on the Unitree Go2-W quadruped with a D1 arm. Alongside that I build offline, on-device voice agents for service robots. My goal is to do a PhD in robot learning, especially on how robots can learn to perceive, reason, and act reliably in real-world environments.

Based in the Hengqin Island & Macau area. M.S. from Hohai University. TOEFL 100/120, CEFR C1.

Selected Projects

Perception + Manipulation · my work

Voice-Controlled Navigation & Manipulation on Unitree Go2-W

Go2-W quadruped + D1 arm · team project with Macau Millennium College · ROS 2

A quadruped that hears a spoken request, navigates to a target, and grasps it. I owned the perception and manipulation halves of the system.

Perception
Fine-tuned YOLO11 by transfer learning to detect a specific bottle; a wrist-mounted depth camera and 2D-to-3D transform locate the target in the world frame.
Manipulation
A grasping pipeline on MoveIt 2: KDL inverse kinematics, OMPL / RRTConnect planning, FCL collision checking, planning a collision-free path and executing the grasp.
Integration
Teammates built SLAM navigation and the ASR / LLM / TTS pipeline; we defined the interfaces and integrated the full system, tested in MuJoCo and Gazebo and on the real robot.
ROS 2YOLO11RealSenseMoveIt 2KDLOMPL / RRTConnectFCLMuJoCoGazebo
Real robot · nav + grasp + deliver
Offline on-device agent · my work

Offline Voice RAG Assistant on AgiBot Lingxi X2

In-store retail assistant · industry project for CTM (Macau Telecom) · runs fully offline on Jetson

Customers ask about products, plans and pricing by voice; the robot answers with no network dependency, so it stays reliable during live events.

Pipeline
A bridge node streams the X2's microphone audio to a local FastAPI backend running local ASR (FunASR / Paraformer-zh), a local LLM (Qwen2.5-7B via Ollama), and pgvector RAG over a PostgreSQL product database.
Output
Answers are spoken through the X2's built-in offline TTS. An online variant integrates with third-party business systems to act on live backend data.
JetsonFastAPIFunASRQwen2.5-7BOllamaPostgreSQLpgvectorRAGoffline TTS
Real robot · voice assistant
RL locomotion + bring-up · my work

RL Locomotion & Robot Bring-up on the AKI Humanoid

Company education product · team reproduction of Unitree's open-source Qmini

Bringing a reproduced humanoid to life and teaching it to walk in simulation.

Bring-up
Brought up and debugged the control system and power system on the reproduced robot.
Learning
Ran reinforcement-learning locomotion training in Isaac Gym; set up Gazebo, MuJoCo and Isaac Gym environments for AKI and the Unitree G1 / R1 humanoids.
Isaac GymReinforcement LearningMuJoCoGazeboUnitree G1 / R1control systems
Simulation
Curriculum + real-robot ROS 2 · my work

Humanoid Robotics Curriculum on AgiBot Lingxi X2

Education product for Macau junior and senior secondary schools

A full humanoid robotics curriculum on the AgiBot Lingxi X2 for Macau secondary schools, from no-code tools to ROS 2 programming and real-robot capstones.

Curriculum
Architected a 20-lesson core course and four grade-level modules as a spiral curriculum that progresses from no-code tools (LingXin, LinkCraft) to ROS 2 programming with the AimDK SDK, and finally to multi-platform integration.
Modules
Designed and built the four grade-level modules, each culminating in a ROS 2 capstone that runs on the real robot, as themed scenarios of rising difficulty: "X2 Says Hello" (preset arm motions), "X2 Expresses" (screen, LED, speech), "X2 in Motion" (bipedal locomotion), and "X2 as a Smart Service Robot" (sensing and full-chain integration).
Capstones
Engineered the four capstones on the real robot; the senior-secondary one links a head touch to a facial emotion, a spoken greeting, an arm wave, and a walk. Making it reliable meant resolving a single-threaded-executor deadlock, service-priority conflicts that dropped the speech and expression commands, and the control handoff between the robot's walking mode and the stance its arm gestures require.
ROS 2AimDK SDKLingXinLinkCraftAgiBot Lingxi X2
Real robot · capstone demo
Competition design · my work

Humanoid Robot Task Challenge, MYAIR 2027

Event and rulebook design · Macao Youth AI and Robotics Competition · Macao Science Center

Designer of the humanoid-robot task challenge and its rulebook: on a start signal, the robot autonomously shoots a ball one-handed into a QR-coded 3×3 grid.

Task design
After a start signal, the robot autonomously aims and executes a one-hand shot, planning throw direction, force and release to put the ball through the target cell of a QR-coded 3×3 grid.
Rulebook
Authored the full rulebook (equipment and safety, field layout, scoring rubric, timing and ranking) with an age-tiered design: the junior division reads a numbered grid; the senior division scans QR codes to build a position-to-code mapping and starts each shot by voice command.
Skills framed
Humanoid structure and motion principles, 3D-printing of the single-arm shooter, vision recognition and coordinate judgment, shot planning and release control, and task and time management.
Humanoid robotComputer VisionQR mappingVoice command3D printingRulebook design
MYAIR 2027 humanoid robot task challenge: robot shooting a ball into a QR-coded 3x3 grid, with event banner Competition concept · MYAIR 2027

Research Interests

01
Robot Learning & Reinforcement Learning
Learning locomotion and manipulation policies in simulation and transferring them to hardware.
02
Vision-Language-Action & World Action Models
Mapping language and perception to robot actions, and learning world models that predict how actions change the environment.
03
Manipulation & Grasping
Reliable grasping and motion planning under real-world perception noise.
04
Simulation, SLAM & Navigation
Building the sim-to-real and navigation stack that manipulation depends on.

Publications & Patent

My published work is from an earlier machine-learning and hydrology research period; it reflects a full research pipeline from data to peer review.

2019
Runoff Prediction Method Based on Adaptive Elman Neural Network
Chenming Li, Lei Zhu, Zhiyao He, Hongmin Gao, Yao Yang, Dan Yao, Xiaoyu Qu
Water Paper
2017
Annual Average Runoff Ensemble Forecast for Jinping I-Stage Hydropower Station Based on Elman Neural Network
Zhiyao He, Mingxiang Yang, Hao Wang, et al.
Water Resources and Power Paper
2020
Medium- and Long-term Runoff Forecasting Method Based on Mutual Information, Kernel PCA, and Elman Neural Network
Zhiyao He · Chinese National Invention Patent, granted 2020
Patent CN107463993B Google Patents

Experience

Mar 2026 – Present

AI Engineer

WellWin Technology, Hengqin Island & Macau, China
  • Built the perception and grasping pipelines for the Go2-W voice manipulation system.
  • Built offline and online voice agents for the AgiBot Lingxi X2 on Jetson.
  • Set up simulation and ran RL locomotion training for AKI and Unitree G1 / R1 humanoids.
  • Designed and built a full humanoid robotics curriculum for Macau secondary schools on the AgiBot Lingxi X2, covering three platforms (LingXin conversational agent, LinkCraft no-code choreography, AimDK ROS 2 SDK) and four grade-level extension modules of rising difficulty, from "X2 Says Hello" to "X2 as a Smart Service Robot".
  • Designed the MYAIR humanoid-robot task challenge and wrote its rulebook (Macao Youth AI and Robotics Competition, Macao Science Center): autonomous one-hand ball-shooting into a QR-coded 3×3 grid, with vision recognition, voice-command activation, shot planning, and a 3D-printed single-arm shooter.
Jul 2019 – Feb 2026

Independent Study & Research

Nanjing, China
Completed a full CS degree at University of the People (GPA 4.00); second journal paper published and invention patent granted during this period. Admitted to the M.Sc. Data Science at Trinity College Dublin (2025), declined for funding reasons.
Jul 2018 – Jun 2019

Software Engineer

Huawei Technologies · Nanjing R&D Institute, Nanjing, China
Intelligent security systems team (face, vehicle and other recognition products): system development, deployment and feature validation; cross-team debugging and system-level testing.
Jun 2016 – Jun 2017

Research Intern

China Institute of Water Resources and Hydropower Research, Beijing, China
Built a complete runoff-prediction pipeline: data collection and cleaning, feature engineering, model development, tuning, training and evaluation. Produced two journal papers and one patent.

Education

2015–2018

M.S., Signal & Information Processing

Hohai University, Nanjing, China
Outstanding Graduate Student (research-based degree). Thesis on machine learning for runoff forecasting.
2011–2014

B.Eng., Electrical Engineering & Automation

Anhui Agricultural University, Hefei, China
CGPA 3.65 / 4.00. Finished the four-year program in three years, the only student in the cohort to do so.
2021–2024

A.S., Computer Science

University of the People, USA
CGPA 4.00 / 4.00. President's List, 12 terms.

Technical Skills

Programming & systems
PythonC++LinuxDockerGit
Robotics
ROS / ROS 2MoveIt 2Nav2GazeboMuJoCoIsaac GymSLAMNavigationPerception
AI & robot learning
PyTorchMachine LearningComputer VisionReinforcement LearningLLMsRAGVLA
Robot platforms
Unitree G1 / B2 / Go2 / Go2-WAgiBot Lingxi X2

Honors & Awards

Contact

Open to PhD positions in robot learning.
Happy to talk about embodied intelligence and physical AI.
zhiyaohe@outlook.com GitHub CV