Zhiyao He

Zhiyao He

Robot Learning · Embodied AI

Building robots that perceive, reason, and act in the real world, from simulation to real hardware.

Seeking a PhD in robot learning · 2027 start
Download CV GitHub LinkedIn Email Blog
Zhiyao He, portrait

About

I am a robotics and AI engineer working on embodied AI. I build robot pipelines that connect perception, language models, planning and control, taking systems from simulation in Gazebo, MuJoCo and Isaac Gym to real hardware.

My most recent system is voice-controlled navigation and manipulation on the Unitree Go2-W quadruped with a D1 arm. Alongside that I build offline, on-device voice agents for service robots. My goal is to do a PhD in robot learning, especially on how robots can learn to perceive, reason, and act reliably in real-world environments.

Based in the Hengqin Island & Macau area. M.S. from Hohai University. TOEFL 100/120, CEFR C1.

Research Interests

01
Robot Learning & Reinforcement Learning
Learning locomotion and manipulation policies in simulation and transferring them to hardware.
02
Vision-Language-Action & World Action Models
Mapping language and perception to robot actions, and learning world models that predict how actions change the environment.
03
Manipulation & Grasping
Reliable grasping and motion planning under real-world perception noise.
04
Simulation, SLAM & Navigation
Building the sim-to-real and navigation stack that manipulation depends on.

Selected Projects

Perception + Manipulation · my work

Voice-Controlled Navigation & Manipulation on Unitree Go2-W

Go2-W quadruped + D1 arm · team project with Macau Millennium College · ROS 2

A quadruped that hears a spoken request, navigates to a target, and grasps it. I owned the perception and manipulation halves of the system.

Perception
Fine-tuned YOLO11 by transfer learning to detect a specific bottle; a wrist-mounted depth camera and 2D-to-3D transform locate the target in the world frame.
Manipulation
A grasping pipeline on MoveIt 2: KDL inverse kinematics, OMPL / RRTConnect planning, FCL collision checking, planning a collision-free path and executing the grasp.
Integration
Teammates built SLAM navigation and the ASR / LLM / OpenClaw / TTS pipeline; we defined the interfaces and integrated the full system, tested in MuJoCo and Gazebo and on the real robot.
ROS 2YOLO11RealSenseMoveIt 2KDLOMPL / RRTConnectFCLMuJoCoGazebo
Real robot · nav + grasp + deliver
Voice agent, offline + online · my work

Retail Voice Agent on AgiBot Lingxi X2

In-store retail assistant · industry project for CTM (Macau Telecom) · offline and online versions

Customers ask about products, plans and pricing by voice. Built in two versions: an offline one for sites with unreliable network, and an online one with a Cantonese voice for Macau customers.

Offline version
Runs on a Jetson. A bridge node streams the X2's microphone audio to a local FastAPI backend running FunASR / Paraformer-zh ASR, Qwen2.5-7B via Ollama, pgvector RAG over a PostgreSQL product database, and Piper TTS.
Online version
Built with the Lingxi AgentSDK, using DeepSeek as the LLM and MiniMax TTS with a Cantonese voice. It draws on three knowledge sources: the Lingxi platform knowledge base, the local RAG system, and third-party business systems such as CTM's.
JetsonFastAPIFunASRQwen2.5-7BOllamaPostgreSQLpgvectorRAGPiper TTSLingxi AgentSDKDeepSeekMiniMax TTS
Real robot · voice agent
RL locomotion + bring-up · my work

RL Locomotion & Robot Bring-up on the AKI Humanoid

Company education product · team reproduction of Unitree's open-source Qmini

Bringing a reproduced humanoid to life and teaching it to walk in simulation.

Bring-up
Brought up and debugged the control system and power system on the reproduced robot.
Learning
Ran reinforcement-learning locomotion training in Isaac Gym; set up Gazebo, MuJoCo and Isaac Gym environments for AKI and the Unitree G1 / R1 humanoids.
Isaac GymReinforcement LearningMuJoCoGazeboUnitree G1 / R1control systems
Simulation

Publications & Patent

My published work is from an earlier machine-learning and hydrology research period; it reflects a full research pipeline from data to peer review.

2017
Annual Average Runoff Ensemble Forecast for Jinping I-Stage Hydropower Station Based on Elman Neural Network
Zhiyao He, Mingxiang Yang, Hao Wang, et al.
Water Resources and Power Paper
2019
Runoff Prediction Method Based on Adaptive Elman Neural Network
Chenming Li, Lei Zhu, Zhiyao He, Hongmin Gao, Yao Yang, Dan Yao, Xiaoyu Qu
Water Paper
2020
Medium- and Long-term Runoff Forecasting Method Based on Mutual Information, Kernel PCA, and Elman Neural Network
Zhiyao He · Chinese National Invention Patent, granted 2020
Patent CN107463993B Google Patents

Experience

Mar 2026 – Present

AI Engineer

WellWin Technology, Hengqin Island & Macau, China
  • Built the perception and grasping pipelines for voice-controlled navigation and manipulation on the Unitree Go2-W quadruped with a D1 arm.
  • Built offline and online voice agents for the AgiBot Lingxi X2, used in CTM retail stores and at live events with unstable network.
  • AKI humanoid (team reproduction of Unitree's open-source Qmini): brought up and debugged the control and power systems, and ran RL locomotion training in Isaac Gym.
  • Set up simulation environments and ran locomotion training for the Unitree G1 / R1 humanoids in Gazebo, MuJoCo and Isaac Gym.
Jul 2019 – Feb 2026

Independent Study & Research

Nanjing, China
Completed an A.S. in Computer Science at University of the People (GPA 4.00 / 4.00) to strengthen my foundation in computer science and AI. Second journal paper published (Water, 2019); invention patent granted (2020). Admitted to the M.Sc. in Data Science, Trinity College Dublin (2025 intake).
Jul 2018 – Jun 2019

Software Engineer

Huawei Technologies · Nanjing R&D Institute, Nanjing, China
Intelligent security systems team (face and vehicle recognition products): system development, deployment and feature validation; cross-team debugging and system-level testing.
Jun 2016 – Jun 2017

Research Intern

China Institute of Water Resources and Hydropower Research, Beijing, China
Built a complete runoff-prediction pipeline: data collection and cleaning, feature engineering, model development, tuning, training and evaluation. Produced two journal papers and one patent. Certificate

Teaching & Outreach

Curriculum + real-robot ROS 2 · my work

Humanoid Robotics Curriculum on AgiBot Lingxi X2

Education product for Macau junior and senior secondary schools

A full humanoid robotics curriculum on the AgiBot Lingxi X2 for Macau secondary schools, from no-code tools to ROS 2 programming and real-robot capstones.

Curriculum
Architected a 20-lesson core course and four grade-level modules as a spiral curriculum that progresses from no-code tools (LingXin, LinkCraft) to ROS 2 programming with the AimDK SDK, and finally to multi-platform integration.
Modules
Designed and built the four grade-level modules, each culminating in a ROS 2 capstone that runs on the real robot, as themed scenarios of rising difficulty: "X2 Says Hello" (preset arm motions), "X2 Expresses" (screen, LED, speech), "X2 in Motion" (bipedal locomotion), and "X2 as a Smart Service Robot" (sensing and full-chain integration).
Capstones
Engineered the four capstones on the real robot; the senior-secondary one links a head touch to a facial emotion, a spoken greeting, an arm wave, and a walk. Making it reliable meant resolving a single-threaded-executor deadlock, service-priority conflicts that dropped the speech and expression commands, and the control handoff between the robot's walking mode and the stance its arm gestures require.
ROS 2AimDK SDKLingXinLinkCraftAgiBot Lingxi X2
Real robot · capstone demo
Competition design · my work

Humanoid Robot Task Challenge, MYAIR 2027

Event and rulebook design · Macao Youth AI and Robotics Competition · Macao Science Center

Designer of the humanoid-robot task challenge and its rulebook: on a start signal, the robot autonomously shoots a ball one-handed into a QR-coded 3×3 grid.

Task design
After a start signal, the robot autonomously aims and executes a one-hand shot, planning throw direction, force and release to put the ball through the target cell of a QR-coded 3×3 grid.
Rulebook
Authored the full rulebook (equipment and safety, field layout, scoring rubric, timing and ranking) with an age-tiered design: the junior division reads a numbered grid; the senior division scans QR codes to build a position-to-code mapping and starts each shot by voice command.
Skills framed
Humanoid structure and motion principles, 3D-printing of the single-arm shooter, vision recognition and coordinate judgment, shot planning and release control, and task and time management.
Humanoid robotComputer VisionQR mappingVoice command3D printingRulebook design
MYAIR 2027 humanoid robot task challenge: robot shooting a ball into a QR-coded 3x3 grid, with event banner Competition concept · MYAIR 2027
Outreach

UoPeople Ambassador, University of the People

Promoted the university’s tuition-free model and answered questions from prospective students. Received the Ambassador Award (AY2022, Term 5). Certificate

Education

2015–2018

M.S., Signal & Information Processing

Hohai University, Nanjing, China
Thesis on machine learning for runoff forecasting, leading to two peer-reviewed journal papers and one Chinese invention patent. Outstanding Graduate Student.
2011–2014

B.Eng., Electrical Engineering & Automation

Anhui Agricultural University, Hefei, China
CGPA 3.65 / 4.00. Finished the four-year program in three years, the only student in the cohort to do so.
2021–2024

A.S., Computer Science

University of the People, USA
CGPA 4.00 / 4.00. President's List, 12 terms.

Technical Skills

Programming & systems
PythonC++LinuxDockerGit
Robotics
ROS / ROS 2MoveIt 2Nav2GazeboMuJoCoIsaac GymSLAMNavigationPerception
AI & robot learning
PyTorchMachine LearningComputer VisionReinforcement LearningLLMsRAGVLA
Robot platforms
Unitree G1 / R1Unitree Go2 / Go2-WAgiBot Lingxi X2
Certificate
Deep Learning Specialization (DeepLearning.AI)

Honors & Awards

Contact

Open to PhD positions in robot learning.
Happy to talk about embodied intelligence and physical AI.
zhiyaohe@outlook.com GitHub CV