Zhiyao He · robotics
Robotics · Embodied AI · Robot Learning

Zhiyao He

Robotics & AI Researcher · Embodied AI

I build robot systems that connect perception, language models, planning and control, from simulation to real hardware. I am looking to do a PhD in robot learning, focused on manipulation skills that hold up in the real world.

Download CV GitHub Google Scholar soon LinkedIn Email
About

I am an AI engineer working on embodied AI. I build robot pipelines that connect perception, language models, planning and control, taking systems from simulation in Gazebo, MuJoCo and Isaac Gym to real hardware.

My most recent system is voice-controlled navigation and manipulation on the Unitree Go2-W quadruped with a D1 arm. Alongside that I build offline, on-device voice agents for service robots. My goal is a PhD in robot learning, and specifically how robots can acquire manipulation skills that transfer reliably to the real world.

Based in the Hengqin & Macau area. M.S. from Hohai University; A.S. in Computer Science from University of the People (GPA 4.00). TOEFL 100/120, CEFR C1.

Research Interests

Where I want to go deeper

01
Robot Learning & Reinforcement Learning
Learning locomotion and manipulation policies in simulation and transferring them to hardware.
02
Vision-Language-Action Models
Grounding language and perception into robot actions for open-ended tasks.
03
Manipulation & Grasping
Reliable grasping and motion planning under real-world perception noise.
04
Simulation, SLAM & Navigation
Building the sim-to-real and navigation stack that manipulation depends on.
Selected Projects

Systems I have built end to end

Perception + Manipulation · my work

Voice-Controlled Navigation & Manipulation on Unitree Go2-W

Go2-W quadruped + D1 arm · team project with Macau Millennium College · ROS 2

A quadruped that hears a spoken request, navigates to a target, and grasps it. I owned the perception and manipulation halves of the system.

Perception
Fine-tuned YOLO11 by transfer learning to detect a specific bottle; a wrist-mounted depth camera and 2D-to-3D transform locate the target in the world frame.
Manipulation
A grasping pipeline on MoveIt 2: KDL inverse kinematics, OMPL / RRTConnect planning, FCL collision checking, planning a collision-free path and executing the grasp.
Integration
Teammates built SLAM navigation and the ASR / LLM / TTS pipeline; we defined the interfaces and integrated the full system, tested in MuJoCo and Gazebo and on the real robot.
ROS 2YOLO11RealSenseMoveIt 2KDLOMPL / RRTConnectFCLMuJoCoGazebo
GO2-W · D1
photo / demo GIF
Offline on-device agent · my work

Offline Voice RAG Assistant on AgiBot Lingxi X2

In-store retail assistant · industry project for CTM (Macau Telecom) · runs fully offline on Jetson

Customers ask about products, plans and pricing by voice; the robot answers with no network dependency, so it stays reliable during live events.

Pipeline
A bridge node streams the X2's microphone audio to a local FastAPI backend running local ASR (FunASR / Paraformer-zh), a local LLM (Qwen2.5-7B via Ollama), and pgvector RAG over a PostgreSQL product database.
Output
Answers are spoken through the X2's built-in offline TTS. An online variant integrates with third-party business systems to act on live backend data.
JetsonFastAPIFunASRQwen2.5-7BOllamaPostgreSQLpgvectorRAGoffline TTS
LINGXI X2
photo / demo GIF
RL locomotion + bring-up · my work

RL Locomotion & Robot Bring-up on the AKI Humanoid

Company education product · team reproduction of Unitree's open-source Qmini

Bringing a reproduced humanoid to life and teaching it to walk in simulation, then setting up the training stack the team builds on.

Bring-up
Brought up and debugged the control system and power system on the reproduced robot.
Learning
Ran reinforcement-learning locomotion training in Isaac Gym; set up Gazebo, MuJoCo and Isaac Gym environments for AKI and the Unitree G1 / R1 humanoids.
Isaac GymReinforcement LearningMuJoCoGazeboUnitree G1 / R1control systems
AKI · G1 / R1
photo / demo GIF
Publications & Patent

Earlier research record

My published work is from an earlier machine-learning and hydrology research period; it reflects a full research pipeline from data to peer review.

2019
Runoff Prediction Method Based on Adaptive Elman Neural Network
Chenming Li, Lei Zhu, Zhiyao He, Hongmin Gao, Yao Yang, Dan Yao, Xiaoyu Qu
Water Paper
2017
Annual Average Runoff Ensemble Forecast for Jinping I-Stage Hydropower Station Based on Elman Neural Network
Zhiyao He, Mingxiang Yang, Hao Wang, et al.
Water Resources and Power
2020
Medium- and Long-term Runoff Forecasting Method Based on Mutual Information, Kernel PCA, and Elman Neural Network
Zhiyao He · Chinese National Invention Patent, granted 2020
Patent CN107463993B Google Patents
Experience

Where I have worked

Mar 2026 — Present

AI Engineer

WellWin Technology · Hengqin & Macau
  • Built the perception and grasping pipelines for the Go2-W voice manipulation system.
  • Built offline and online voice agents for the AgiBot Lingxi X2 on Jetson.
  • Set up simulation and ran RL locomotion training for AKI and Unitree G1 / R1 humanoids.
Jul 2019 — Feb 2026

Independent Study & Research

Nanjing, China
Completed a full CS degree at University of the People (GPA 4.00); second journal paper published and invention patent granted during this period. Admitted to the M.Sc. Data Science at Trinity College Dublin (2025), declined for funding reasons.
Jul 2018 — Jun 2019

Software Engineer

Huawei Technologies · Nanjing R&D Institute
Intelligent security systems team (face, vehicle and other recognition products): system development, deployment and feature validation; cross-team debugging and system-level testing.
Jun 2016 — Jun 2017

Research Intern

China Institute of Water Resources and Hydropower Research · Beijing
Built a complete runoff-prediction pipeline: data collection and cleaning, feature engineering, model development, tuning, training and evaluation. Produced two journal papers and one patent.
Education

Degrees

2015 — 2018

M.S., Signal & Information Processing

Hohai University, Nanjing
Outstanding Graduate Student (research-based degree). Thesis on machine learning for runoff forecasting.
2021 — 2024

A.S., Computer Science

University of the People, USA
CGPA 4.00 / 4.00. President's List, 12 terms.
2011 — 2014

B.Eng., Electrical Engineering & Automation

Anhui Agricultural University
CGPA 3.65 / 4.00. Finished the four-year program in three years, the only student in the cohort to do so.
Technical Skills
Programming & systems
PythonC++LinuxDockerGit
Robotics
ROS / ROS 2MoveIt 2Nav2GazeboMuJoCoIsaac GymSLAMNavigationPerception
AI & robot learning
PyTorchMachine LearningComputer VisionReinforcement LearningLLMsRAGVLA
Robot platforms
Unitree G1Unitree B2 / Go2 / Go2-WAgiBot Lingxi X2
Honors & Awards
Contact
Open to PhD positions in robot learning.
Happy to talk about manipulation, sim-to-real, and on-device robot agents.
zhiyaohe@outlook.com GitHub CV