Problem
Human following is not only a detection problem. The robot must continuously associate the correct target, smooth noisy observations, recover through visual uncertainty, and convert perception output into navigation behavior.
Back to systemsDesigned for illumination, occlusion, pose, and camera-motion challenges from the start. Used KD-tree association to improve target consistency across frames. Applied Kalman filtering to reduce the effect of noisy observations. Connected perception output to motor control and path planning. Validated behavior through Gazebo simulation and mobile robot integration.
National Centre of Robotics & Automation
Vision-Based Human-Following Robot
Research Intern / Sep 2020 - Jun 2021
Built a vision-based human-following robotics workflow using visual tracking, KD-tree target association, Kalman filtering, ROS 1, motor control, Gazebo simulation, and Dijkstra path planning.
Visual trackingKD-tree associationKalman filteringROS 1Motor controlDijkstra planning
At a glance
Technical Overview and Key Facts
- Role
- Research Intern
- System type
- Vision-based mobile robotics prototype
- Core stack
- Python, OpenCV, ROS 1, Kalman filter, KD-tree, Gazebo
- Constraints
- Target association, noisy observations, visual uncertainty, robot motion integration
- Delivered
- Visual tracking, state estimation, ROS integration, motor control, path planning
My Role
Built the perception-to-motion workflow for a human-following robot. Developed visual tracking logic, used KD-tree association to maintain target identity, applied Kalman filtering to stabilize target state, and connected the output into ROS 1, motor control, simulation, and navigation planning.
Impact
Produced a working robotics prototype that connected computer vision tracking, state estimation, ROS integration, simulation, motor control, and mobile navigation into one perception-to-motion workflow.
System Architecture
- Visual tracking layer: camera input was used to extract point-based visual features from the target and surrounding scene.
- Target association layer: KD-tree-based association helped match visual observations over time and maintain target identity.
- State estimation layer: Kalman filtering smoothed noisy visual observations and produced a more stable target state.
- Robotics integration layer: ROS 1 connected perception output to motor-control and robot-system logic.
- Navigation & simulation layer: Dijkstra path planning and Gazebo simulation connected the estimated target state to mobile robot movement and validation.
Tools
PythonOpenCVROS 1GazeboKalman filterKD-treeMotor controlDijkstra planning
Robotics Constraints
- Tracking needed to remain useful through occlusion, illumination changes, pose variation, and camera motion.
- Target association had to stay stable enough for downstream robot-following behavior.
- State estimation had to smooth noisy visual observations before they influenced motion.
- ROS 1 integration needed clear interfaces between perception, motor control, and navigation.
- Simulation and path planning had to connect target tracking to robot movement in a testable workflow.
What I Delivered
- Built visual feature tracking for human-following behavior.
- Used KD-tree-based association to support target matching across frames.
- Applied Kalman filtering to stabilize noisy target-state estimates.
- Connected perception output to ROS 1 and motor-control logic.
- Used Gazebo simulation to test robot-following behavior.
- Integrated Dijkstra path planning to connect target state with mobile navigation.
Simulation Validation & Integration
Simulation connected tracking output to robot movement.
Validation focused on whether the tracking workflow remained coherent under visual uncertainty and whether the estimated target state could support simulated mobile robot navigation.
- Target tracking through occlusion, lighting variation, pose changes, and camera motion.
- KD-tree target association behavior.
- Kalman filter state stability.
- ROS 1 and motor-control integration.
- Gazebo simulation behavior.
- Dijkstra path-planning response to changing target position.
Implementation Highlights
The useful work sat between visual tracking and robot motion.
Next Step