Following CartA cart that follows you and avoids obstacles.

Most autonomous cart projects stop at simulation. We built the real thing — a cart that sees you with a depth camera, knows where you are with UWB, and follows you while dodging obstacles. The system covers the full robotics stack: perception (YOLO + depth), localization (vision + UWB fusion), planning (following algorithm + obstacle avoidance), and control (rear-wheel drive with front-wheel steering). The hardest part wasn't any single component; it was making vision, localization, and motor control work together reliably in the real world.

The system follows a layered robotics pipeline: Sensors (OAK-D Camera + UWB Module) feed into a Perception System (YOLO person detection, target tracking, depth processing, obstacle detection), which passes through a Sensor Fusion layer combining vision position with UWB distance/angle data. The fused target estimation feeds into a Motion Planner (follow algorithm, obstacle avoidance, speed control), which outputs ROS2 cmd_vel commands. These are split into drive speed and steering angle and sent through the ESP32 to the motor driver: the rear motor drives, the front motor steers.

• Hardware + Software Integration — most student projects stop at simulation; this project delivered a complete working prototype with AI + robotics + hardware
• Multi-Sensor Fusion — vision + UWB asymmetric fusion architecture provides robust tracking even during occlusion
• Full Robotics Stack — covers perception, planning, control, and hardware integration end-to-end

• OAK-D Camera — RGB + Stereo Depth + Myriad X VPU (onboard neural inference)
• YOLOv8 person detection, real-time inference on Raspberry Pi
• DWM3001CDK UWB module — distance + angle estimation via BLE
• ROS2 on Raspberry Pi 4/5 with SSD, Python
• Visual servoing-based following control algorithm
• Depth-based obstacle avoidance (1m safety threshold)
• Rear-wheel drive + front-wheel steering — ESP32 PWM → Cytron MDD3A motor driver
• 24V LiPo battery, ~25 min prototype runtime
• DepthAI SDK, OpenCV, cv_bridge, NumPy, pyserial
• Vision-primary / UWB-backup asymmetric sensor fusion

• Depth Noise in Outdoor Environments — Stereo depth accuracy degraded significantly under direct sunlight and with reflective surfaces, requiring adaptive threshold tuning and confidence filtering.
• UWB Connection Stability — BLE connection between the UWB module and iPhone was intermittent, causing gaps in backup localization data. Required reconnection logic and graceful fallback handling.
• Battery Runtime Limitations — 24V LiPo provided only ~25 minutes of operation with all systems active. Power budgeting and duty cycling of non-critical sensors were explored but not fully implemented.
• Sensor Synchronization — Vision (30fps) and UWB (variable BLE rate) operate on different timing cycles. Timestamp alignment and data freshness validation were critical for reliable fusion.

• Single-sensor dependency is dangerous — multi-sensor fusion architecture is essential for robustness in real-world conditions
• Simulation ≠ reality — real-world noise, lighting variation, and mechanical play introduce problems that never appear in simulation
• Hardware integration is harder than software — mounting, cabling, vibration, and thermal management are underestimated engineering challenges
• Full-stack robotics (perception → planning → control → hardware) requires deep integration testing that can't be unit-tested in isolation

• SLAM integration (slam_toolbox, rtabmap) for map-based navigation
• ROS2 Nav2 with DWA planner for path planning in complex environments
• Terrain adaptation — slope detection and rough terrain control
• Multi-user tracking with specific user identification and re-identification

• System Architecture
• Sensor Fusion
• ROS2 Integration
• Perception Pipeline
• Motor Control
• Hardware Build