Overview
<10ms latency real-time perception for TartanDrive autonomous vehicle. All-weather navigation through thermal sensor integration.
Problem
Cameras and LiDAR degrade in rain, fog, snow, darkness. Autonomous vehicles need reliable perception in adverse conditions with real-time performance.
Solution
Integrated thermal cameras with ROS driver architecture and Teensy microcontroller synchronization. Custom SOLIDWORKS mounting hardware. Multimodal pipeline combining thermal + LiDAR + camera data. Validated across rain, fog, nighttime conditions.
Impact
- <10ms latency: Real-time perception pipeline
- All-weather navigation: Rain, fog, night conditions
- Multimodal fusion: Thermal + LiDAR + camera integration
- Research dataset: Synchronized multimodal data collection
Impact & Learnings
Research Impact:
- Enabled all-weather autonomous navigation research for TartanDrive platform
- Created valuable multimodal dataset for perception algorithm development
- Demonstrated thermal sensing viability for real-time autonomous systems
- Contributed to CMU AirLab’s outdoor robotics capabilities
Technical Learnings:
- Real-time systems require careful attention to latency at every layer
- Time synchronization is critical for multimodal sensor fusion
- Hardware design iterations benefit from close collaboration with fabrication teams
- Field testing reveals edge cases impossible to predict in lab environments
Engineering Skills Developed:
- ROS 2 driver development and real-time optimization
- Microcontroller programming (Teensy) for time-critical applications
- CAD design (SOLIDWORKS) and hardware prototyping
- Sensor calibration and characterization
- Multimodal data pipeline architecture
Systems Engineering:
- Balanced performance, cost, and integration complexity
- Made build-vs-buy decisions for mechanical components
- Documented system design for future team members
- Applied rigorous testing methodology for validation
