RobotX 2026 > Software
RobotX - Drone Software
Overview
We implement off-the-shelf solutions like Mission Planner, and ArduPilot controls the drone's autonomous maneuvers. We have a Jetson Orin Nano containing sensor data and interfacing with the Robo command module as well as the USV to inform situational awareness. We utilize the RFD 9000 mini for RF remote-to-telemetry links between the UAV and the USV.
YOLO26 Training Pipeline
To teach YOLO26 how to spot buoys, we run it through a step-by-step training process. First, we split all our buoy pictures into two piles: 80% for teaching the model, and 20% for testing it later, before doing anything else to the images. This matters because if you mix up copies of the same picture between the two piles, the test results look better than they really are. Next, we use our simpler HSV (Hue, Saturation, Value) method (the shape-and-color one) to automatically draw labels on all the training pictures, marking where each buoy is and what color it is. Then we feed those labeled pictures into YOLO26 so it can learn the patterns. After training, we run several checks to make sure it actually learned buoys and didn't just memorize the training pictures, plus a stress test where we blur and add noise to images to see how well it still performs. Our best training run scored very high, over 99% accuracy on buoys it hadn't seen during training.
Buoy Detection
Our drone finds buoys using two different methods that can be swapped without changing any code. The first method looks for round shapes first, then checks the color (red, green, or blue) to confirm it's really a buoy. It doesn’t need a neutral network, and it is fast. The second method uses a trained AI model (YOLO11n) that spots the buoy and its color in one step. Both methods track each buoy over time so it isn't counted twice, then send its exact location using GPS math and radio signals back to the ground team. In simulation testing, the drone found all six buoys with an average position error of just 0.16 meters.
GPS Projection
First flight test with buoy mapping (9/13/26)
Able to distinguish flashing blue buoy from solid blue buoy, able to map them out accurately, +-0.3 meter spread across GPS waypoint data and taking the average to get the actual location of the buoys. 26 fps.
Once the drone spots a buoy in its camera, it needs to figure out the buoy's real-world location, not just where it appears in the picture. We do this with some geometry: knowing how high the drone is flying and the exact specs of its camera, we can calculate how far away the buoy is on the ground, then combine that with the drone's own GPS position to get the buoy's actual latitude and longitude. We also account for which direction the drone is facing (its heading), since a buoy to the "left" of the camera means something different depending on which way the drone is pointed. One thing we haven't added yet is correcting for the drone tilting during flight, so that's a planned improvement.
Mission 1: Safe Passage Path Planning
Ekko flight test with autonomous path planning mapping out buoy field for Mission 1: Safe Passage
Simulation
Before flying the real drone, we test everything in a computer simulation using Gazebo, a program that creates a fake 3D world with physics, water, and a virtual copy of our drone. We built three practice courses: a straight line of buoys, a scattered field of buoys, and an L-shaped course with a turn. Each course also has decoy objects, like colored panels and crates, that aren't buoys, to make sure our detector doesn't get confused by them. A special "light buoy" even blinks between red, green, and blue every few seconds to test that feature too. Running the straight-line course, our drone found all six buoys with very little error, about 0.16 meters off on average, which is only about the width of a basketball. Simulation lets us catch problems and improve accuracy before ever risking the real drone outdoors.