“AI drone” gets used for everything from a camera that follows you on a bike to a research platform that maps a forest on its own. Underneath the hype are a few concrete capabilities, and you can build most of them into a DIY drone.
What AI actually does on a drone
On a drone, “AI” usually means computer vision: software that looks at camera frames and recognizes something useful in them. Combined with an autopilot, that recognition turns into action.
| Capability | What it does | Typical sensors |
|---|---|---|
| Object detection | Finds people, vehicles, animals or objects in the image | Camera |
| Tracking / follow-me | Keeps a chosen subject centered while the drone follows | Camera |
| Obstacle avoidance | Stops or routes around obstacles | Depth camera, lidar, rangefinders |
| Visual positioning | Holds position without GPS, indoors | Downward camera, optical flow |
| Precision landing | Lands on a marker or pad | Camera, marker |
| Inspection | Flags cracks, rust or hot spots in images | Camera, thermal camera |
The two-computer architecture
Most DIY AI drones split the job between two computers:
- The flight controller (running ArduPilot or PX4) keeps the drone stable and executes commands. It runs a hard real-time loop and must never be overloaded.
- The companion computer (a Raspberry Pi, Jetson or similar) runs the AI: it reads the camera, runs a neural network and sends high-level commands like “move left 1 m/s” to the flight controller.
They talk over MAVLink, a lightweight messaging protocol. If the companion computer crashes, the flight controller keeps flying and can trigger a failsafe. See companion computers and connecting a Pi with MAVLink.
Edge AI: why processing happens onboard
Sending video to the cloud and waiting for an answer is too slow for a moving drone. AI models therefore run on the drone itself, on small, efficient “edge” hardware. Modern models such as compact YOLO variants are designed to run in real time on this hardware, especially when exported to formats optimized for it.
What you can realistically build
- Follow-me filming: detect and track a person, and keep them centered with guided-mode commands. See object tracking.
- Obstacle stop: rangefinders or a depth camera that make the drone stop before hitting something. See obstacle avoidance.
- Autonomous survey: fly a grid mission and process photos. See mapping drones.
- Precision landing: land on a printed marker.
Test in simulation first
Autonomous code can move a drone in ways you don’t expect. Test it in software first with SITL and Gazebo, where a bug costs nothing. See simulation with SITL.
How an AI drone makes a decision, step by step
Take follow-me filming as an example. Many times per second, the drone runs a loop like this:
- Capture: the camera grabs a frame.
- Detect: a neural network finds people in the frame and draws boxes around them.
- Track: a tracker decides which box belongs to the subject you chose.
- Decide: a controller compares the box’s position and size to where it should be.
- Command: the companion computer sends a velocity command over MAVLink.
- Fly: the autopilot turns that command into motor outputs, keeping the drone stable.
Each step adds a little delay. Keeping the whole loop fast, ideally 15–30 times per second or more, is what makes tracking smooth.
Models and frame rates
Smaller models run faster but miss more. Larger models are more accurate but slower. On edge hardware, builders often:
- Use a compact model at a modest input resolution.
- Convert it to an optimized format for their hardware.
- Run detection every frame and a lightweight tracker in between.
What AI can’t do (yet) on a DIY drone
- Reliably see thin wires and branches with one camera.
- Replace a pilot’s judgment about people, property and airspace.
- Guarantee safe flight in crowded or complex environments.
That’s why every AI feature should sit on top of an autopilot with geofences, failsafes and a pilot ready to take over.
A learning path into drone AI
| Step | Project | Guide |
|---|---|---|
| 1 | Build a GPS drone | ArduPilot quad |
| 2 | Fly autonomous missions | Waypoint missions |
| 3 | Connect a companion computer | MAVLink on a Pi |
| 4 | Simulate everything | SITL and Gazebo |
| 5 | Add detection and tracking | Object tracking |
| 6 | Add obstacle sensing | Obstacle avoidance |
Rules still apply
AI doesn’t change the law. In the U.S., you must keep the drone in visual line of sight, follow altitude and airspace rules, and respect privacy. Follow-me and autonomous flights still need a pilot ready to take over.
Frequently Asked Questions
What makes a drone an AI drone?
Usually onboard computer vision: software that recognizes objects, people or obstacles in camera images and uses that information to guide the drone.
Can I add AI to my existing drone?
If it runs ArduPilot or PX4 and has spare capacity for a companion computer and camera, yes. Most closed commercial drones don’t allow it.
What computer do AI drones use?
A companion computer such as a Raspberry Pi or NVIDIA Jetson runs the AI, while a separate flight controller keeps the drone stable.
Do AI drones fly themselves?
They can carry out tasks on their own, such as following a subject or flying a mission, but a pilot must still supervise and be ready to take control.