Add Object Tracking to Your Drone With a Raspberry Pi

This guide contains Amazon affiliate links. If you buy through them, DroneCoreGuide earns a small commission at no extra cost to you.

Object tracking lets your drone find a subject in its camera feed and follow it, keeping it centered in the frame. It’s the core of follow-me filming for bikes, boats, runners and vehicles. Here’s how to build it on a Raspberry Pi and ArduPilot.

Prerequisites: a working ArduPilot GPS quad and a Raspberry Pi connected over MAVLink.

How the system works

1. A camera feeds frames to the Raspberry Pi.
2. A detection model finds objects of the chosen class (for example “person”) and returns bounding boxes.
3. A tracker keeps the same subject’s ID from frame to frame.
4. A controller compares the box’s position to the image center and computes velocity commands.
5. The Pi sends those velocity commands to ArduPilot in GUIDED mode.

Hardware

Software

• A compact detection model (a small YOLO model is a common choice), exported to a fast format for your hardware.
• OpenCV to capture and process frames.
• A simple tracker that matches detections between frames.
• pymavlink or MAVSDK to send velocity commands.

The control logic

Measure how far the subject’s box is from the center of the image:

• Horizontal error → yaw toward the subject, or move sideways.
• Box size → move forward if the box is getting smaller, back if it’s getting larger.
• Vertical error → adjust camera tilt (with a gimbal), or altitude, within limits.

Use a gentle proportional controller with hard limits on speed, and add smoothing so the drone doesn’t jerk when detections flicker.

Build in safety

• Lost subject: if the subject disappears for more than a second or two, stop and hover in place.
• Speed limit: cap commanded speed well below the drone’s maximum.
• Geofence and altitude limit in ArduPilot.
• Minimum distance: don’t let the controller approach closer than a set distance.
• Override: a radio switch that instantly leaves GUIDED mode.
• Obstacles: tracking code does not see obstacles. Fly only in open areas, or add obstacle avoidance.

Test step by step

1. Run detection on the bench and confirm boxes are accurate.
2. Run the full control loop against SITL in simulation. See SITL and Gazebo.
3. On the real drone, test with a cooperative subject in a wide open field, starting with yaw-only tracking.
4. Add forward and backward movement once yaw tracking is smooth.

A simple tracking controller

A proportional controller with limits is often enough for a first version:

yaw_rate   = clamp(k_yaw * x_error, -max_yaw, max_yaw)
forward_v  = clamp(k_fwd * (target_size - box_size), -max_v, max_v)

Here x_error is the horizontal distance between the box center and the image center, normalized to the image width, and box_size is the box height relative to the frame. Start with small gains and low limits, then increase slowly. Add a dead zone around zero error so the drone doesn’t twitch for tiny changes.

Smoothing detections

Detections jitter from frame to frame. Smooth them:

• Average box positions over the last few frames.
• Ignore detections below a confidence threshold.
• Keep tracking the same ID instead of jumping to whichever box is largest.

Measuring end-to-end delay

Point the camera at a phone showing a running stopwatch, record what the Pi sees with timestamps, and compare. Knowing your loop delay helps you choose gains: a longer delay needs gentler control.

Ideas for next steps

• Add a gimbal so the camera can tilt instead of the drone climbing and descending.
• Use GPS position from a phone or tracker on the subject as a backup to vision.
• Combine tracking with obstacle sensing for safer flights.
• Log every flight’s detections to improve your model over time.

Privacy and rules

Only track people who’ve agreed to be filmed. Keep the drone in visual line of sight, follow local rules, and don’t fly over crowds.