Real hardware evidence
Typed objective in.
PX4 drone flight out.
Over the last four months I built a natural-language control stack for a
real drone. The milestone is simple: one typed mission objective became an
outdoor PX4 flight with arm, takeoff, lateral movement, hold,
return-to-launch, landing, disarm, and final state verification.
The takeaway: this software replaces the remote controller with an AI conversation to fly drones.
Real PX4 aircraft
MCP tool calls
Human supervised
Final state verified
The flight
The important part is not how impressive the flight was but what controlled it.
I typed the objective into Codex (or Claude), which called structured MCP
tools exposed by droneserver. From there,
droneserver talked to the aircraft through MAVSDK/MAVLink
over a SiK telemetry radio, PX4 executed the flight actions, and the same
interface checked aircraft state during preflight, flight, and
post-mission return to the launch position.
The video keeps the evidence visible, with tool calls and telemetry on the
left and the physical drone on the right. The product direction is a
higher-level mission interface that translates operator intent into
observable flight mechanics, with the remote controller becoming a safety
fallback rather than the primary interface.
01
User Intent
Typed objective: take off, move south, hold, return, land.
02
AI-native control
Codex selects and sequences MCP tools exposed by droneserver.
03
PX4 flight
PX4 executes arm, takeoff, movement, hold, and RTL.
04
Verification and Safety
The system confirms landed, disarmed, on ground, and healthy.
What this is
Next generation control surface above the aircraft
The drone has not changed; the new thing is the way the user controls it. Instead of beginning with remote controller stick movement or low-level flight commands, the operator can state the mission objective in plain language while the software turns that objective into structured flight actions, constantly checks the drone status, and exposes what it is doing to the user. My custom software is made available to an AI model (testing was done with OpenAI's GPT 5.4) via a Model Context Protocol (MCP) server.
The product idea is not "replace pilots with AI." This is a next-generation
way to fly drones, so the operator can spend less time and energy flying and more time deciding what outcome matters. Also keep in mind, flying a drone is hard but describing what you want the drone to accomplish is something anyone can do.
Operator
mission objective
MCP tools
structured actions
droneserver
state and telemetry
MAVSDK / MAVLink
PX4 interface
Why it matters
This is software that solves real problems.
Ease of use expands adoption
Skilled drone pilots will always matter, especially when conditions get messy. But many useful drone tasks do not start with a desire to manually manage sticks, modes, and waypoints. They often start with a person who needs to inspect a structure, search a hard-to-reach area, record content, help fight a wildfire, engage in warfare, film nature, and so much more. My prediction is that not every one of these use cases will continue to require a skilled pilot if my software can advance far enough in capability. This software could help increase drone hardware sales, by expanding who the end user can be.
Fills gaps where drone pilots are scarce
A natural-language control layer could also give experienced drone
operators an easier tool for routine flight mechanics. That matters most in places where trained pilots are scarce, overloaded, or hard to position quickly. Think about wildland firefighting, search and rescue, infrastructure inspection, and other field operations where aerial
evidence is valuable but it's hard to bring in human pilots.
Easier to scale and improve existing pilot efficiency
If you are flying a drone via the remote controller, that means your hands are occupied on the sticks and you can physically only fly one drone at a time. If instead you use my software, you can fly a second drone by opening a second tab. Imagine if every drone operator could double their efficiency by controlling 2 drones at once instead of 1? That seems like software people would pay for.
Proven now
What the demo supports
- Real PX4 hardware can be reached through an LLM, MCP, MAVSDK, MAVLink, and a SiK telemetry radio.
- The aircraft can execute software-led flight maneuvers that stack together into basic missions.
- The AI-native control path works on real hardware, not just simulation environments.
Not proven yet
What I am not claiming
- This is not unattended autonomy yet.
- This is not obstacle avoidance or cluttered-environment navigation yet.
- This is not a mature hardware perception loop yet.
- This is not multi-drone control, BVLOS readiness, or a production deployment model yet.
Where this goes
The next step is external feedback
The milestone is strong enough to take outside the lab. I am using it to
start conversations with people building drone hardware, autonomy stacks,
simulation environments, field robotics systems, and dual-use mission
software.
The questions I care about now are product questions: where does this
interface reduce real operator burden, which hardware stack should it be
validated on next, and what evidence would make this useful to someone
besides me? I think this software has real value, but where do I take it next?
For the full engineering buildout, see
Phase Three: Real Hardware
and
Phase Four: Real Flight.
If you want to discuss the work or compare notes on where this should go
next, reach out on
LinkedIn.
Thank you so much for reading. This is 4 months of hard work I am proud of!
-Jake