Building a DJI Tello Desktop Controller with Rust and Tauri

I learned Rust by rebuilding something that already had real constraints: my DJI Tello desktop controller. The result is not a UI mockup. It connects to the drone, sends flight commands, displays telemetry and live video, captures photos, records MP4, writes flight logs, and fails conservatively when input or network state becomes stale.

The interface uses Vue 3 inside Tauri 2. Rust owns everything that can affect the aircraft: UDP sockets, acknowledgements, control state, timers, decoder processes, files, and shutdown.

Why this was a useful Rust project

A drone controller has several independent activities:

Vue interface
  ↕ Tauri commands and events
Rust controller
  ├─ UDP command socket
  ├─ telemetry receiver
  ├─ video receiver
  ├─ FFmpeg decoder
  ├─ 20 Hz RC loop and watchdog
  ├─ recording and CSV logs
  └─ shutdown and resource cleanup

That made ownership and concurrency concrete. Background workers have explicit responsibilities; shared flight state is protected; lifetime flags are atomic; expensive media work stays outside the control loop.

Tauri let me keep familiar Vue UI development without pretending the browser could directly own UDP sockets or spawn FFmpeg. Unlike Electron, Tauri uses the operating system webview rather than shipping a complete Chromium runtime.

Talking directly to the Tello

The Rust backend does not use djitellopy. It talks to the Tello SDK endpoints with std::net::UdpSocket:

One subtle bug appeared before a packet reached the drone. Linux could route 192.168.10.1 through the normal home network when the computer was not joined to TELLO-*. A socket can be valid while using the wrong interface.

The controller connects the command socket first, inspects the local address selected by the OS, and refuses to continue unless it belongs to 192.168.10.x. That produces “join the Tello Wi-Fi” instead of three unrelated timeouts.

Telemetry and video use fixed local ports, so only one controller can own them. The application surfaces address already in use and releases resources during disconnect rather than hiding bind failures behind a generic connection error.

A control loop that does not trust the webview

Vue reports the currently held controls. Rust combines the axes and sends RC values at 20 Hz. Slow mode begins at 30 and accelerates to 100 after two seconds; fast mode selects 100 immediately.

The important detail is what happens when the UI stops reporting. A desktop window can lose focus or miss a key-up event. If the controller relied only on browser events, the last movement might remain active.

Rust records the most recent input update. After 750 ms without a refresh, the RC loop clears movement. Opposing directions cancel, takeoff is guarded by battery state, and disconnect clears control state before closing sockets.

This is the general rule I took from the project: an interface can express user intent, but the native control layer must enforce the safety invariant.

Turning raw H.264 UDP into live video

Receiving video packets was easier than displaying them. The first version could count incoming bytes while the video surface remained blank.

The Tello sends important H.264 SPS/PPS initialization near the beginning of a stream. Starting FFmpeg too late means the decoder may never learn how to interpret later frames.

The reliable startup sequence became:

1. Bind telemetry and video sockets.
2. Start FFmpeg and its stdout reader.
3. Start the UDP receiver.
4. Send streamon only when the pipeline is ready.
5. Buffer H.264 until an SPS NAL unit appears.
6. Feed FFmpeg from that valid boundary.
7. Extract complete JPEG images from image2pipe output.

H.264 UDP → SPS-aware buffer → FFmpeg → JPEG frames → Tauri event → Vue

FFmpeg produces 960×540 JPEG frames at up to 15 FPS for the interface. The original H.264 packets can simultaneously be written to disk and remuxed into MP4 without re-encoding. That preserves quality and avoids adding a competing encoder workload.

Photos, recordings, and flight logs

The current decoded JPEG becomes the photo source. Recordings keep the original compressed H.264 stream and use FFmpeg only for the final container conversion. CSV logs record timestamps, control values, and recent telemetry.

Outputs live under ~/Pictures/Tello-Rust/, outside source and build folders. Uninstalling the application therefore does not erase captures.

FFmpeg is an honest runtime dependency. The application launches the system binary rather than pretending its small Rust executable contains a multimedia stack.

Testing beyond compilation

The normal check builds Vue and runs Rust tests:

make test-rust

For a release candidate I also use strict Clippy and an optimized build:

cargo clippy \
  --manifest-path desktop-rust/src-tauri/Cargo.toml \
  --all-targets --all-features -- -D warnings

make build-rust

Unit tests cover pure telemetry and control logic. They cannot prove Linux route selection, Tello radio timing, decoder initialization, window focus, or physical movement. Those require a staged test with the real drone: connect while landed, inspect telemetry and video, verify landing, then fly low in a clear area.

The core controller was physically validated for commands, telemetry, video, photos, and MP4 recording.

Linux installation without root

make install-rust builds the optimized application, installs it under the user’s local data directory, adds a .desktop launcher, and registers the icon. No root access is needed. Uninstallation removes the binary and launcher but preserves captures.

The optimized program measured about 8.7 MB during development. That small size has context: Linux supplies WebKitGTK and GTK, while FFmpeg remains external. The binary is small because it reuses system components, not because the full runtime stack disappeared.

Optional variants without destabilizing the core

I kept experimental capabilities in sibling projects:

• desktop-rust-face adds a pure-Rust CNN detector and visual overlay.
• desktop-rust-route adds typed JSON routes, validation, progress events, and conservative mission controls.

The face worker uses a one-frame bounded queue so stale inference is discarded. The route engine validates complete plans in Rust, blocks manual RC during a mission, and always leaves landing available.

Separate variants made the architectural cost of each feature visible while keeping a small manual controller as the reference.

What I would reuse

• Put native authority and safety rules below the web UI bridge.
• Validate the selected network, not merely socket creation.
• Treat decoder startup order as part of the media protocol.
• Bound optional workloads and drop stale real-time work.
• Keep control, video, and inference on independent paths.
• Measure the final binary separately from compiler caches and bundles.
• Test physical systems in layers; compilation is only the first one.

Rust did not eliminate operating-system dependencies or drone risk. It made the custom part of the system small, explicit, and difficult to misuse accidentally. That was exactly what I wanted from the exercise.

Frequently asked questions

Can Rust control a DJI Tello without a vendor library?

Yes. The Tello exposes text commands, telemetry, and H.264 video over UDP. This controller implements those paths directly with Rust standard networking.

Why combine Rust with Vue?

Vue is productive for the camera-first interface. Rust owns networking, concurrency, media processes, files, and safety state. Tauri provides a narrow command/event bridge between them.

Is FFmpeg embedded in the Rust binary?

No. The current application expects ffmpeg in PATH for live decoding and MP4 remuxing.

Was it tested with a real drone?

Yes. The core controller was tested for physical control, telemetry, live video, photos, and recording. New flight-affecting changes still require conservative revalidation.