HyperDrone: Simulation-to-Reality Transfer at Millions of Frames per Second

HyperDrone. Simulation-to-Reality Transfer at Millions of Frames per Second. By Jonas Eschmann and one other, from the University of California, Berkeley. Published on the twentieth of October. Its code repository is public but slight: three source files.

Training a quadrotor from camera images in simulation depends on rendering those images in enormous batches, and speed is the constraint this work goes after. Existing batch quadrotor simulators set the bar it measures itself against. The test of any such simulator is twofold: it has to produce observations quickly enough to train on, and realistic enough that what is learned from them still works once a real vehicle leaves the screen and takes to the air.

HyperDrone is a quadrotor simulator built around high-performance batch rendering, and it reports being more than twenty times faster than existing state-of-the-art batch quadrotor simulators. It pairs highly optimized multirotor dynamics with a new ray-tracing engine called HyperLux, and both components can run entirely on the GPU.

HyperLux is designed to take full advantage of modern ray-tracing hardware, with backends for Nvidia OptiX, Vulkan, Apple Metal and WebGPU. It renders RGB, depth, semantic and instance segmentation, surface normals and optical flow. Fidelity is configurable, and at the high end includes reflections, realistic lighting, motion blur, anti-aliasing, and physically based rendering.

Even at its highest-fidelity settings, HyperLux reaches millions of frames per second on consumer-grade GPUs while rendering realistic 3D scenes. To show the simulator is useful and not only fast, the authors train a vision-based yaw control policy purely in simulation and transfer it to a real small quadrotor, a Crazyflie with a camera, where it runs completely onboard.

The claim in the title is the pairing. Millions of frames per second on ordinary hardware is the speed half, and a policy that leaves simulation and flies a real drone using its own onboard compute is the transfer half. HyperDrone presents them as one package, with rendering fidelity left as a setting the user can turn up or down.