Vision took a revolutionary leap with Neural Radiance Fields (NeRF)—NerfStudio brings the power of 3D volumetric reconstruction to your scenes, transforming collections of 2D images into immersive 3D neural representations that can be viewed from any angle in real time on your Jetson device.

In this article you’ll learn how to train and visualize high-quality NeRF models using NerfStudio on Jetson, including advanced applications like FruitNeRF for semantic segmentation and fruit counting in agricultural robotics.

Neural Radiance Fields revolutionize 3D reconstruction by learning to represent scenes as continuous volumetric functions. Unlike traditional photogrammetry, NeRFs can:

• Synthesize novel viewpoints from sparse input images
• Handle complex lighting and transparent materials
• Generate high-quality renderings with photorealistic detail
• Enable semantic understanding when combined with segmentation (FruitNeRF)

The technology is particularly promising for edge robotics applications, where real-time 3D understanding enables precise navigation, object counting, and environmental mapping.

When training starts, navigate to:

• On-device: http://localhost:7007
• Remote: Replace <jetson-ip> with your board’s address: http://<jetson-ip>:7007

For synthetic data (used in this demonstration):

For real data:

The training process typically takes 1 hour on Jetson AGX Orin to complete 30,000 training steps.

Below is a demonstration of FruitNeRF’s 3D reconstruction capabilities, showcasing how the trained neural radiance field can render novel viewpoints of the synthetic tree dataset with semantic understanding for agricultural applications.

Experience the power of neural radiance fields with this rendered flythrough of a FruitNeRF model trained on synthetic fruit tree data. The model demonstrates accurate 3D reconstruction with semantic segmentation capabilities for precise fruit counting and agricultural monitoring.

Training on existing datasets is only the beginning. To train NeRF models on your own captured data, you need to process it into the nerfstudio format with known camera poses for each image.

To process your own data, use the following command:

For standard images or video files, install COLMAP first:

Process your data:

Train on your processed data:

The synthetic FruitNeRF data used in this demonstration provides an excellent starting point for learning NerfStudio. For comprehensive guidance on processing your own custom data with various capture methods and devices, refer to the official NerfStudio documentation.

For more information about NerfStudio and advanced configurations, visit the NerfStudio GitHub repository and FruitNeRF project.