Image: mixed

If your computer and headset are compatible, you can create your own NeRF and use our free software to view it in your VR headset. Method is as follows.

You may have seen NeRF on Twitter, but rather than watching 3D objects on a flat screen, you can bring them into your VR headset. In virtual reality, you can use your controller to move around in NeRF and turn your head to look around the virtual scene. Make the experience more immersive.

There is currently only one way to view NeRF in 3D on a VR headset. This is his Nvidia’s Instant-NGP app for Windows. This will likely change in the future, as NeRF technology could be a great solution for quickly filling the metaverse with realistic virtual objects.

What is NeRF?

NeRF is an AI model trained on a collection of images. It can reproduce the appearance of light captured at any point in the scene. This allows you to move through this neural radiance field (NeRF) in VR as if you were in the physical location of the photo or video and were looking at the real object. We have a guide detailing what NeRF is and how this technology can benefit his VR, AR and Metaverse.

Creating NeRF: An Introduction to Instant-NGP

First, you need a VR or AR headset with OpenXR and a Windows PC with an Nvidia graphics card manufactured in the last 6 years.

• RTX 3000 & 4000 series, RTX A4000–A6000, and other Ampere & Ada cards,
• RTX 2000 series, Titan RTX, Quadro RTX 4000–8000, and other Turing cards
• GTX 1000 series, Titan Xp, Quadro P1000–P6000, and other Pascal cards

Some software is required. All links are at the end of this guide.

To test Instant-NGP, follow these steps:

Download Instant-NGP from GitHub.

1. Download version of Instant NGP Look for one that matches your GPU type and extract the file.
2. Open the Instant-NGP folder and run Instant-NGP.
3. A Microsoft Defender warning may appear.click More informationafter that run anyway continue.
4. In Windows Explorer, data/nerf Drag the folder. fox Add the folder to the Instant-NGP app window.
5. Wait a few seconds for the model to train.
6. Click and drag inside the Instant-NGP window to see the 3D effect of the fox head on your computer screen. You can also scroll to zoom, middle-click and drag his NeRF in the window.
7. After 1 minute, the model probably won’t improve any further.[トレーニングを停止]Click the button to increase the displayed frame rate.
8. Click the small arrow on the top left to close the options menu and view details.

Create your own NeRF with video

If you have access to your iPhone, Record3D The app makes it a little easier to capture video along with the camera path data needed to create NeRF. Unfortunately there is no his android version of this app.

After installing the Record3D app on your iPhone, record a video of the object you want to see in VR as NeRF. It’s best to move slowly and keep the object of interest centered in the viewfinder. Export as an R3D file and transfer the file to your computer.

The Record3D iPhone app helps create NeRF. | | Image: Mixed

The rest of the process takes place on your computer.

1. R3D is actually a zip archive. Change the extension to .zip, right click and select. Extract All…
2. A new folder is created containing metadata file and RGBD folder.
3. if you don’t have python must be installed on your computer.
4. open Instant NGP Open the folder in Windows Explorer, right-click on the folder name and select copy address.
5. Tap the Windows key and start typing Command Prompt to open the app.
6. Please enter cd
7. Tap the space bar, paste the folder address, and press Enter.
8. In Windows Explorer, locate your Record3D video folder and copy its address.
9. At the command prompt, type python scripts/record3d2nerf.py --scene
10. paste the address of Record3D Open the folder and press Enter.
11. A Python script should quickly process this data and create new files with the following names: transforms.json.
12. Open Instant-NGP and Record3D Insert the folder into the window and start training.

Follow the steps below to refine NeRF and view it in virtual reality.

Crop and center the NeRF view

NeRF often has floaters and cloudy edges. It can be trimmed to remove these distractions. This is also a good way to center the object you want to see in VR. Cropping also improves performance, so it’s a useful option when the frame rate is so low that it can be disorienting when viewed in VR.

Instant-NGP includes a cropping tool to improve NeRF. | | Image: Mixed

Here’s how to crop NeRF with Instant-NGP:

1. Scroll down the Options menu and click the arrow next to it. World Transform & Crop Box Show more options.
2. Drag the six sliders labeled minutes and Max Trim and center the portion of NeRF of interest to remove clutter.
3. You can also select to rotate NeRf. rotate the worlddrag the three circles that appear.

Instant-NGP with Quest Pro’s trimmed NeRF. | | Image: Mixed

After cropping, centering and rotating, you should be able to see NeRF on your screen.

Watch Instant-NGP NeRF in VR

Now that NeRF is on your computer screen, check it out in VR.

1. Launch your VR headset or compatible AR headset/glasses.
2. In the Instant-NGP app on your computer, Connect to VR/AR headset.
3. Put on your VR headset and look around. NeRF is displayed.
4. Click and drag with the controller to move around the scene.

Editor Alan really likes watching NeRF in VR. | | Image: Mixed

If you don’t see anything on your VR headset, it may not be set as the default OpenXR device in Windows. In most cases, you can select OpenXR devices in SteamVR or the Oculus app on PC. Quest 2 and Quest Pro are compatible. Most headsets that work with SteamVR and Windows Mixed Reality should also work.

VR enthusiast Russ tweeted a video of his Instant-NGP experience using the Quest 2.

Using images for NeRF generation

You can also create NeRF using your photos. This gives you more control over the images used to train your model. This method is more complicated because you need to install some packages, and you can do more in the Windows Command Prompt.

Decoder has A no-code way to create NeRF from images Things you might want to check out.

No matter how you create NeRF, check it out in a VR headset if possible. Instead of looking through a window on your computer or phone, VR lets you move into a scene.

Note: Links to online stores in articles may be so-called affiliate links. MIXED receives a commission from the provider when purchasing from this link. In your case the price remains the same.