Virtual Reality

Oculus Quest Development with Houdini Unity and USD

Oculus Quest Development with Houdini, Unity and USD

Part One

This is the first in a series of articles where I look at Oculus Quest Development with Houdini, Unity, and USD. USD is Pixar’s comprehensive scene format, Universal Scene Description. I first saw the power of USD a few years ago during a Pixar demonstration that showed its power. For example, it was able to render scenes with 20+ million polygons in real-time. Now that Houdini 18 has Solaris, which is built on USD, it is now possible to easily read and write USD files.

Oculus Quest Development with Unity

First, let’s look at the Oculus Integration package in Unity. We want to make sure that we have the correct settings. In my examples, I am working in Unity 2019.3.f6. First, I always switch the platform to Android when creating a new project for the Quest. Next, I set the texture compression to ASTC. This can take time depending on how many textures there are in the project.

Oculus Quest Player Settings in Unity


Next, we want to make sure the player settings are correct. With the player settings open we want to make sure of the following:

  • Remove the Vulkan API from the Graphic APIs
  • Color Space is set to Linear
  • That you have the Oculus SDK installed in the XR settings.

Stereo Rendering Mode – I have Single Pass selected. I have this based on that Multi-Pass has been broken so far. Oculus released a new version February 6th but I have not yet tested Multi-View

Installing USD in Unity

Installing Universal scene description is easy. First, go to window / package manager. This will open Unity’s Package Manager. Under Advanced, we need to make sure that show preview packages are checked. Lastly, now that we have preview packages selected, all we have to do is install USD. Find USD in the list and click install.

Lastly, if you don’t want to use the Package Manager you can always pull from Unity’s USD Git Hub

Once USD has been installed, a new USD menu will appear. This is where we can import and export USD files.  Unity gives us the option to import a USD as a game object, a prefab,or a timeline clip. Furthemore, Unity gives us the option to export USDZ,the zipped format for USD.

USD in Houdini with Solaris

Other Houdini Tutorials:

Houdini Height-Fields for Terrains

Houdini Height-Fields for Terrains

With Houdini height-fields for terrains, we cover the basics of creating terrains in Houdini. Houdini height-fields are 2-dimensional volumes. In addition, they are also native in Unity and Unreal Engine as terrain objects. Above all, creating terrains in Houdini artist-friendly. Moreover, we can create complex and realistic terrains.  Most importantly though Houdini allows us to use this work in other applications. Secondly, as a Houdini Digital Asset, we can use it within Unity, Unreal, Maya, and 3DS Max. Most Importantly, we will learn how to manipulate Houdin terrains in an art directable way. We will look at ways of masking, applying erosion, and adding in variation.  These masks can be used to drive foliage and textures. In conclusion, this makes creating terrains for use in real-time applications a very powerful feature. In further lessons, we will examine how we can use the Terrain Scatter tools

At the conclusion of Houdini height-fields for terrains one should have:

  • A solid understanding of heightfield workflows
  • How to create complex masks 
  • What is the Erode tool and how to use it.
  • How to interact with geometry

This is as a follow up to a lecture for a class I teach in Houdini so the students can re-cap the in-class lecture. I apologize for any lack of polish. Hopefully there will be some good information there.

You can download a free version of Houdin here

Other Houdini Tutorials:

Filming Stan Lee with Stereo Red Weapons in 8k

Filming Stan Lee with Stereo Red Weapons in 8k

Last week I had the amazing experience of filming Stan Lee with Stereo Red Weapons in 8k. This was for a virtual reality interview with Kevin Smith. This was a stereo shoot and took place in Stan’s small office in his home above Sunset Plaza. The shoot was for Legion Entertainment. They wanted to document important people in the comic book industry so that we could view them in Virtual Reality in the future. Which is why we went with the new Red Weapons in 8k. Stan was there with his wife and daughter and was very engaging and friendly with everyone.

red dragon 8k stereo

Behind the Scenes Filming Stan Lee

Derrin Turner, our resident VR Director and Head of Production, frames the camera.

On set with Stan Lee
on set with Stan Lee and his wife

Technical Details

As the Technical Director for VR Playhouse I designed the camera system that we used. For this shoot, we used the new Red Weapon 8K. We shot the cameras as stereo pairs on an offset nodal rig. We made sure to keep all action in one shot. This was why we had to use such a wide angle lens. This way we could shoot a clean plate, then the whole interview from one position. After the interview was finished, we then shot the remaining angles of the office.


This way we were able to blend between the interview footage and the rig removed clean plates. In the end we shot eight wedges at 8k resolutions. We were able to get decent stereo stitches out of Cara VR for Nuke. It will be challenging to rectify the lighting between the final wedges, and will probably require some finessing.

Red Weapon Stereo Arii 8mm Primes
Stereo Red Stan Lee

Other Houdini Tutorials:

Volumetric Video for Virtual Reality

Volumetric Video for Virtual Reality

Volumetric Video for virtual reality using Google Jump, Houdini, and Nuke. I love the visual aesthetic of point clouds and their ability to represent and recreate three-dimensional space. I used to bake out final gather points out of Mental Ray. I would ombine them to re-create a static visual cue of the entire animation sequence. At the time, I would also use them as combined elements for lighting calculations. With Houdini, working with point clouds is just so much fun. Currently, point clouds are everywhere,. We use them from everything from storing values, geometric reconstruction, and volumetric video. I can also just enjoy their pure visual beauty.

Google Jump Point Clouds in Post Production

Depth Maps

We can now get depth maps from the Google Jump Assembler in resolutions up to 8k. This allows us to implement some more advanced film techniques into the world of virtual reality production. As well, we can begin to see how we will start to create new production workflows. In the near future there will be tools for handling volumetric video. We will use this for 3d reconstruction in order to create a more immersive experiences. While not “light-field” in that this won’t represent correct light changes that are view-dependent, it will allow for greater depth and parallax. As well, it allows for better integration with CGI in the world of Stereo Virtual Reality. This is very tricky and inherently has problems.  In these examples, I am using the Foundry’s NukeX, Cara VR, and Ocula on the front end. Houdini from Side Effects on the back end.

Google Jump Point Clouds in Post Production

Point Clouds

This image shows the perspective of the viewing camera of the scene with both point clouds from Nuke and Houdini merged using deep and viewing through nukes viewer. The 3D tree is rendered through Mantra with a deep pass using the point cloud generated out of Nuke as reference geometry. This is important for correct spatial alignment of depth, as the depth, in this case, comes from the Google Jump Stitcher. This is not a spatially accurate representation of real-world depth, rather a spherically projected approximation. At this time, the biggest caveat is the 8-bit banding. This can be solved with a special plug-in that would incorporate a bilateral filter that would process the points while preserving edge detail.

Google Jump Point Clouds in Post Production
Google Jump Point Clouds in Post Production


Reality Capture  is my current go-to program for photogrammetry. I am just getting the hang of how it functions but it is incredibly fast, especially if you compare it Agisoft. These photos were a test using my studio space. This is with only 377 photographs. I can see where I need more camera positions to capture the whole space and will shoot it again soon. The maximum amount of images that I can do with the version of Reality capture that I have is 2500, I am not sure that I need that much, but I would like to test with around 1500.

Other Relevent Projects:

Tracking and Stabilizing in a Spherical World for Virtual Reality.

Tracking and Stabilizing for Virtual Reality.

Recently, I have the need for tracking and stabilizing footage in a spherical world for virtual reality. As a result, I started building some tools and methodologies for dealing with stitched spherical footage. As well, the footage would be coming from a variety of go-pro rigs. In addition, there is a need for 3D development and integration with 360 stitched spherical lat-long footage. In this case, all shot footage is usually coming from a six, ten, or sixteen camera rig with Autopano doing the initial stitching.

There is surprisingly little information out there on this subject, but there are some great videos done by Frank Reuter that got me on my way. As well as a few posts on the Foundry’s forum.

The idea is pretty straightforward. We want to convert our spherical stitch into a rectilinear cube map or 6 pack. In Nuke this is done with the Spherical Transform node. Through rotating along the Y we can get the four sides, and with a 90 and -90 rotations along X, we can get the top and the bottom.

Now that we have the six rectilinear images we can pipe them each into a project3D node, which in turn gets piped into their respective cameras. Create one camera with a 45-degree FOV and a 90 degree horizontal and vertical aperture. Duplicate the camera six times, one for reach axis and rotate them into position. I then project those cameras onto a sphere, re-creating my initial image perfectly. I then chose one sequence out of the six that would be best for performing a 3d track. Once the footage has an acceptable track, I then link the tracking data to an axis node that drives my camera rig.

I then create my render camera and pipe in the same tracking data and then invert the rotational curves for a nodal setup.

Other Houdini Tutorials:

Vertex Colors in Maya From a Houdini Alembic File

Vertex Colors in Maya From a Houdini Alembic File

Since I have been using Redshift as my need speed, go to render engine when I just won’t have time for Mantra.  I have been beefing up my Houdini to Maya Alembic pipeline. While straightforward, there are a few caveats. In this particular case, I was wanting to get my point/vertex colors from Houdini into Maya. One needs this if you want to render geometry with the baked-in colors from Houdini. A few notes, you will need to have a Cd attribute in Houdini of type vertex. In Houdini, Cd will usually be a point attribute, so a simple attribute promote will do the trick here. Just promote your Cd attribute from point to vertex and you should be good to go.

Importing into Maya

Unfortunately, the GUI Maya alembic importer does not do the trick, so we just need to import via the script editor with some very simple Mel. All you need to do is make sure you have the alembicImport plug-in on and type the following Mel command:

AbcImport -mode import -rcs “myPathToMyAlembic\”;

Obviously, you will substitute your own path and file name. You should now see your alembic file in Maya with the vertex colors in the viewport.
Now that we have our geometry imported correctly and we can see the vertex colors, we need to set up our Redshift shader, though we are using Redshift in this example, this method works for Mental Ray and Vray. In the modeling context, under Mesh Display you can see the Color Set Editor, which you can open to find the name of your color set. In our case, coming from Houdini, it’s just Cd.

Next, just create a redshiftVertexColor and under General. In the box labeled Vertex Set, just add in the name of our color set. In this case this is Cd. Now you just conncet the out Color to whatever slot you want, and hit the render button.

Other Houdini Tutorials: