Category

Real-Time

Oculus Quest Development with Houdini Unity and USD

Oculus Quest Development with Houdini, Unity and USD

Part One

This is the first in a series of articles where I look at Oculus Quest Development with Houdini, Unity, and USD. USD is Pixar’s comprehensive scene format, Universal Scene Description. I first saw the power of USD a few years ago during a Pixar demonstration that showed its power. For example, it was able to render scenes with 20+ million polygons in real-time. Now that Houdini 18 has Solaris, which is built on USD, it is now possible to easily read and write USD files.

Oculus Quest Development with Unity

First, let’s look at the Oculus Integration package in Unity. We want to make sure that we have the correct settings. In my examples, I am working in Unity 2019.3.f6. First, I always switch the platform to Android when creating a new project for the Quest. Next, I set the texture compression to ASTC. This can take time depending on how many textures there are in the project.

Oculus Quest Player Settings in Unity

 

Next, we want to make sure the player settings are correct. With the player settings open we want to make sure of the following:

  • Remove the Vulkan API from the Graphic APIs
  • Color Space is set to Linear
  • That you have the Oculus SDK installed in the XR settings.

Stereo Rendering Mode – I have Single Pass selected. I have this based on that Multi-Pass has been broken so far. Oculus released a new version February 6th but I have not yet tested Multi-View

Installing USD in Unity

Installing Universal scene description is easy. First, go to window / package manager. This will open Unity’s Package Manager. Under Advanced, we need to make sure that show preview packages are checked. Lastly, now that we have preview packages selected, all we have to do is install USD. Find USD in the list and click install.

Lastly, if you don’t want to use the Package Manager you can always pull from Unity’s USD Git Hub

Once USD has been installed, a new USD menu will appear. This is where we can import and export USD files.  Unity gives us the option to import a USD as a game object, a prefab,or a timeline clip. Furthemore, Unity gives us the option to export USDZ,the zipped format for USD.

USD in Houdini with Solaris

Other Houdini Tutorials:

Real-Time Point Clouds Zed Mini

Real-Time Point Clouds with the Zed Mini

Lately, I have been experimenting with real-time point clouds with the Zed Mini from Stereolabs. Firstly, I am a big believer in real-time point clouds being a viable solution for co-located, virtual reality experiences. Concurrently, I am also interested in examining the development of this technology, and how we use artificial intelligence and machine learning to examine the world we live in. The Zed Mini functions much in the same way that the Kinect Azure does. The big difference is that the Kinect is more plug and play, while the Zed needs external libraries. Lastly, another big difference is the image and point-cloud quality, with the Zed being far superior. For some people, the fact that some of the tools need to be built using CMake and Visual Studio will be a deal-breaker. Stereolab provides resources in their Git Repository.

The Zed Tools

After I built the tools from the source, I now have access to some tools in samples/bin. As an example, the most immediately useful is ZED_SVO_Recording.exe. Firstly, this allows me to write an SVO file to disk. SVO is a proprietary format from Stereolab. Most importantly, this format allows for the recording of all the data from the Zed camera into a compressed file format. This will allow me to construct the point-clouds after shooting. In order to run these tools, I need to call them from the command-line. For ZED_SVO_Recording the only argument I need is a path to where I want the SVO file saved. So assuming I am in the shell in the correct directory, you would just write:

ZED_SVO_Recording.exe c:SVO_save_folder/mySVOFile.svo

The prompt should start scrolling frame numbers, and a ctrl-c will break the operation. One the SVO is saved, one can use some of the other tools that Stereolab provides.

Zed in Touchdesigner

I am working with a friend, Shaoyu Su to get TensorFlow and YOLO working through the Zed and Touchdesigner for object recognition and tracking.

You can download a free version of Touchdesigner here:

Other Houdini Tutorials:

Introduction to Houdini Digital Assets

Introduction to  Houdini Digital Assets. In this tutorial, we will examine the fundamentals of creating a simple H.D.A. As an example, we will create a tool that will make procedural I-beams of any size. In addition to being able to adjust the shape and size of the i-beam, the tool will automatically UV our object. Lastly, once the tool is finished it can be imported into Unity or Unreal Engine. Therefore we can use our tool in any application that takes an H.D.A. As a result, we will be able to create any i-beam of any size and use them however we want. In conclusion, Houdini Digital Assets are a great way to create tools for use in other applications. This lets us build modular reusable assets. This is a very quick introduction to show how easy it is to make a useful asset in Houdini. This will show you the basics of:

  • Linking parameters
  • Auto UV Maps
  • How to Build a simple interface in Houdini
  • How to correctly name and save H.D.A’s

Once you have done this tutorial one should have a solid basic understanding of Houdini Digital Assets and can explore further tutorials to increase your skills.
This is as a follow up to a lecture for a class I teach in Houdini. This is so the students can re-cap the in-class lecture. I apologize for any lack of polish. Hopefully, there will be some good information there.

You can download a free version of Houdini here.

Other Houdini Tutorials:

Lidar with Ouster and Touchdesigner

Real-Time Lidar with Touchdesigner and Ouster

I have been wanting to experiment with real-time lidar using the newer, small Ouster Units and Touchdesigner. The last time I did any experiments with real-time lidar it was with a Velodyne 16. Using Touchdesigner we can now easily use lidar that runs in real-time. Ever since I first got to play with that Velodyne 16, I have been an advocate of the point cloud as a means of geometry display. Points are light in memory and can contain their RGB information. Because of this, it is far easier to display real-time evolving data than it would be to try to mesh, UV and texture. Even as a post-process the very concept of meshing will always be subject to artificating and resolution problems.

Lidar Compared Depth Based Techniques

Currently, real-time Lidar solutions lag behind any depth-based solution like a Kinect Azure or a Zed Camera in regards to a resolution. As well, we do not yet get any RGB data. Moreover, to get RGB data we would need an additional device, such as a small Virtual Reality camera. However, we do get an intensity value. This acts like a black and white image.

Testing

One of my students was able to get a loaned out OS2, a $16,000 unit so that he can use it in his Masters Thesis at USC School of Cinematic Arts. We used the Lidar with the Ouster running through Touchdesigner. Touchdesigner has an Ouster Top and Chop that allows us to pull in all the data. The unit has the following specs:

  • The OS0 lidar sensor: an ultra-wide field of view sensor with 128 lines of resolution
  • Two new 32 channel sensors: both an OS0 and OS2 version
  • New beam configuration options on 32 and 64 beam sensors

Volumetric Video

You can see some of my research with Google and the Foundry here. I believe this is how we will eventually represent large, realistic datasets for both virtual and augmented reality. Self-driving car and machine learning have brought us smaller, better and faster lidar and I am seeing the technology trickle down into interesting alternative use cases.

Intensity Pass

These images below show the full image range of a normalized “intensity” pass.

Other Houdini Tutorials:

Instancing Geometry with Touchdesigner

Instancing with Touchdesigner

In an introduction to instancing geometry in Touchdesigner, I show how we can instance geometry on the GPU.  Instancing allows us to create multiple copies of an object. Since they are just copies, we can create a large number of copies. Moreover, we can give them random scale, coloring, and rotation for variety. In Touchdesigner we use  CHOPS, and now TOPS in order to provide channels for instance. As a result, we can alter and change the instances over time.
An introduction to instancing geometry in Touchdesigner will provide a basic understanding of:

  • How we can generate channel data for Instancing in CHOPS
  • How we use that data on a Geometry Container

You can download a free version of Touchdesigner here.

Other Houdini Tutorials:

Houdini Height-Fields for Terrains

Houdini Height-Fields for Terrains

With Houdini height-fields for terrains, we cover the basics of creating terrains in Houdini. Houdini height-fields are 2-dimensional volumes. In addition, they are also native in Unity and Unreal Engine as terrain objects. Above all, creating terrains in Houdini artist-friendly. Moreover, we can create complex and realistic terrains.  Most importantly though Houdini allows us to use this work in other applications. Secondly, as a Houdini Digital Asset, we can use it within Unity, Unreal, Maya, and 3DS Max. Most Importantly, we will learn how to manipulate Houdin terrains in an art directable way. We will look at ways of masking, applying erosion, and adding in variation.  These masks can be used to drive foliage and textures. In conclusion, this makes creating terrains for use in real-time applications a very powerful feature. In further lessons, we will examine how we can use the Terrain Scatter tools

At the conclusion of Houdini height-fields for terrains one should have:

  • A solid understanding of heightfield workflows
  • How to create complex masks 
  • What is the Erode tool and how to use it.
  • How to interact with geometry

This is as a follow up to a lecture for a class I teach in Houdini so the students can re-cap the in-class lecture. I apologize for any lack of polish. Hopefully there will be some good information there.

You can download a free version of Houdin here

Other Houdini Tutorials:

Houdini Terrains In Unreal Engine

Terrain Generation with Houdini Heightfields

Houdini Terrains in Unreal Engine. Since Houdini 16 we can do our Unreal terrain generation with Houdini Heightfields. Heightfields are 2D volumes that are commonly used in modern game engines like Unreal Engine or Unity for terrain creation.  I use the terrain tools in Houdini to both create a realistic terrain, but also for the ability to embed splat masks. As an example, I can use the curvature of the surface as in the image above. There are multiple ways of masking heightfields in Houdini. Even more so, I can use geometry to drive shapes in the terrain.

Watch a tutorial on terrain generation in Houdini here

Anyone can download a free learning edition of Houdini on the Sidefx Website

Houdini Terrains in Unreal Engine
Houdini Terrains in Unreal Engine

Terrain Shader

Using Houdini terrains in Unreal Engine I can embed masks that I can access in a material. We can use the landscape blend node for this. Moreover, we can use any mask created with the erode node or with the mask by feature. The masks will come into Unreal with the correct naming. As an example, the erode node will give us debris, water, bedrock, etc.

Houdini Terrains in Unreal Engine

Adapative Tesselation

For the shader, I created an adaptive tessellation shader. As a result, the model will increase in resolution the closer that the viewing camera gets to the surface.

Houdini Terrains in Unreal Engine
Houdini Terrains in Unreal Engine

Procedural Foliage Spawner

Any mask that I create using Houdini terrains in Unreal Engine will be automatically available. Therefore, I can quickly create and prototype. As an example, I am using masks to drive the procedural foliage.  Moreover, All of the terrain and foliage generation is handled procedurally through Houdini’s heightfields.

Other Houdini Tutorials: