Object Placement with Spatial Understanding in HoloLens made easy for Unity

Object Placement with Spatial Understanding in HoloLens made easy for Unity

Spatial Understanding in hololens is one of the coolest feature in Hololens, also one of the complex ones. In HoloToolkit-Unity there are prefabs and scripts provided to make it easier. All the same there still needs to be done quite a bit of work to make it work in our Apps/Games. So, what I did is take the spatial understanding example provided in the HoloToolkit, cleaned it up, refactored it and enhanced it to add some features to make a Unity designer tool out of it so that the object placement based on Spatial Understanding could be done without writing any code. I have uploaded the unity package for the tool here. Here I explain how to use it step-by-step. You can use it in existing projects also the same way. Let’s start:

1. Start a new Unity project

2. Import the Holotoolkit-unity into the project.

3. Apply the hololens project settings. Make sure to enable the Spatial Perception capability for the project.

4. Now import the Spatial understanding package which you downloaded from here.

5. Now add the following components to the scene:

  • Hololens Camera – HoloToolkit -> Input -> Prefabs
  • InputManager – HoloToolkit -> Input -> Prefabs
  • CursorWithFeedback – HoloToolkit -> Input -> Prefabs ->Cursor
  • SpatialMapping – HoloToolkit -> SpatialMapping -> Prefabs
  • SpatialUnderstanding – HoloToolkit -> SpatialUnderstanding -> Prefabs
  • SpatialScanner – Prefabs
  • SceneDesigner – Prefabs

6. Now add an EventSystem under the InputManager in the scene to make the gesture inputs work properly in the Unity editor.

7. Save the scene.

8. Now select the SpatialMapping component in the project Hierarchy. In the Inspector panel select the Room Model under Object Surface Observer.

9. Now apply a room model mesh which you have into this. For this scenario I have used a room model from another one of my projects.

10. That is it. Now we need to create/import the prefabs that we want to place in the scene. For the current purpose I am using the Western Props I got from here

11. Now select the SceneDesigner component. In the Inspector panel, under the Scene entity manager section, select Object Placement Definitions change the size to 2.

12. Under the Element 0, set the Scene Content Prefab to the prefab that you want to place in the scene. In the other options they options are the Placement Type, Placement Rule and Placement Constraints. Here I have selected Place On Shape as the Placement Type. If you select On Shape as the Placement Type, then you need to select a Shape. Here I have selected the Couch as the shape. (I’ll explain how to add more shapes in detail below). I have selected the Rule and Constraint as “Away from Other Objects” and left the other fields for now. The other setting properties are pretty straightforward and have tooltips attached to them.

So this is the final set up, where I want the “Wooden Bucket” on a couch if there is one and the “Wooden Barrel” on the floor of the room.

13. Now run the scene in Unity/Emulator after building the game, and it starts the scanning process.

14. Once the scanning has covered enough space tap on the message to finish scanning.

15. Once the scanning is complete the prefabs you configured previously will be placed based on the rules and definitions if the criteria are met.

Well, that was easy wasn’t it. Not a single line of code was needed.

Some additional notes: Once you start using it you’ll notice that, if you select “On Shape” as the placement type then there are currently four pre-defined shapes provided.

These shapes are defined in the SceneEntityManager.cs file with an enum

public enum Shapes {
    Platform,    
    Chair,     
    Couch,     
    EmptyTable
}

These shapes are defined in ShapeFinder.cs

private void CreateShapeDefinitions()
    {
        if (!SpatialUnderstanding.Instance.AllowSpatialUnderstanding)
        {
            return;
        }
        List<SpatialUnderstandingDllShapes.ShapeComponent> shapeComponents;
        List<SpatialUnderstandingDllShapes.ShapeConstraint> shapeConstraints;
        //Platform
        shapeComponents = new List<SpatialUnderstandingDllShapes.ShapeComponent>()
        {
            new SpatialUnderstandingDllShapes.ShapeComponent(
                new List<SpatialUnderstandingDllShapes.ShapeComponentConstraint>()
                {
                    SpatialUnderstandingDllShapes.ShapeComponentConstraint.Create_SurfaceHeight_Between(0.2f, 0.6f),
                    SpatialUnderstandingDllShapes.ShapeComponentConstraint.Create_SurfaceCount_Min(1),
                    SpatialUnderstandingDllShapes.ShapeComponentConstraint.Create_SurfaceArea_Min(0.035f),
                }),
        };
        AddShape("Platform", shapeComponents);
...
...

So if you want to add more shape definitions to the tool, just add more shape definitions to the shape finder class. Make sure that the enum value matches the name that you give to the shape while you add the shape. All other plumbing for it is taken care of.

// SceneEntityManager.cs
public enum Shapes {
    Platform,
...}

// ShapeFinder.cs
AddShape("Platform", shapeComponents);

Well, that’s it. I am still in the process of enhancing this further with few more features and adding some comments too 🙂 In the mean time go ahead and try it out and let me know your feedbacks and or queries.

Courtsey: HoloToolkit-Unity examples, Chad Carter from Globalcove

 

Leave a Reply

Your email address will not be published. Required fields are marked *