Share holograms between Hololens Emulator and Unity3D using UNet

Share holograms between Hololens Emulator and Unity3D using UNet

Hologram sharing is one of the coolest feature in Hololens. Many articles are out there which shows you how to share holograms between hololenses or between a hololens and an emulator using the HoloToolkit. There are two approaches in it – One needed a Sharing server console app for it to work, the other one works on UNET -Unity’s networking feature. Examples for each of it are there in the Holotoolkit samples (when I started only the Sharing Service solution was there which is what actually prompted me to do this), but starters might get overwhelmed by it. So here I am going to explain how to do this from scratch with the bare minimum setup. I am not considering features like anchoring or location synching etc. – just to share the same scene and manipulated objects using Unity and he Hololens Emulator. This should work with 2 hololenses too. Oh, remember, you still need the Holotoolkit, for gestures and all

Here is the step by step process:

  1. Start a Unity 3D project and import the Holotoolkit into it.
  2. Make the initial settings for a hololens apps -Scene, Project and Capability Settings. Note:Make sure that in Capabilities make sure to check “Internet Client Server” and “Private Network Client Server” capabilities.

3. Now in the project window from Holotoolkit -> Input -> Prefabs add the HololensCamera and InputManager prefabs to the Hierarchy window. Add a cursor (doesn’t matter which one) from Holotoolkit -> Input -> Prefabs -> Cursor

4. Now In the InputManager add an EventSystem by Right clicking on it then UI -> EventSystem. This is needed for the InputManager to work properly.

5. Now save the Scene and Project.

6. Add a simple 3D cube into the scene and keep it away in front of the camera at a distance where it is visible to you in the game window when the app starts. This is our Player – the objet which would be manipulated by the respective devices and the changes seen in real-time by other devices – in this case between the emulator and the Unity 3D game window. Now play the game in unity and you should see something like this.

7. Now add three components to the cube – Network Identity (Unity), Network Transform (Unity) and Hand Draggable script (Holotoolkit). Now check the “Local Player Authority” option in the Network Identity component of the cube. Once you are done it would look like the below screenshot.

8. Save the cube as a prefab and delete it from the scene.

9. Create a new GameObject in the scene and name it as NetManager. Add Network Manager conponent to it. In the Network Manager component expand the Spawn Info section and add the cube prefab we just saved to the “Player Prefab” option.

Now we have all the set up to create and manage the network and our “player” (cube)object which would be spawned every time a new client joins the network. Each players will be able to see what other player is doing too. All we need now is a UI to let the player join as a Host (acts as both Server & Client) or as a Client. Now many people who already knows about the Unity’s networking feature would say we could use the NetworkManagerHUD instead of creating a UI of our own. I tried that already and somehow it is not working with the emulator or hololens (works in Unity as it should :)). If anyone manages to figure out how to use the HUD let me know. Hence we are going to create our own, though a simple one. Let’s do that.

10. Add to buttons under the NetManager gameobject we created and Name them as Host and Client. Make sure that the Canvas which contains these buttons is set it’s Render Mode option as World Space. Adjust the button style and position according to your needs.

11. We need a script to do the actual operation of starting a host and a client. Create a folder “Scripts” and add a new C# script to it named “NetworkController”. In the NetManager gameobject select Add Component and attach this script to it.

12. Place this code inside the script.

using UnityEngine;
using UnityEngine.Networking;
public class NetworkController : MonoBehaviour
    NetworkManager netManager;
    NetworkClient host;

    void Start()
        netManager = GetComponent<NetworkManager>();

    public void StartHost()
        Debug.Log("Starting host");
        netManager.StartHost(); Debug.Log("Started host");

    public void StartClient()
        NetworkClient client = netManager.StartClient();

    public void OnDestroy()

13. Now select the Host button under the Canvas in NetManager. Add a OnClick function in its Inspecter panel. Select the NetManager as the object and Network Controller-> StartHost as the function.

14. Do the same for the Client button. Selecting NetworkController->StartClient as the function.

15. We need one more thing before we are done. The ip of the Hololens emulator. Launch the emulator and Navigate to the device portal. Copy the IP shown in the browser address.

16. Now go to the NetManager gameobject in our Unity scene and select the NetworkManager Component in it. In the Network Info section paste the copied IP address in the Network Address option.

17. That’s it. Now build your project targeting Hololens and adding the current open scene to the build.

18. After the build is complete go to the folder where the project was built and open the project in Visual Studio.

19. Once you open it in Visual Studio make sure to set the Architecture to x86 and Deployment device to Hololens Emulator

20. Press F5 or the Play button. Your app would deploy in hololens start. At the same time press play in Unity too.

21. In the emulator simulate a tap on the “Start Host” button. Now a Cube would be created at your position. You need to move back to see it since it gets created at (0,0,0). Using tap and Hold you’ll be able to drag it around.

22. Now in Unity click on “Start Client” button.

You should see your player cube along with the emulator’s player cube in your scene. At the same time your emulator will also be updated with the Unity’s player appearing in the Emulator scene (Again, you’ll have to move a bit backwards to see both the cubes together. You can adjust the spawn position if you need).

23. Now try holding and dragging their respective cubes (better to add a tag with player names to each cube in order to recognize them) and you can see the same cube moving in the other “device’s” scene at real time.

Well, that’s it. You have created a hologram sharing app to share between the Unity editor and Hololens Emulator. Now once you are comfortable playing around this you can go to the SharingWithUNet sample in the Hololens Toolkit. Also, I would strongly recommend reading the Unity documentation on the UNet. I hope this would be helpful to those who has been looking for a starting point on Hololens sharing feature development in their apps without using any Hololenses or the Sharing services.

Note: It goes without saying that always test your app with the real devices before rollout 🙂

3 Replies to “Share holograms between Hololens Emulator and Unity3D using UNet”

Leave a Reply

Your email address will not be published. Required fields are marked *