hmd-eyes

Building blocks for eye tracking in Augmented Reality AR and Virtual Reality VR or more generally Head Mounted Displays HMD. The purpose of this repository is do demonstrates how to implement Pupil with Unity3D. For details on Pupil computations, please have a look at the related repositories.

VR Getting Started

This guide walks you through a first time setup for your Pupil Labs VR add-on.

1. HTC Vive Add-on setup - Install your Pupil eye tracking add-on into your HMD and connect the add-on to your computer.
2. Download Pupil Software. Extract the Pupil Capture app to your Desktop.
3. Start Pupil Capture via the pupil_capture.exe in the Pupil Capture App folder.
4. Please refer to the Pupil getting started and user guide to ensure that eyes are well captured and that the pupil detection runs with high confidence (~0.8).
5. Download and start hmd-eyes demo app - This demo runs a VR experience. The demo app talks to Pupil software in the background. You will use it to calibrate and visualize gaze data within a demo scene. The demo app will start with settings dialog.
6. Start the demo with default values. You will see a view of the left eye of the hmd.
7. This would be a good point to put said device on your head.
8. Use the displayed realtime videos of your eyes to make sure they are as centered as possible and in focus.
9. Press 'c' on your keyboard to start a calibration and focus your gaze on the displayed marker as it changes position.
10. After a successful calibration, the example scene will appear again and the gaze estimate will be visualized.

Develop, integrate, and extend

Installing this package to your project

Option 1 - From the Editor

Open the Package Manager Window, click on the + dropdown, and choose Add package from git URL.... Enter the followingURL:

https://github.com/SachsLab/hmd-eyes.git

See the Unity docs for more info.

Option 2 - Modify Project Package Manifest

Edit <your project folder>/Packages/manifest.json and add the following to your dependencies section:

"com.pupillabs.hmdeyes": "https://github.com/SachsLab/hmd-eyes.git"

The advantage of this approach is that it will allow you to specify this package as a dependency of another.

See Unity docs for more info.

Adding this package as a dependency to another package

We do not yet have this package added to a "Scoped Registry", and UPM cannot resolve package git dependencies.

By default, Unity Package Manager (UPM) does not resolve git URLS as package->package dependencies. There are a couple ways to allowthis:

• For each dependency, add its URL to the project package manifest (see above).
• Use a custom solution, such as https://github.com/k0dep/Originer

Alternatively, we could host this package in a custom "Scoped Registry". For open source projects, a good option might be https://openupm.com/docs/. If/when we do decide to add this to a scoped registry, we should read this blog post on the topic: https://sandolkakos.github.io/my-blog/upm-studies/

Dependencies:

• Unity 2020.3+
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Community

Chat with the hmd-eyes community on Discord.