This project is face main features tracker and gaze estimator
- Gaze estimation
- Blink detection
- Tracking the user’s face in real time
- Mouth opening detection
- Developed in C++ within OpenCV and DLib
- Clone this repository:
git clone https://github.com/silviu-dev/UserFeaturesTracker.git- Navigate to the project directory:
cd UserFeaturesTracker/UserFeaturesTracker- Build and run the project using build.sh script:
./build.shThe plugin is build as an .dll which exports an simple interface:
extern "C" __declspec(dllexport)
void getUserFeatures(UserPositionCallback, UserGazeCallback, UserMouthCallback, UserBlinkCallback, bool needCalibration);There are four callbacks to handle the information from the .dll and one flag that sets the calibration step. If the calibration is enabled, then before the first call to the .dll, the user is prompted with a window and asked to look at a series of red dots in order to better calibrate the underlying models.
This plugin demonstrates versatile applicability.
In the main folder ./UserFeaturesTracker/, there are two additional Unity-based applications that leverage this plugin:
-
The first application, named Human Imitator, features an avatar capable of accurately mimicking the user's movements and the camera angle is calculated based on user relative position to the screen.
-
The second application, CubeEyeMover, introduces a unique interaction mechanic where users can manipulate a cube solely through gaze control.
Both examples highlight the diverse capabilities facilitated by the integration of this plugin with Unity (version 2020.3.22f1 was used).
- OpenCV 4.8.0
- Cmake 3.14 or newer
- MSVC 19 compiler
- Dlib 19.24.0 or newer
Contributions are welcome! If you'd like to contribute to this project, you can fork it and then submit a pull request with your improvements.
This project is licensed under the GNU Lesser General Public License v3.0. See the LICENSE file for details.