In this post, I will discuss the work I have been doing recently. I needed to extract facial features from the recorded video and for this task, I decided to use OpenFace, an open-source face recognition library. In this post, I am sharing the installation process and tutorial on detecting facial landmarks.
I tried to install OpenFace on Mac OS but couldn’t succeed. There were lot of errors and compatibility issues that I couldn’t get through. Therefore, I decided to install it on Ubuntu. For that, I installed a Virtual box on Mac and installed Ubuntu 18.04.
To install OpenFace, I followed the steps given here.
sudo apt-get update sudo apt-get install build-essential sudo apt-get install g++-8 sudo apt-get install cmake sudo apt-get install git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev wget https://github.com/opencv/opencv/archive/4.1.0.zip sudo unzip 4.1.0.zip cd opencv-4.1.0 mkdir build cd build cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D BUILD_TIFF=ON -D WITH_TBB=ON .. make -j2 sudo make install wget http://dlib.net/files/dlib-19.13.tar.bz2 tar xf dlib-19.13.tar.bz2 cd dlib-19.13 mkdir build cd build cmake .. cmake --build . --config Release sudo make install sudo ldconfig cd ../.. sudo apt-get install libboost-all-dev git clone https://github.com/TadasBaltrusaitis/OpenFace.git cd OpenFace mkdir build cd build cmake -D CMAKE_CXX_COMPILER=g++-8 -D CMAKE_C_COMPILER=gcc-8 -D CMAKE_BUILD_TYPE=RELEASE .. make
At this point, we have installed OpenFace. Now we need to download models. You can either download it manually or using a script provided in the OpenFace library. Manual download links
I ran the following command to ran the script to download the model.
cd .. sh ./download_models.sh
The script will download models in the directory
After completing this process, I ran the demo program by running the following command.
./bin/FaceLandmarkVid -f "../samples/changeLighting.wmv" -f "../samples/2015-10-15-15-14.avi"
I got an error
CEN patch expert not found. The command was searching the models in the
OpenFace/build/bin/model/patch_experts. So I copied the files (
cen_patches_1.00_of.dat) in `
If you have a video with a single face, you can use
FaceLandmarkVid or in case of multiple faces, you can use
Following is the demonstration of OpenFace on a video clip with single face.
Note: These files will be available in
OpenFace/build/bin directory. Either you can specify the full path to facial landmark detector or cd to the bin directory and run the following command.
FaceLandmarkVid -f file_name
OpenFace also provides extracted features in a form of CSV file. To extract the features from a video with a single face, run the following command
FeatureExtraction -f file_name