The US National Institute of Standards and Technology (NIST) is to conduct new tests that will assess the capability of face recognition algorithms to correctly identify or ignore people appearing in video sequences. The tests —Face in Video Evaluation (FIVE)— will assess both comparative and absolute accuracy measures.

The goals are to determine which algorithms are most effective and whether any are viable for the following primary operational use-cases:

• high volume screening of people in the crowded spaces (e.g. an airport);
• low volume forensic examination of footage from a crime scene (e.g. a convenience store);
• people in business meetings (e.g. for video-conferencing);
• people appearing in television footage.

According to NIST these applications differ in their tolerance of false positives, whether a human examiner will review outputs, the prior probabilities of mate vs. non-mate presence, and the cost of recognition errors.

This test follows on from the Face Recognition Vendor Tests of 2000, 2002, 2006, 2010, and 2013.

These tests gave quantitative statements of accuracy and speed of mostly still-image face recognition algorithms.

The last test included a video track (FRVT class V) – results from that work are being provided to participants.

The new FIVE program supersedes the FRVT work but proceeds in an almost identical manner.

Software submitted to NIST will be evaluated on sequestered sets to quantify accuracy and speed.

Algorithms must be implemented behind the formal C++ API to be published by NIST. […]