Physiological information recognition: Drowsiness-level (Arousal-level) estimation from facial videos
Featured Technologies
July 25, 2018
Background
Improving productivity through "work style reform" has become an issue that impacts not only corporations but also broader society. Against this background, intensified efforts are being made to implement initiatives to improve performance and drive forward the cycle of work reform such as by visualizing working hours to cut down on overtime work.
A lowered level of arousal or drowsiness is one factor associated with lowered performance*1. The visualization of drowsiness to help each and every employee achieve an optimal mental and physical state to improve performance is an important consideration for business, but achieving this at low cost using a widely applicable approach is not an easy task. For instance, in the automobile industry where drowsiness can lead to serious accidents, manufacturers have developed a way to detect signs of drowsiness using operation log for the steering wheel of a vehicle. It would be complicated to incorporate this type of technology in a general office environment. In another approach, technology has been developed using facial images of the driver to measure the duration of eye closure and frequency of eye blinking. This technology can estimate the level of drowsiness with high accuracy, but the duration of a blink is several hundred milliseconds, which would require high-speed image processing. It would be next to impossible to achieve this level of output using low-end devices.
The characteristics of the new technology
In order to turn the impossible into a reality, our labs have come up with new technology that detects signs of drowsiness by focusing on eyelid variability. Because eyelid variability involves slow motions, signs of drowsiness can be sufficiently captured using facial image data at one third the frame rate needed for measuring eye blinking. As a result, drowsiness can be estimated using a low-end device which would enable the wide-scale application.
Fig. 1: Drowsiness can be accurately estimated using a low-end IoT device.
Detecting signs of drowsiness by focusing on slow-motion eyelid variability
In order to accurately estimate the level of drowsiness*2 using facial images captured on web cameras and other devices, our labs have newly discovered features for two types of eyelid variability.
Time variability Drowsiness makes it difficult to keep the eyelids motion-constant. Variability of the state in which the eyelid is open (indicated with red line in Fig. 2) is the feature for time variability. Because eyelid variability involves slow motions, this value can be derived from facial image data at a low frame rate of about five frames per second (five images captured every second). Fig. 2: The higher the level of drowsiness, the higher the time variability.
Left-right discrepancy Drowsiness makes it difficult to keep left and right eye movements synchronized. The discrepancy in movement between the left and right eye (indicated with a red line in Fig. 3) is the feature for left-right discrepancy. Because eyelid variability also involves slow motions in the case of left-right discrepancy, this value can be derived from facial image data at a low frame rate just like for time variability. Fig. 3: The higher the level of drowsiness, the higher the left-right discrepancy. In a test to verify effectiveness, this technology was used to process 41 hours of facial image data collected from 29 subjects. The results indicated that compared to 15 frames per second of facial image data processed with previous technology, this technology could achieve the same level of accuracy by processing facial image data at 5 frames per second (Fig. 4). Fig. 4: The new technology can achieve the same level of accuracy at one third the frame rate of conventional technology.
Using this technology, we intend to work in collaboration with partners to create solutions that will help each and every employee achieve an optimal mental and physical state for performance, as well as to contribute to such outcomes as improved lifestyle habits to maintain appropriate levels of arousal. Our research findings were presented at the 40th International Engineering in Medicine and Biology Conference (URL: https://embc.embs.org/2018/), the flagship conference of the medical engineering field.
*1
The relationship between level of arousal and performance. The Yerkes-Dodson's law, a basic psychophysiological model, dictates that suitable arousal levels can elevate performance.
*2
Level of drowsiness. Three trained observers ranked the expressions on the faces of the participants from a scale of 1 to 5, indicating "No sign of drowsiness" to "Extreme signs of drowsiness." The estimates were derived from the eyelid movements with the average value taken as the correct value.
※
The information posted on this page is the information at the time of publication.
https://embc.embs.org/2018/), the flagship conference of the medical engineering field.