Face Express, a new boarding procedure that utilizes NEC’s face recognition system, is now in operation at Narita International Airport. Once passengers register their facial image in Face Express, they will be able to access and proceed through subsequent procedures at the airport, including checking in baggage, entering the security checkpoint, and boarding the plane, all without showing their passport and boarding pass. This will enable seamless and contactless check-in procedures. This paper discusses the difficulties in demonstrating the accuracy of face recognition in the Narita International Airport “One ID” system development project, the importance of adjusting image quality for each installation location, the design for realizing a “walking pace”, and the efforts to coordinate with various stakeholders related to existing operations.

1. Introduction

With the increase in global air travel, the airline industry has been challenged by congestion that impedes a smooth and reliable entry and exit process and a pleasant travel experience for passengers. In order to improve this situation, IATA (International Air Transport Association) established a working group to study improvements under the keyword “FAST TRAVEL”.One of the key solutions focused on was the One ID Passenger Process ― a new concept that emphasizes the use of biometric technology including face recognition to streamline boarding and immigration procedures. Face Express is a Japanese version of One ID developed in order to bring the One ID concept to fruition in Japan. The first Face Express system in Japan was built by NEC at Narita Airport¹⁾²⁾.

2. Overview of the Face Express System Installed at Narita

The Face Express system at Narita Airport consists of a face recognition platform with four touchpoints where self-service kiosks and turnstiles are provided to enable to passengers to carry out boarding procedures on their own (Fig. 1 & Photo 1). Passengers can register their faces with Face Express at any of the touchpoints except the self-service boarding gate.

The face recognition infrastructure uses the NeoFace engine, which won first place in a benchmark test of face recognition technology conducted by the U.S. National Institute of Standards and Technology (NIST)³⁾. By using multiple engines to perform parallel processing, the NeoFace engine has sufficient performance to handle the number of passengers during Narita Airport’s busy season, enabling authentication to be performed without delay.

The four touchpoints are outlined below.

3. Integrated with the Aviation Industry’s Standard System

In the airline industry, airlines from all over the world (especially those operating international flights) fly into a variety of airports in different countries. In order for airlines to be able to provide the same passenger handling services everywhere, a system has been established to allow the sharing of boarding systems based on the concept of “common use. In order to achieve this, solutions that comply with IATA’s regulations (CUSS/CUPPS Technical Specification) have been installed in airports around the world.

One of the greatest challenges NEC faced while working on this project was how to make a strong argument for the incorporation of face recognition technology in the IATA standard-compliant environment where information sharing between airline host systems was already well established. So we talked to the vendors who had already been providing common-use environments to airports (all European and North American companies because the post-World War II development of the aviation industry was led by Europe and North America) and asked them how we might best go about incorporating IATA standard-compliant face recognition technology. We also discussed how this could be achieved without having to modify existing boarding operation applications used by the airline companies. Consequently, we were able to design One ID so that it not only conforms to IATA standards, but also complies fully with the standards of the Association of European Airlines (AEA), which specify detailed ICT communications rules in a common use environment.

At times these discussions with common use vendors in Europe and North America became quite intense, reflecting the different cultures and backgrounds of the various participants. Fortunately, after many hours of tough negotiations, we finally succeeded in hammering out an agreement.

4. Coordinating Specifications with Stakeholders

Since Face Express was the first working system of this type introduced in Japan, it was difficult to provide potential users with a proper understanding of how it actually worked. This was not only an issue for us, but also for Narita International Airport Corporation (NAA) and for the airlines that would be using the system. To help people picture what the system would look like when actually operating, we created mockups of the gates with virtual reality (VR) support. This made it easier to finalize detailed specifications according to customer requirements.

Because machines would be taking over identity verification from human staff, security standards based on the assumption that humans would be performing identity verification had to be revised for compatibility with the new technology. NAA played a central role in this process, coordinating with the Japan Civil Aviation Bureau (JCAB) under the Ministry of Land, Infrastructure, Transport and Tourism. NEC proposed technical means of identity verification and the relevant rules were eventually revised as necessary.

In addition, because the system would be using facial data, NAA engaged in repeated discussions with the Personal Information Protection Commission. JCAB also established a study committee to investigate the best way to handle personal data for the introduction of One ID. Subsequently, they released a guidebook on how to handle personal data in the One ID service at airports where face recognition technology is being utilized. After numerous discussions with NAA and airlines, NEC found a way to meet apparently conflicting requirements ― that is, on the one hand, to obtain informed consent from the passenger to provide facial data and, on the other hand, to improve speed and convenience by reducing the number of screens displayed on kiosks as the airlines demanded. By successfully integrating these two capabilities, we were ultimately able to complete the system in accordance with the guidelines.

5. How we met the requirement for high face recognition accuracy

At the time, kiosks that performed identity verification using passports and registered facial images were already operating at immigration counters at Narita Airport. Similarly rigorous face recognition accuracy was required for this system, but would be even more difficult to achieve because facial images would be captured on-site, rather than being pre-registered, meaning that lighting conditions would vary by time and location at different kiosks. Identity verification would therefore have to be made using photos shot under different conditions. We discuss the steps to achieve this high accuracy in the following section.

5.1 Dealing with variable on-site lighting

An airport provides a challenging environment for face recognition. Lighting conditions vary depending on such factors as the presence of large windows to allow in external light, types of lighting fixtures, as well as the location in the airport, time of day, and season. For this reason, we had to visit the various locations at Narita Airport where the touchpoints would be installed at different times of the day to check the lighting conditions and devise appropriate measures to deal with the angle of incidence and intensity of light (by setting up partitions and curtains). As a result, we were able to create a lighting environment suitable for face recognition without compromising the décor and ambience of those locations.

5.2 Designing touchpoints

It was difficult for us to accurately convey to overseas touchpoint vendors the exact nature of the issues regarding the lighting environment. To facilitate their understanding, we set up experimental environments in the vendors’ offices that simulated the lighting environment of the airport. Photos taken in these test environments were evaluated on the face recognition platform provided in the cloud. In collaboration with the vendors, we analyzed the evaluation values of the images returned from the platform to optimize the parameters.

In addition, an animation screen was added to draw the users’ attention toward the camera, thereby allowing it to capture a frontal view of the face which is most suitable for face recognition. Furthermore, the camera control takes into account the time it takes for the user to turn their gaze to the camera to further ensure reliable face recognition.

5.3 Quantitative evaluation

Ideally, system accuracy would be evaluated on location at the airport. However, it is extremely difficult to get a large enough number of test subjects to pass through the system. Consequently, we executed an evaluation with a test group in a test environment. We also conducted tests to fine-tune cameras to achieve image quality equivalent to the test environment. These tests helped assure that we would achieve the required degree of accuracy when the system was implemented at Narita Airport.

5.3.1 Evaluation with a large test group

To achieve the required degree of accuracy, we would need images of several hundred people captured at the touchpoints. To get as many subjects as possible, we called for volunteers from throughout the NEC group. With more than 300 NEC Group employees participating in each test run, we conducted the evaluation several times and collected thousands of facial images. When shooting these pictures, we set up a lighting environment that simulated actual locations and used the actual kiosks at the touchpoints. As a result, we were able to capture facial images that incorporated various human behaviors such as how people would operate the screen and how fast they would walk through the gates. Thanks to the vast number of facial images collected in these tests, we proved that the required accuracy could be achieved in the test environment. These tests also helped to determine the quality of camera images.

5.3.2 Camera adjustment test

Depending on the locations and directions of the touch points, the conditions in which the light falls on the face differ. To make it possible to perform face recognition irrespective of the color tones of faces resulted from the lighting condition, we adjusted camera parameters at each kiosk of each touchpoint. As a result, we confirmed that capturing facial images would be possible with the image quality equivalent to the test environment at all the touchpoints of Narita Airport.

6. Achievement of Ideal Walking Speed through the PRS and ABG

One of the most effective ways to make passengers walk through a face recognition gate at the ideal speed is to install a gate with a longer lane ― such as the double flapper gate used at customs, for example. The idea is to extend the time the passenger spends walking through the gate, giving the system more time, from when the camera starts capturing the facial image of the passenger to when they get through the flapper gate, to complete face recognition.

However, there are hallways and waiting areas in front of the PRS and ABG at Narita Airport that make it difficult to obtain sufficient installation space, leaving only enough room for a single flapper gate with a shorter lane. Furthermore, from the standpoint of security, it was necessary to initiate face recognition as soon as the sensor in the lane detects the passenger in order to ensure fail-free identity verification (Fig. 2).

Working within these restrictions, we narrowed down the detection range of a facial area in the shooting range of the camera, optimized internal processing of everything from camera operation to feature extraction of passengers’ faces, and developed a mechanism to choose good quality images from multiple shots of each passenger’s face as they pass through the gate. These efforts succeeded in achieving compatibility between the world’s highest levels of authentication speed and recognition rate.

In addition to gate clearance using face recognition, the PRS also offers various gate clearance modalities including the use of conventional boarding passes and Face Express face registration at the PRS for passengers who don’t stop by the CUSS or CUBD kiosk.

As well as increasing the speed with which passengers can be processed by using face recognition alone, Face Express can also optimize overall throughput of gate lanes by improving the allocation of passengers to lanes and methods for guiding passengers.

Because this was Japan’s first system of its kind, we also created VR images of entire lanes for this project, which were used from the beginning of the design stage so that we could get a more tangible feel for what actual operation would be like. Using these VR images, we conducted new trials to experience how passengers would feel when passing through the gate and while waiting in the queue in various situations with various passenger loads. By asking the NAA and airline staff to actually experience this VR image-supported environment, we were also able to have more specific discussions on the operation of the entire lanes even when the actual gate didn’t exist yet. This all enabled us to create an operation plan to facilitate passing the gates (Photo 2).

7. Conclusion

NEC has successfully developed and installed at Narita Airport Japan’s first face recognition boarding system. In addition to improving the speed and convenience of boarding procedures, this system is ideally positioned to handle the imperatives of the so-called “new normal” because of its contactless feature. Moreover, the system can be made even more convenient by connecting to peripheral systems and to other airports. NEC is now working to promote this system and deploy it at various airports around the world where it can help achieve convenient, contactless airport services that meet today’s needs for safety and security.

Reference

Authors’ Profiles