For more than a century, latent fingerprints have been one of the most important forms of evidence used by law enforcement and criminal investigative agencies worldwide. While tremendous progress has been made in conventional fingerprint matching, matching latent fingerprints has proven to be a more difficult problem due to the poor quality of the prints. Typically obtained under less than ideal conditions, latent prints are frequently incomplete or degraded by background noise. Since fingerprint identification depends on the quality of the print, matching technology with much higher accuracy is required to identify suspects, increase the arrest rate, and help build a safer and more secure society. NEC has been working on fingerprint recognition for nearly fifty years, constantly striving to increase the accuracy of our matching technology. This paper provides a historical overview of NEC’s research in fingerprint matching technology and introduces our latest technological advances in this field.

1. Introduction

Today, biometrics such as face recognition, iris recognition, and vein recognition are employed in a wide range of applications from logging on to a computer or mobile device to national identification systems¹⁾. All of these modalities are relatively new — having only been made possible by advanced electronic technology. Fingerprint recognition, on the other hand, has a long and storied history, stretching back more than a century. NEC first got involved in fingerprint recognition R&D nearly 50 years ago and has developed and deployed a wide variety of solutions since then. It is worth noting that our first fingerprint identification solution was developed for use in criminal investigations.

Before the introduction of computerization to fingerprint matching, the process was a long and laborious one in which all collected fingerprints had to be visually compared with the reference pattern to see if they matched. Computerization made the process much more efficient, significantly reducing the time required for individual identification, thereby enhancing the effectiveness of criminal investigations. However, the quality of the latent fingerprints left behind at crime scenes remains a problem. In many cases, only partial prints can be obtained and these are often further corrupted by background noise as shown in Fig. 1. Finding a way to match fingerprints with high accuracy using limited information is essential.

Improving the accuracy of latent fingerprint matching has been NEC’s central focus since entering the field 50 years ago. In this paper, we will review history of NEC’s research into latent fingerprint matching technology and introduce our current effort to improve matching accuracy using a new feature amount.

2. History of Research into Latent Fingerprint Matching

2.1 Establishment of automatic matching technology (1970s–1980s)

NEC started its research in fingerprint matching in 1971 and developed a system using the Minutiae and Related Method. Minutiae are the major features of a fingerprint image. In addition to minutiae location and orientation, this system incorporated ridge-count information present in the local four surrounding quadrants of each minutiae under consideration for pairing. This technique made it possible to overcome the inability of conventional fingerprint matching to cope with substantial distortion in latent fingerprints. Building on this technique, NEC introduced its first automated fingerprint matching system in 1983. Combining the minutiae and relation data with feature sets called zones that divide the fingerprint in two parts — the part where the central axis indicating the fingerprint’s central position and ridges is clear and the part where it is unclear, this system established the basic feature amount as shown in Fig. 2, which would serve as the basis of NEC’s fingerprint matching system.

2.2 Improvement of accuracy and palmprint matching (1990s)

Moving into the 1990s, we began working on improving the accuracy of an automatic fingerprint feature extraction system and developing techniques for latent palmprint matching. Earlier feature extraction systems often missed the more minute feature points (minutiae), but our new system made it possible to extract them with high accuracy. Palmprint matching posed a different set of challenges. Despite the apparent similarity between palmprints and fingerprints, they are actually quite different. A palmprint has more wrinkles than a fingerprint; it is often difficult to automatically extract features as is as shown in Fig. 3. This meant that we had to develop a new feature extraction system that could better manage wrinkles. Another difference is that the center of the finger provides a clear reference point, making positioning much less difficult. Palmprints have no such reference point. Moreover, the palm is much larger than the tip of a finger. These factors made it necessary for us to develop new positioning technology, which would provide the foundation for a palmprint matching system.

2.3 Automated latent fingerprint matching

By the 2000s, research into automated latent fingerprint matching was accelerating thanks to the rapidly improving performance of computers. Until then, excessive noise and blurring had made it difficult to automatically extract features from latent fingerprints and match them. As a rule, minutiae such as ridge endings and bifurcations had to be entered manually by latent print examiners before the prints could be processed. Replacement of this painstaking and time-consuming process with automated matching was a matter of considerable urgency. In response, we developed noise reduction technology²⁾ as shown in Fig. 4 and new feature extraction and matching technologies using multiple algorithms called fusion matching technology. We also began working on a system that would offer even greater accuracy by combining the new technologies. Ultimately, we were successful, and our new system was ranked first in the Evaluation of Latent Fingerprint Technologies 2017 (ELFT07)³⁾ conducted by the United States National Institute of Standards and Technology (NIST).

2.4 Putting our automated latent fingerprint matching technology into practical use and developing a new feature amount (2010s)

In the second decade of the 21st century, our research focused on developing a system that would bring high accuracy and high performance into alignment. Up until the 2000s, the focus had been entirely on improving accuracy. Unfortunately, this meant sacrificing speed. Automated latent fingerprint matching systems were generally so slow that it was difficult to put them into practical use. One of the factors that caused the slow matching speed was the multiple fusion method used to improve accuracy, which combined hundreds of thousands of different algorithms in an effort to ensure precise, error-free matching. To increase speed, it was necessary to reduce the number of algorithms without compromising accuracy. The route we took was to combine multiple feature sets into a single feature set and to develop a high-speed system that could perform each match itself. This allowed to develop a system able to operate at speeds comparable to conventional matching speed while maintaining high accuracy. At the same, the limitations of matching systems that used conventional feature sets began to emerge. Starting the mid-2010s, we started trials to define a new feature set that would make it possible to match fingerprints that were normally difficult to match.

3. Our Efforts to Improve the Matching Accuracy Using the New Feature Set

3.1 Rich relation

When latent print examiners examine the relationships between paired minutiae in detail, they track ridges and compare the number of the ridges in most cases. Simulating this task with a conventional relation feature amount was insufficient, so we expanded the relation feature amount and defined a new feature amount. This is what we call the rich relation.

(1) Contour ridge-count now included

With respect to the count of ridges between minutiae, we also included the count of contour ridges as part of the definition, in addition to the count of intersections with straight lines as shown in Fig. 5. The contour ridge-count can be interpreted as the height difference between two minutiae assuming the mountain model generated by fingerprint ridges (assuming the fingerprint core/center as the mountain peak).We also included the directions of child minutiae when traced from parent minutiae in the new feature set. By adding the number of contour ridges, it is now possible to compare high curvature regions and areas with significant elastic deformation.

(2) Introduction of connection types

We also introduced connection types as shown in Fig. 6. These newly defined connection types are used when a single ridge connects two minutiae. They are useful for classification of how minutiae connect with each other and can be used for matching and comparison of minutiae. This is also expected to help improve accuracy.

(3) Expansion of the number of relations

The number of relations in the conventional method is small. It has only the total of 4 — 1 in the nearest neighbor in each quadrant. When there are many false minutiae, many undefined cases occur, resulting in the deterioration in the accuracy. In the rich relation, this number has been expanded to 32 in the nearest neighbor.

3.2 Secondary direction of minutiae

It is often difficult to reliably discern minutiae that exist in high curvature regions. To solve this problem, we have defined a secondary direction for minutiae as shown in Fig. 7.

3.3 Loop top spur and delta dot

Although there are no minutiae in the looped skeleton (the innermost ridge) and delta region, they can still provide a very effective feature set for use in crime scene investigations. As these configurations are susceptible to minor factors and likely to generate different extraction results depending on the condition, they cannot normally provide a stable feature set. However, by generating pseudo-short ridges at the top of a loop and in the delta region as shown in Fig. 8, we succeeded in making it possible to use them to create a stable feature set.

3.4 Dubious zones

There are often cases where latent print examiners wonder whether minutiae in a particular zone should be specified or not when they enter latent fingerprints. For areas such as these, we defined dubious zones as a new feature amount, making it possible to enter questionable minutiae.

3.5 Skeleton

Skeletons (ridges converted to thin lines) can be treated as a feature set, and the relationships between them can be compared. We call this procedure skeleton matching. In skeleton matching, paired minutiae are first determined using minutiae matching. Then minutiae that have the same type of ridge ending and bifurcation are searched. These minutiae are then paired with previously paired minutiae. As minutiae propagated in this manner, their validity is verified while any distortion is absorbed.

4. Future Prospects

Automation, high speed, and high accuracy are now expected from any latent fingerprint matching technology used in criminal investigations. In the coming years, applicable databases will be constantly expanded, increasing demand for matching technology with ever greater speed and accuracy. One way of meeting this demand will be through the incorporation of AI technology, which has seen dramatic advances in recent years, with deep learning especially showing spectacular results. By skillfully integrating new technology such as AI with the powerful technology we have developed thus far, we expect to produce faster, more accurate matching technology that will help achieve our dream of a world that is truly safe and secure.

Reference

Authors' Profiles

Related URL