Global Site

Displaying present location in the site.

“My Number” Collection Service Utilizes Several Key Image Recognition Technologies

In January 2016, Japan introduced the Social Security and Tax Number System. Since then, all residents have been required to submit the number issued to them - nicknamed “My Number” - for all tax and social insurance procedures. One consequence of this is that employers must now collect the “My Numbers” of their employees and dependent family members. To address this, NEC Nexus Solutions has developed a smartphone app (for iPhone and Android) to collect “My Numbers.” This app takes advantage of our various image recognition technologies to enable “My Number” holders to handle their numbers more easily and more accurately by using their smart devices to read their numbers. This paper discusses what can be accomplished with our image recognition technologies today and what we can expect to accomplish in the future.

1.Introduction

In Japan, it is now mandatory for employers to collect the so-called “My Numbers” issued to their employees as part of the country’s new Social Security and Tax Number System. This has prompted a variety of companies to begin offering agent services to facilitate collection of “My Numbers,” including NEC Nexus Solutions (hereinafter referred to as NEXS). We offer a collection agent service called “My Number” BPO Service, which uses conventional mail and smart devices to collect “My Numbers.”

The smart device technology used to collect “My Numbers” is described below.

2.Collection Using Smart Devices

2.1 Collection Mechanism

The standard mechanism to collect “My Numbers” is to have users use their smart devices’ cameras to photograph the notification cards or individual number cards - on which their “My Numbers” are printed - and transmit the photographed images for registration. Likewise, identity verification documents such as driver’s licenses can be transmitted in the same manner when and as required (Fig. 1).

Fig. 1 Collection of “My Numbers.”

Utilization of smart devices makes it much easier to submit “My Numbers” than physically mailing documents and photocopies through the post.

2.2 Collection Issues

While it is easy to collect “My Numbers” by photographing, there are a number of issues that need to be dealt with.

  • (1)

    Imaging failure

    In some cases the “My Number” may be illegible due to camera shake, poor focus, or incorrect framing.
  • (2)

    Wrong card/person

    Another potential problem is that an image of the wrong card may be transmitted by mistake. Similarly, when sending pictures of dependent family members, an image of the wrong person could be sent mistakenly.
  • (3)

    Converting “My Numbers” into processable data

    If “My Numbers” remain in the form of image data, they cannot be processed by IT systems for application to such procedures as form entry and printing. Consequently the image data must be translated into usable text data.

    One way to solve this would be to have employees enter their “My Numbers” when they take pictures of their cards; however, in this case there is always the possibility of erroneous entry.
  • (4)
    Data leakage

    Imaged cards are sent to collection databases via network, leaving open the possibility that unauthorized persons could intercept the data - for example, when a public wireless LAN used by a large number of people is used.

    Also if the “My Number” in a registered image is illegible or somebody else’s number has been sent, the data will have to be collected again, inconveniencing both employer and employee.

    “My Numbers” are highly confidential information and therefore require comprehensive security measures (Fig. 2).
Fig. 2 Collection issues.

3.Utilization of Image Recognition Technologies

In collaboration with NEC Central Research Laboratories, NEXS is conducting validation tests on several image recognition technologies. These include face recognition technology and object fingerprint authentication, to name just two. For the collection of “My Numbers,” we are utilizing object identification technology and OCR* (Table).

Table Utilization of image recognition technologies to solve problems with “My Number” collection.

3.1 Object Identification

This technology recognizes the identity and the location of objects in an image by comparing it with images of pre-registered objects (Fig. 3). Since the objects themselves are recognized - unlike barcodes and IC tags where the information for recognition is added to the objects - this technology offers benefits such as fewer design restrictions, as well as enabling recognition without having to modify existing objects.

Fig. 3 Recognition of objects using the object identification technology.

The object identification technology uses NEC’s originally developed BRIGHT feature1), a compact local feature that describes the feature of an image patch centered around distinctive points (feature points) in an image. BRIGHT features are extracted from the feature points detected in the photographed image and then matched with the local features in the images of objects registered in the master database. The location and orientation of the object in the photographed image are accurately calculated based on the correspondences between the local features in the images (Fig. 4). Since the BRIGHT feature is invariant to scale, orientation, and brightness of the object captured in the image, recognition is still possible even when the shooting angle, size, and lighting condition are different from those of the registered image or even when part of the object is occluded. Moreover, the data size of the BRIGHT feature is less than one tenth of that of a conventional local feature, which enables high-speed matching using a mobile device such as a smartphone.

Fig. 4 Matching using the BRIGHT feature.

3.2 OCR

The types of characters contained in the regions specified in the image are identified (Fig. 5). First, the direction information of the character lines in the photographed image is extracted as features. The feature vectors in multiple dimensions are then calculated, followed by calculation of the distance between the feature vectors in the photographed image and the feature vectors (templates) of each character type registered in the dictionary. The character type with the smallest distance is used as the recognition results.

Fig. 5 Identification processing of character types using OCR.

When the card is imaged with a scanner, a clear image without distortion can be obtained. Images input in this manner are easy to recognize. However, when the card is photographed with a device like a smartphone, the angle between the card and device may distort the characters. Image quality or focus may also be a problem, depending on the performance and capabilities of the device used to photograph the card. To address these issues, we have used the following techniques to improve performance. First, in preprocessing, we have made it easier to recognize obliquely distorted characters by using object position and orientation information obtained from object identification to correct the distortion (Fig. 5 (a)). Moreover, to increase the reliability and usability of templates, we artificially generated a wide variety of images including ones that were out-of-focus ones to increase the system’s ability to recognize characters despite inferior image reproduction (Fig. 5 (b)).

3.3 Utilization of Recognition Technologies

Now, let’s take a look at how we utilize the object identification and OCR technologies described above.

  • (1)
    Countermeasures against camera-shake and blurriness
    If significant camera-shake or blurriness occurs when a notification card or “My Number” card is photographed, the object identification technology will be unable to recognize the card. This makes it possible to check whether the card in question has been satisfactorily photographed or not (Fig. 6).
Fig. 6 Successful and unsuccessful imaging.
  • (2)
    Countermeasures against out-of-frame cards
    It is also possible that the card does not fit in the frame because the picture was taken from too close or the framing is too far to the left or right. The object identification technology makes it possible to recognize where the object is in the photographed image. Similarly, utilization of this capability makes it possible to check whether the object is out of frame or not (Fig. 6).
  • (3)
    Confirmation of objects that have been photographed
    There may also be cases where another card was photographed, instead of the one the photographer intended to shoot. In this case too, the object identification technology will be unable to recognize the card. Here again, this inability makes it possible to check whether the correct card has been photographed or not (Fig. 6).
  • (4)
    Turning “My Numbers” into data

    The “My Numbers” printed on the cards are turned into data with OCR. This makes it possible to use “My Numbers” in IT systems without entering them manually. OCR is not applied to the entire picture; instead, the object identification technology first identifies the card type and the number position to determine the target region of the OCR. Elimination of unnecessary data beforehand facilitates a more accurate reading (Fig. 7).
Fig.7　Turning “My Numbers” into data.

4.Security

“My Numbers” are highly confidential information and require appropriate handling. In Japan’s Act on the Use of Numbers to Identify a Specific Individual in Administrative Procedures2)3)4), there are stipulations regarding the provision and protection of specific personal information such as “My Numbers.” This law carries more severe penalties than Act on the Protection of Personal Information. This means that security is a critical consideration when handling “My Numbers.” Consequently, any information regarding “My Numbers” such as photographed images and recognized numbers should be handled only in memory and promptly erased as soon as it is no longer required (Fig. 8).

Fig. 8 Security measures.

5.Future Prospect

In addition to “My Number” collection, we plan to develop and offer a variety of services that take advantage of these technologies which we believed will be suitable for application in a wide range of business and administrative settings. For example, when a customer presents their card in order to open a bank account or obtain a membership card, the sales associate will no longer have to take the card to the back of the office to make a photocopy of it. Instead, the clerk can take picture of the ID card right in front of the customer and turn it into data. This will help assuage the discomfort of people who are uneasy about letting their ID cards be taken away even temporarily.

6.Conclusion

In this paper, we have shown how we can significantly reduce the cost of “My Number” collection by applying a system that integrates object recognition technology and OCR technology to quickly collect imaged “My Numbers” and turn them into usable data. We have also pointed to a number of other potential applications for these technologies. In cooperation with NEC, NEXS is committed to providing services that take advantage of leading-edge technologies to help build a society where the values of safety, security, efficiency, and equality are achieved.

  • *
    iPhone and iPad are trademarks of Apple Inc. The iPhone trademark is used under license from AIPHONE CO., LTD.
  • *
    Android is a trademark and/or a registered trademark of Google Inc.
  • *
    All other company names and product names that appear in this paper are trademarks or registered trademarks of their respective companies.

Reference

1) K. Iwamoto, R. Mase, and T. Nomura: BRIGHT: A Scalable and Compact Binary Descriptor for Low-Latency and High Accuracy Object Identification, Proc. of ICIP2013, pp.2915-2919, 2013.

Authors’ Profiles

UDAGAWA Hiroyuki
General Manager
1st Service Sales Division
NEC Nexsolutions, Ltd.

MASUDA Takashi
Assistant Manager
4th Systems Division

IWAMOTO Kota
Principal Researcher
Data Science Research Laboratories

AKIYAMA Tatsuo
Senior Researcher
Data Science Research Laboratories