March 26, 2021

Detecting "Sound" to Realize a Safe and Secure Society

Data Science Research Laboratories
Sakiko Mishima

After receiving a master's degree in engineering, Mishima joined NEC in April 2018. She has specialized in acoustic signal processing since her student days and is now also developing that research at NEC where she is engaged in research and demonstration experiments. Mishima participated as a member of the Organizing Committee of the "DCASE2020 Workshop," an international conference hosted in FY 2020 which focuses on the detection and classification of acoustic scenes and events.

Acoustic event detection technology which supports automation and manpower reduction

I am currently researching acoustic event detection technologies at the Data Science Research Laboratories. Within that research, our group is focused on understanding the status of moving objects to understand the surrounding environment from chronological signals such as the sounds and vibrations of moving objects. Because it is easy for noise to get mixed into the sounds of cars, trains, and other moving objects, it is important to utilize signal processing and machine learning to accurately detect the target sound and cumulatively develop and improve technologies with a higher degree of accuracy.
Most recently, I have been involved in the development of an algorithm which can detect and identify impulsive sounds, which can only be observed for a very short period of time, with high accuracy. This algorithm detects slight sounds which occur during the operation of machinery or equipment to identify the factors which are generating the sounds. Moreover, I am also conducting technology research to detect component anomalies from the vibration information obtained with an acceleration sensor. I am now struggling to expand the scope of application of technologies related to acoustic event detection to other types of sensors as well.
Understanding the state of a moving object is a type of technology which will lead to the early detection of malfunctions and anomalies. There was a serious incident several years ago in which a crack was discovered in the bogie of a Shinkansen high speed train, but if we could predictively detect such a situation from unusual noises, it would ensure the safety and security of many users. In addition, it would have significant advantages in terms of maintenance as well. Conventional maintenance and inspection work has relied on experts with extensive knowledge and experience. However, in recent years there have been concerns about a lack of manpower due to the rapidly aging population. Anomaly detection through automation will become important in order to cover such a situation. In addition, it will certainly become an essential technology required for the detection of vehicle body anomalies and conditions inside of vehicles such as self-driving buses in the future. Our team is now conducting research and development and demonstration experiments on a daily basis in anticipation of such a future.

Simultaneously participating in academia while working at a company

I first became interested in NEC when I participated in an information processing conference. Amidst lectures by various universities and companies, the NEC lecturer gave a presentation that was extremely easy to understand and accurately communicated the technological values. When it comes to top-level research, there is a tendency for the technological values to become difficult to understand, but I felt that NEC was making an effort to broadly communicate the values and effects of their own technologies. If a company can communicate in such a way, then they should be able to broadly communicate the results of cutting-edge research to society and deploy businesses and services with a greater utility value. So I became interested in NEC based on that thinking.
Moreover, NEC is a company with a long history in the field of acoustics. The company has been developing noise processing algorithms from the time when it was founded to manufacture telephone switchboards and has continued to cultivate new and unique technologies such as voice authentication derived from speech recognition and acoustic event detection. As a result, the company possesses valuable and abundant know-how that is not found in other research laboratories.
NEC also has a significant presence in academia, and the ability to engage in genuine research that is not only limited to development may also be appealing. In fact, to use conferences as an example, you can freely participate even just to attend discussions, and the company is also proactive about presenting research papers. I think that it is the perfect situation for people who would like to go out into the world and conduct applied research but also continue doing academic research.

An environment that is blessed with customers

I have been researching acoustic signal processing since my student days, so my field of research may not have significantly changed since that time. However, what has definitely changed is that I now have the opportunity to engage in direct dialogue with the customers implementing my research. As a result, I am able to clearly conceptualize how my technologies will be useful to society. This has led to a great sense of purpose.
Because data science depends on the ability to acquire as much live data as possible, our customers are important. In that respect, NEC is a major enterprise which has conducted business for many years in a variety of fields. The company has already built trust relationships with many customers. I feel that it is an environment which is blessed with customers who even suggest, "Why don't we acquire the data together?"
Moreover, in research such as ours, we believe that the attitude of "let us first try it and see what happens" is important. First, we propose a prototype, and then it is important to further polish it through discussion between the team members and the customers. During my student days, I tended to be the type of person who would think things over by myself. However, since joining NEC, I have become aware of a style of working in which I first try to flesh out an idea if I think of something and then solicit the opinions of people around me while improving the idea. Part of the reason is that I am surrounded by many colleagues and senior researchers that I can easily ask for their opinions, so at some point I started to think that it would be a waste not to take advantage of such an environment. In fact, there is a tendency to get stuck on one aspect when you are thinking about something by yourself, but receiving opinions from different perspectives generates creative ideas and also increases the speed of development. The fact that there are many researchers may also be part of NEC's appeal.
Acoustic event detection is a technology which is useful for achieving the safety and security of society. Going forward, I hope that through my research I can contribute in even a small way to creating a safe and secure society.

A day at work

Message to my past self in my school days