April 2015

1. Analyzing video from cameras in groups rather than individually

2. Analysis system closer to human capabilities born out of NEC know-how

—What was the impetus behind developing this world's first technology, and what were the breakthroughs?

Ikeda: During the initial stage of development, we kept repeating experiments while engaging in trial and error in an attempt to analyze individuals, but it didn't go well. While watching videos over and over again, I thought, "Wouldn't it be better to look at patterns created by human distribution patterns in small areas?" This was the inception of the idea.

Miyano: This is actually the same as what we humans do, right? When we attempt to count people in a short period of time, we don't count each person, we subconsciously pick out groupings to count.

Ikeda: Based on this concept of patterns created from human distribution patterns, we developed a proprietary analysis engine that separates videos into grids. We prepared hundreds of thousands of sample images showing various human distribution patterns in the same shape as the grid and had the engine learn them. By doing this, the engine is able to efficiently determine, for example, that there are three people within the grid when the image inside the grid is similar to one of the learning samples that shows three people.

Miyano: Another breakthrough is that these hundreds of thousands of images are created artificially. We take images of individuals and paste them together to create them. If you try to actually shoot videos of groups, they follow the same patterns of movement, and this creates a problem in that you only get a narrow set of data. However, if you can create them artificially, you can create various cases with random patterns, enabling analysis no matter what the situation is. Moreover, the learning technique for creating a high accuracy analysis method from hundreds of thousands of combined images is also important. This is where NEC's know-how from being a world leader in OCR analysis since the 1960s comes into play.

3. Achieving a more secure society through accurate alerts and predictions

—How is crowd behavior analysis used in society?

Miyano: The first thing we can say is that it can make security more efficient. Recently, many live cameras have been installed in various places in public facilities and cities. However, at the same time, this increases the number of images to be monitored. For that reason, in recent years there has been a problem in that the burden on the person checking the videos has grown. Crowd behavior analysis is a system that can analyze abnormalities accurately and in real time and create an alert. That means that security personnel can quickly catch abnormal situations from many different videos based on reliable alerts.

Ikeda: On the screen, the correlation with the actual location is shown with a heat map display, so the location where the abnormality occurred can be seen at a glance. They can quickly rush to the scene. This will enable more efficient and reliable security.

Also, the issue of privacy is always there with live cameras, but with crowd behavior analysis, individuals are not identified. Instead, they are captured as part of the crowd, ensuring privacy, and that's a big advantage.

Miyano: We also believe that implementations can leverage big data. When you record data over a long period of time, you can predict what situation will occur next and what risks will arise based on the current situation of the crowd. This also makes it possible to optimize the deployment of security personnel beforehand.

We believe that using this technology in many places, not only at stations and event venues where congestion occurs easily but also in other places for guidance during disasters can contribute to the creation of a more secure society.

4. Possibilities of expansion through linking with SNS and sales

Miyano: Generally speaking, crowd behavior analysis is a technology for analyzing human gatherings as a group. In conventional video analysis, the primary methodology was to capture each person individually for analysis. However, this is often not very useful out in the field. In places where there are not many people, humans can be recognized individually, but in crowded places, people's bodies and faces overlap on the video, so individual recognition does not really function effectively. However, what is actually required of analysis technology is to function in crowded places. In crowded places, accidents like falls are more likely to occur, and the risk of incidents increases. That is precisely what led us to develop this technology, because we wanted to create analysis technology that would truly be useful even at train stations and public facilities that are often crowded. This crowd behavior analysis is what we developed to respond to the social need for creating a safe and secure environment even in crowded places.

Ikeda: Additionally, this technology has a new feature of focusing on the movement of surrounding people. For example, if someone falls in a crowded place, it is very difficult to pinpoint just that person in a video. This delays the initial response of security personnel as well. However, if a person falls, a group of people is likely to gather around them to protect them. Crowd behavior analysis can analyze such group changes in real time and generate an alert. It was this shift of thinking to focus not on identifying an individual for which to generate a warning but on the people around them that produced this technology.

Go back to Featured Technologies