Contributing towards public safety and comfort through image and video analysis Public Safety Solutions from NEC
March 31, 2017
Airports, seaports, stations, commercial facilities and event venues are places where enormous crowds of people gather together, many people being of unknown identity. When congestion becomes severe, this can lead to unforeseen accidents and other problems. Crime rates also tend to be high in large urban centers. NEC hopes to contribute towards building public spaces with a high level of safety and comfort that will be welcomed by all, by delivering Public Safety Solutions based on NEC's advanced face recognition technologies, which boast a world-class level of analysis accuracy and processing speed.
The international movement of people across borders in a globalizing world
As the cross-border connections between different nations and regions continue to grow deeper in our globalizing world, we are continuing to witness dramatic advances in a whole variety of domains, from politics and economics to culture. The World Bank defines globalization as “Individuals and companies having the freedom and ability to commence trade with citizens of other countries of their own volition." In economic terms, globalization means that the boundaries between domestic markets and overseas markets are gradually disappearing. This trade results in major increases in transactions in goods and services, and greater overseas investment.
But it is not just physical objects and money that move around our world as it globalizes. A growing number of people are also moving to other countries in search of work, in the hope of higher wages or a place where they can achieve success, while companies may also aim to import the labor forces they need from overseas. This “movement of people" across borders is set to become a striking feature of globalization.
According to the figures announced by the United Nations (UN) in 2012, the numbers of migrants (including both those on short-term stays such as tourists and those who stay for longer periods) moving across international borders in member nations of the UN have risen from 150 million in 1990 to 180 million in 2000, reaching 210 million in 2010. As these figures also include political asylum seekers, not all of this increase has occurred as a result of globalization; nevertheless, it is clear that the numbers of people moving across international borders are on an increasing trend.
The limits to human-based video surveillance
At the same time, a new problem is also becoming more and more apparent. This problem is most conspicuous in the immigration controls that are carried out at international border checkpoints, airports and seaports. While the increased movement of people is leading to stronger connections between different countries and regions, it may also be linked to a growing risk of international crime and other problems. The question of how to spot risks as quickly as possible at immigration inspection and ensure safety has become an issue of great importance.
We cannot afford to skimp in the checks carried out at immigration control if we are to ensure safety. It is essential that both individual travelers themselves and their luggage are examined with the strictest care. This means that the staff in charge of immigration inspections will be required to spend more and more time and effort on these tasks. Waiting times will grow longer, which will take its toll on travelers. What is more, places where large numbers of waiting passengers are crowded together are places where crimes and other problems are liable to occur. As a result, additional checks and patrols looking out for suspicious persons and items will also have to be stepped up even in places outside the immigration inspection zones themselves.
There is no quick and simple way to increase the numbers of staff in order to meet these requirements. Trying to secure the appropriate human resources while meeting budget constraints is no easy task. The question of how to create setups which will allow immigration inspections and checks within facilities to be carried out as efficiently as possible is a common problem shared among all countries which are facing the challenge of globalization.
Ensuring safety and security in places where large numbers of people gather together is not only an issue for airports and seaports; it also applies to busy downtown areas in large cities. Surveillance of video footage provided by security cameras installed around a city can be of service in spotting abnormal developments quickly, but trying to ensure that all such footage is monitored with nothing being missed is very difficult. With so many security camera is installed in large-scale facilities, there is no way to ensure that all video footage can be checked in full. Working out when abnormal developments are likely to occur in the future using human judgment alone is also difficult. In other words, there are inevitable limits to countermeasures that are based solely on human capabilities.
Preventing problems before they can occur, by instantly identifying people who appear frequently at a particular location
Information and communication technology (ICT) is set to play a major role in resolving these challenges. As a company which aims to take up the revolution of Social Value Innovator, NEC has now delivered a number of Public Safety Solutions based on NEC's advanced face recognition technologies, which boast a world-class level of analysis accuracy and processing speed. The technologies are able to perform tasks such as picking out a particular person from among large gatherings of people or from huge volumes of video footage, and then tracking only the video footage which shows that particular person. They are also able to analyze human behavior and detect abnormal developments.
For example, when some sort of trouble has occurred any video footage of the incident which remains is likely to be highly competent evidence, but in most cases the footage only becomes useful for dealing with the aftermath of the incident in question. NEC's solutions, by contrast, are able to instantly pick out suspicious persons and detect signs which warn of abnormal developments that are about to occur. By using these technologies to take appropriate responses before problems can develop, we can help prevent incidents and accidents before they have a chance to happen.
One example of this kind of technology is NeoFace Image data mining, an artificial intelligence software that can analyze large volumes of video footage and conduct high-speed searches for any human beings who tend to appear according to a particular pattern. NeoFace, the face recognition technology developed by NEC, is used to group together data based on its level of similarity; this is then combined with "Profiling Across Spatio-Temporal Data," a technology which can conduct high-speed searches for specified patterns. NeoFace, boasting the world's highest level of facial authentication accuracy, won the top position for four consecutive years in benchmark tests for face recognition technologies conducted by the National Institute of Standards and Technology (NIST) (*)
The outstanding feature of NeoFace is its use of n:n facial matching, a technique which allows the technology to search out people who have appeared in a number of different locations. The technology can select a number of faces from large volumes of video footage, grouping them together based on their level of similarity. It then carries out high-speed sorting/searches for individuals, based on particular patterns by which they appear in the footage (time/place/actions etc.). A particular person can then be focused on, and they and any other people accompanying them can be searched for. Patterns considered to be associated with a single individual can be sorted, enabling searches to be conducted based on the times and places where they appear, the number of times they appear and the like.
Traditional face recognition technologies are already capable of sorting and searching for people who appear in a number of different locations, but this has traditionally always required checking everybody who appears in all situations shown in the video footage. The colossal amount of cross-checking required meant that processing took a long time. With the traditional technologies, carrying out n:n matching to identify the same person out of a large volume of video footage in which 10,000 people appeared took more than 40 seconds; NEC's new technology, however, can specify a target in just 1 second.
By allowing us to spot "people who appear frequently in the same location" and "people who have appeared in a number of different locations" this technology is expected to be effective not only in crime-prevention measures but also in speeding up initial responses following an incident, such as when criminal investigations are to be carried out.
The Corporate Games Tokyo 2016 Asia-Pacific, a sports tournament hosted in the Tokyo Bay area on 5 November 2016, served as a test demonstration for an advanced security support system that makes use of this technology. The system carried out integrated surveillance of various video footages taken from cameras installed at several fixed locations around the sports venues, wearable cameras worn by members of staff and cameras embedded in drones, and contributed towards safety and smooth proceedings when managing the tournament.