May 22, 2024

Chat GPT and other large language models (LLMs) are garnering attention in recent years. NEC recently developed a security risk assessment technology using LLM. This technology allows simple yet efficient implementation of security by security experts and non-experts alike. We spoke with the researchers about the details of this technology.

Achieving a system's risk diagnosis to action planning through dialogue with LLM

――What kind of technology is the security risk diagnostics using LLM?

Ueda: This technology automatically analyzes the risks of the target system and explains it to users in an easy-to-understand manner through LLM. You don’t need to be a security expert to understand the system’s potential risks and learn about effective actions and responses through having a dialogue with LLM.

NEC already had a proprietary technology that automatically analyzes attack and intrusion routes to the system and diagnoses their risks. This technology has already been commercialized as our “cyberattack route diagnostics service.” This technology has been adopted primarily by customers that handle important infrastructure such as chemical plants, but it required security experts to operate this technology in order to provide the service.

At the same time, a concern for security of the entire supply chain is emerging in recent years. Stricter regulations and requirements, include, for example, a compulsory security risk management and inspection at equipment installation applicable to enterprises in the supply chain of key infrastructure. This means that even SMEs that do not have enough human resource for security are required to take measures.

Our new technology responds to the needs of such smaller enterprises. The application of LLM expands the scope of cyberattack route diagnostics service offered by NEC’s proprietary technology―even without specialized security knowledge, users can more easily utilize this service. Of course, this service will also benefit large enterprises that have security experts. Security tasks are intensive, so we are also envisioning it as a tool to reduce the burden of operation.

――While there are many other approaches in the world that use LLMs for security, what distinguishes NEC’s technology from others?

Ueda: Our new technology can detect risks in advance. As I explained earlier, the automatic analysis of intrusion routes into the system is a unique feature that is not seen in other companies' technologies. In contrast, most other approaches focus on the responses to security incidents, or actions taken after incidents occur. Their system analyzes a massive amount of log information with LLM and presents the problem to users. To put simply, NEC's new technology is like a "medical checkup" while many other technologies are focusing on "emergency surgery."

An LLM that responses to users and gives answers in an easy-to-understand manner

――Where there any special concepts for applying the LLM?

Mizushima: Standard LLMs return answers to questions without clearly explaining the reasons behind them. Therefore, there is always a certain amount of doubt about whether that answer is correct or not. Since our new technology is to be incorporated into security tasks, it needs to present reliability and persuasive reasoning along with the answers.

While we were able to achieve reliability based on the cyberattack route diagnostics service, the problem was how to have the LLM explain that diagnosis to the users. Essentially, this service is achieved through a process of a specialist interviewing the customer for analysis and organizing a report of the results in such a way that is easy to understand by the customer. It is the specialist’s task to interpret network diagrams and other various information and verbalize them. The new technology has been tuned through many trials and errors so that the LLM can output such sophisticated explanations. As a result, it can output reports with a quality level comparable to those made by specialists.

Another key point of this new technology is that it can produce diagnoses more efficiently. The lead time for report output has been reduced by almost 60% by this technology compared to the time required for a specialist to analyze information and make a report.

Kishimoto: Risks change day by day due to the detection of new vulnerabilities and configuration changes by users. It is important to periodically perform security diagnoses in detail without too much time in between. To that end, it is very meaningful to shorten the time required for making a report and simplifying the process.

Additionally, it is more important for reports to focus on points of notice and present them in a user-friendly manner rather than covering the results of the security diagnosis in detail. This is something that specialists do when they make reports on a regular basis. Only after appropriately abstracting and selecting information instead of simply listing a comparison of risk values or detailed network topology, the reports can be easily understood by users.

Such know-how and implicit knowledge are reflected into the algorithm while having discussions with Mr. Mizushima, who has experience providing services face-to-face with customers. We have implemented the technology with a view to effectively visualize key points at a glance, including where attacks arise and where they are targeting, as well as areas of danger. This enables even users who are not familiar with security produce reports comparable to those made by experts.


Ueda: In addition, points that are worth noting are different depending on the industry, such as chemical, finance, and healthcare. What aspects are important and need resources depend on the security guidelines published by the respective ministries and agencies, so we are considering analyses and reporting that conform to the individual guidelines.

Integrating NEC's know-how and security techniques into the LLM

――How do you envision the future progress of this technology?

Ueda: Currently, we just started using this technology in NEC also as an experiment. We are working on research with the aim to complete it during 2024 while expanding functions as we observe the situation.


Mizushima: We believe that there is room for expansion in the features of the cyberattack route diagnostics service itself. Currently, this service targets the automation of the entire process from analysis to reporting. Information collection and other preparatory tasks for analysis still require specialists to take their time and efforts. On the other hand, if the use of LLM can automate information collection and other pre-analysis tasks, it can achieve further improved efficiency. It enables specialists to focus on other tasks such as setting conditions for analysis, and customers to effectively use their internal resources. We may even be able to create a mechanism where the LLM supports that conditioning as well.


Kishimoto: Yes. Yet, we are still left with the problem that 100% security cannot be achieved even by taking every measure possible based on the risks identified. In that sense, it is important to ensure complete compliance to win the trust of customers regarding security. Compliance with the applicable regulations and guidelines of the specific industry is a valid standard. We are envisioning a system that can effectively check a list of such criteria and incorporating a function that clearly and concisely explains why such criteria are necessary into the LLM.


Ueda: While we combined the cyberattack route diagnostics service with an LLM in this technology, NEC has other various proprietary technologies that can also be applied. One example is the tool that checks for potential errors in the system design. We also have ideas to integrate such tools into the LLM, creating a system that can give user-friendly explanations to customers. In the future, we hope to develop LLMs that can give advice to customers just like security experts with a wealth of NEC’s know-how and techniques working alongside customers.

Go back to Featured Technologies