This paper provides an overview of how we can utilize generative AI in software and system development and introduces the initiatives of the NEC Group. Generative AI has great potential, but it currently has several technical challenges. Therefore, software and system development remains, for the present, a process that must be primarily performed by people. However, depending on the nature of the development project, generative AI can be used to streamline human work in a variety of tasks and to improve quality. The NEC Group plans to improve productivity and quality in software and system development by systematically using generative AI.

1. Introduction

Generative AI opens up the possibility of mechanizing a variety of software and system development tasks that have previously relied on humans performing them. The development and maintenance of large-scale software and systems involve many people, and system requirements and designs are mainly saved in natural language formats. Traditionally, the ability of computers to interpret and generate natural languages has been extremely limited, making it difficult to mechanize development tasks directly. Techniques for saving design information in machine-readable formats have also been proposed but have been difficult for many engineers to use. The advent of generative AI that can interpret natural languages as they are and that can generate natural language and programs is expected to significantly change the methods of software and system development.

On the other hand, because technical challenges remain in automatically synthesizing large-scale systems using generative AI, software and system development will continue to be, for the present, a process that must be primarily performed by people. However, depending on the nature of the development project, the use of generative AI makes it possible to streamline the work done by people in a variety of tasks and improve quality.

This paper provides an overview of how much generative AI can streamline software and system development and introduces how the utilization of generative AI is promoted within the NEC Group.

2. Generative AI and Software and System Development

2.1 Challenges in applying generative AI to software and system development

Generative AI has great potential, but the large language models (LLMs) that make it possible have the following four challenges:

2.2 Countermeasures against challenges

The software development process incorporating generative AI needs to avoid or alleviate the aforementioned challenges. The table shows workarounds to avoid and alleviate respective challenges. Measures shown in parentheses are still in the stage of technical development.

Table Challenges and countermeasures in using generative AI.

2.2.1 Hallucinations

Generative AI makes mistake as humans do, so quality control must be exercised through reviews and tests as usual. Using generative AI for reviews instead of using it to generate design documents or code is one of the effective workarounds. This is because the probability and impact of overlooking mistakes in generated review results are smaller than those of overlooking mistakes in generated code. A lot of research is also being done on automatically checking the generated output. In particular, generating code is a task for which countermeasures against hallucinations can be easily taken, because the generated output can be automatically checked by combining the results with the unit tests²⁾.

2.2.2 Limitations on amounts of input data

Tasks in downstream processes^* (internal designing, coding, and unit tests at the module level) that can be addressed with little prerequisite knowledge are less affected by the limitations on the amounts of input data, so it is easy to directly use generative AI for those tasks. However, for designing and integration testing that are further upstream, a variety of requirements and constraints throughout the system must be considered, and limitations on the amount of input data affect these tasks. In addition, conversion development needs to be performed while considering information on the converted base and is therefore affected by the limitations on the amount of input data. Also, in these cases, generative AI may be used to divide the tasks and reduce the prerequisite knowledge required for performing individual subtasks. Methods of automating this division process are also being actively researched³⁾.

2.2.3 Lack of inference capability

If complex and lengthy questions are broken down into simpler ones, generative AI can more easily produce correct answers. A variety of methods of prompt engineering have been suggested to raise the inference capability, and methods to divide tasks and call external programs to have them perform the processes that generative AI is not good at (calculations, etc.) are being put to practical use⁴⁾.

2.2.4 Lack of expertise

Tasks in downstream processes are highly dependent on the programming language and platform, so you should check if the LLM has knowledge about the programming language or platform before using it. The most likely approach to address this challenge is to create a dedicated LLM by combining the original LLM with search capability or additional training.

3. NEC Group’s Initiatives

The NEC Group has several initiatives to address the challenges set forth in the table.

3.1 Development of a toolchain

3.1.1 Introduction of dedicated services

Improved productivity is expected through the use of generative AI services that link with the integrated development environment (IDE) in downstream processes where generative AI is easy to utilize. At the NEC Group, we have established a cloud-based software development platform as our standard in-house development environment for software developers. This development platform is an IDE that includes information management tools to manage the source code, specifications, and other design information; tools to manage and automate a variety of tasks; and a development work environment for implementation and testing. We have started to provide a commercial service for code generation using generative AI on this cloud-based software development platform.

3.1.2 IDevelopment of peripheral tools

Because many SI projects have written specifications created in Excel, we are promoting the use of generative AI by providing a service to convert Excel files to Markdown format, which is suitable for input into the LLM.

3.2 Generative AI-based development process

3.2.1 Preparation of development guides

We are proceeding with the preparation and release of generative AI-based development guides for the countermeasures set forth in the table in order of feasibility with the current toolset (excluding technical measures shown in parentheses in the table). Until the hallucination countermeasures become sophisticated, generative AI needs to be handled as a tool to assist people in their work rather than as an automation tool. As a result, the development process itself will not dramatically change from before, but the fact that written specifications will be read not only by people but also by generative AI is a significant change. By preparing design documents in a format that is readily processible by generative AI and standardizing the method, we will improve the support effectiveness of tools and processes.

3.2.2 Application to modernization

Modernization is a field where there are high expectations for improved efficiency through the use of generative AI. However, current generative AI has limitations as stated in section 2. We cannot currently expect it to be a silver bullet that solves the challenges to modernization all at once. Therefore, the main use of generative AI is to streamline some tasks in downstream processes (Fig. 1).

Code conversion using generative AI can be done with greater flexibility thanks to its ability to follow changes in or of software architecture and with lower initial costs thanks to its ability to reduce workhours for procurement and implementation of tools or for customization per project when compared to conventional dedicated conversion tools. On the other hand, accurate conversion is difficult to achieve with current technologies⁵⁾ of generative AI and requires human assistance to check and modify results or to divide tasks. This means different methods must be used in accordance with the characteristics of the project. By identifying these characteristics and identifying more and more AI practices, we will promote the use of generative AI for modernization.

3.3 Development of dedicated LLM

With our belief that using a dedicated LLM to address the lack of expertise is especially important among the countermeasures shown in parentheses in the table, we are proceeding with its in-house development.

The development of the business system is changing from building systems without anything prebuilt to combining XaaS (anything as a service), packaged software, and other tools for smart integration. With this method, the proportion of workhours for tasks in downstream processes to overall workhours is small, and requirement definition and designing are main tasks. Performing these tasks requires knowledge and know-how of XaaS and the packaged software to be used as well as business knowledge. In this field, as mentioned in section 2, challenges to current LLMs, including limitations on the amount of input data, lack of inference capability, and lack of expertise, become apparent.

However, business departments responsible for smart integration can turn knowledge and know-how of businesses and packages into an asset as a knowledge base to streamline and improve the efficiency and quality of work done by the development staff. Creating a dedicated LLM incorporating this asset is expected to produce the following effects (Fig. 2):

LLMs and their peripheral technologies are rapidly developing, and the aforementioned countermeasures that are currently being developed are expected to become widely available in the future. Advances may soon be made in countermeasures against hallucinations in downstream processes, and the development will be conducted jointly by people and AI, rather than AI assisting people. As the value provided by vendor companies like NEC is expected to shift upstream more than ever before, we believe that maintaining and strengthening dedicated LLMs with domain specialization in industries and operations is an important role for the technology departments common to all vendor companies.

4. Conclusion

This paper has provided an overview of the use of generative AI in software and system development and presented the NEC Group’s initiatives. Generative AI has great potential but the development of software and systems continues to be primarily performed by skilled people for the time being because of the vulnerabilities generative AI has at the moment. Depending on the nature of the development project, generative AI can nonetheless streamline the work done by people in a variety of tasks and improve quality. The NEC Group strives to improve productivity and quality in software and system development through systematic use of generative AI.

As the value provided by vendor companies is expected to shift upstream more than ever before due to advances in generative AI technology, we plan to proceed with the strengthening of a dedicated LLM with domain specialization in industries and operations.

References

Author’s Profiles