December 13, 2023

Regulations for overtime work have been applied to larger enterprises since April 2019 and to small- and medium-sized enterprises (SMEs) since April 2020. On the other hand, for certain occupations, this application has been put on hold until April 2024. Physicians are one of them. However, in reality, about 30% of the physicians at university hospitals are estimated to be working more than 80 hours of overtime every month.* Eighty hours of overtime work a month is a level recognized as a “cause of death from overwork” in an industrial accident. Under such grave circumstances, what measures can we take to prepare for the coming application of the legal regulation? The medical industry today has been required to undertake a fundamental working style reform.

NEC newly developed a document creation assistance function based on medical language processing technology. Using technologies, NEC aims to streamline physicians’ operations. We spoke with the researchers about this technology and its functions in detail.

AI assists documentation, one of the huge burdens on physicians

― What kind of technology is the medical language processing technology?

Kubo: The technology that enables computers to process the language we generally write and speak, in contrast to programs and other computer languages, is called natural language processing. The recent emergence of large language models (LLMs) has made it rapidly expand its range of applications in society. Nevertheless, the medical field and the like have a lot of jargon and specialized terminology, and in the actual clinical setting, the medical records written by physicians are full of omissions and abbreviations of words and statements. For this reason, ordinary natural language processing is not capable of handling these fields. Which means that we need a more sophisticated language processing system that supports the terminology and abbreviations in the medical field.

This time, NEC developed an original medical language processing technology, which offers two functions: "electronic medical record documentation assistance" and "medical documentation assistance." Both aim to improve operational efficiency for physicians working long hours. We believe that they will be a solution for complying with the new system under the working style reformation that comes into effect in April 2024.


Tsujikawa: The electronic medical record documentation assistance turns the dialogues between the physician and patient into text data using speech recognition and then uses medical language processing to automatically convert the text data into medical record format as output. We have achieved a speech recognition accuracy that can be put to practice right away with only a little fixing. We estimate that this can reduce up to around 116 hours of work time annually for each physician.

Medical documentation assistance supports the creation of medical documents, including referral letters to other medical institutions, medical certificates for submission to insurance companies, and summaries made upon discharge from hospital. Enter written medical records and it will accurately extract and summarize the course of treatment and other elements and output the information according to each document format. This function is already confirmed to cut documentation time by half based on demonstrative experiments, which equals to an estimate of a 63-hour work time reduction a year per physician.

Kubo: We worked on these technologies through collaborative research with the Tohoku University Hospital. Tohoku University Hospital is one of the institutions that have taken initiative in applying biodesign,^*1 a medical device development process widely adopted in the United States. Under the auspices of such an innovative concept, Tohoku University Hospital and NEC have jointly worked on the project with careful and thorough on-site observation. This consequently revealed the magnitude of the burden of the documentation that can be handled by non-physicians puts on physicians. The new technology was developed as a result of our research based on the estimated benefits of significant improvement that can be achieved by our technology.

Using NEC’s AI technologies and LLM

― What are the technical key points of this solution?

Tsujikawa: First, let’s talk about medical record documentation assistance. As mentioned earlier, this function recognizes the dialogues between the physician and patient using speech recognition. Having said that, speech recognition itself has somewhat long history of development and application in medical settings and is nothing new. Notably, speech recognition has been used in preparing interpretation reports for CT and other radiograms at radiology departments.

The new technology is different from the conventional speech recognition in the sense that the former can create medical records upon recognizing dialogues. Speech recognition used for creating radiogram interpretation reports requires the physician to talk into a machine using a microphone, like creating a voice memo. Therefore, all that was needed was to convert the speech made by a single physician intended for documentation into text. In contrast, the new technology recognizes the natural dialogues between the physician and patient, and extracts the contents necessary for creating medical records. Instead of words that are uttered with the intention of having them documented, this requires recording natural dialogues to generate data out of them, which is technically a step ahead of the conventional speech recognition. The new technology also features a versatility in application, supporting terminologies in a broad range of areas beyond radiology.

In this development, NEC extracted as many as 20 million sentences from medical research papers in order to develop high-accuracy speech recognition of medical terms, and created synthesized speech for teaching AI. At the same time, to make it capable of robustly supporting a variety of voices, the AI is also taught a massive number of different speakers’ voices that speak plain language. This achieved precision speech recognition that supports a wide range of areas in medical care.

― What kinds of technologies are used for documentation?

Tsujikawa: We used an LLM developed by NEC. Not only can it adapt to the Japanese language with high accuracy, but it is also lightweight and can be operated on-premise. Medical information is very sensitive, and thus many customers are hesitant to connect to the cloud. Our LLM is designed to accommodate such needs.

― What is the technical key point of medical documentation assistance?

Tsujikawa: This is a semantic inference technology specializing in medical terms, which we developed through collaborative research with Tohoku University Hospital. This maps the input medical record information to the meaning and timeline table. This approach enables physicians to comprehensively capture the progress of treatment and understand it in short time. The LLM’s sentence generation also creates quality summaries while inferring the context.

What bolsters this technology is NEC’s original medical database. Upon borrowing ten years worth of medical records data from Tohoku University Hospital, we created teaching data for machine learning while labeling each data set. The annotation work for this was spearheaded by an NEC employee who has a Ph.D. in medical language processing and experience working as a university faculty member. With a specialist in the field steering the project, we were able to conduct tasks quickly while suppressing variations in annotations among multiple members.

NEC’s rich source of medical knowledge and solution design

― What enabled NEC to develop this technology?

Kubo: NEC has a long track record as a electronic medical record vendor and thus has a rich source of domain knowledge in medical information. As such, we have long been thinking how we can use medical record information to help hospital management. The new technology is a fruit of this attitude.

NEC also has a wide spectrum of AI technologies in its portfolio. Another strength is that we can propose a solution that does not consist of only one or standalone function. The new technology was not developed just for streamlining documentation. Based on the concept of biodesign, we understand it to be the first step in support designed from an upper layer with an aim to achieve both improvement in hospital management and reduction in work time. Currently, in addition to expanding the range of support in testing and medical services, we are formulating a roadmap for implementing a management streamlining function that is coordinated with the accounting system and working toward optimizing the entirety of hospital operations. Development of technologies from such a standpoint is a distinctive characteristic of NEC.

― Please tell us about your future goals.

Kubo: The overall optimization of hospital operations is the big goal. Another one is, if I were to give an even more aggressive goal, it would be global expansion. Currently, we are working on our everyday activities to establish a position for NEC where the first company that comes to people’s minds when it comes to Japanese medical language processing is NEC. As the next phase, we also have overseas expansion in view. In particular, India is a major base for NEC’s healthcare strategy. We will analyze local issues and actively roll out localized technologies.

Team members’ messages

Go back to Featured Technologies