How AP-HP uses knowledge graphs to structure patient data

Since February 2020, Lettria has been working closely with the AP-HP as part of the BoPA Innovation Chair, which was launched by the AP-HP Foundation and the Institut Mines-Télécom.

Introduction

This project is based on 6 systemic blocks: the Human Factor Block, the Viz Block, the Bot Block, the Light Block, the Touch Block and the Box Block (analogous to the black box in aeronautics).

In practice, these blocks cover the fields of surgeon-patient communication, surgical image capture, natural language analysis in the operating room, augmented reality using digital twins or fluorescent light, collaborative robotics or cobotics (design of collaborative robots), and the protection of operating room and patient data.

The consortium involved in the project includes both businesses and academic institutions, such as INRIA, Institut Mines-Télécom and Université Paris-Sacaly, bringing together leading researchers in the fields of virtual reality, digital twins and artificial intelligence.

Language processing to relieve the burden on care teams

Lettria's team offers its technology and expertise in Natural Language Processing (NLP) to solve one of the major challenges of the hospitals of tomorrow: freeing administrative and medical teams from the management of patient reports.

These reports are present at all key moments of the patient journey (pre-, peri- and post-operation) and constitute a real headache for the business teams. Whether they are verbal (via a doctor's voice recorder) or written, they constantly change in form and contain highly technical terms.

For a simple voice dictation, 3 steps are necessary:

• The dictation is transcribed by hand by a secretary.
• The text is then input into a report on the hospital's letterhead by another department.
• This report is then analyzed by a medical secretary to detect key variables about the patient, which will be added to the Electronic Health Record (EHR).

The human time spent on any given report is estimated at 25 minutes.

Helping the machine understand medical language

Thanks to speech recognition technologies (or speech-to-text), it is now possible to automatically transcribe audio and transform it into text. Subsequently, Automatic Language Processing (ALP) technologies are used to automatically analyze the text and extract key information, such as important patient variables.

A collaboration with surgeons during the COVID period

It took three months for the surgeons at the Paul-Brousse Hepato-Biliary Center and the Lettria team to create the language model specific to the healthcare sector. Given that this occurred in the midst of the COVID pandemic, it was a real achievement!

Thanks to our shared project management platform, the two teams were able to jointly teach the machine to read medical reports and understand technical terminology.

Next steps in our collaboration

After an initial prototyping and testing phase, the solution was validated by the AP-HP management and will be put into production in the coming months in several hospitals. The expected gains are significant, as tens of thousands of reports could be automatically analyzed each month.

‍