How AP-HP uses knowledge graphs to structure patient data

Since February 2020, Lettria has been working closely with the AP-HP as part of the BoPA Innovation Chair, which was launched by the AP-HP Foundation and the Institut Mines-Télécom.

Introduction

This innovative project is based on 5 major systemic blocks: the Human Factor Block, the Viz Block, the Bot Block, the Light Block, the Touch Block and the Box Block (an analogy with the black box in aeronautics).

They cover the fields of surgeon-patient communication, surgical image capture, natural language analysis in the operating room, augmented reality using digital twins or fluorescent light, collaborative robotics or cobotics (design of collaborative robots), and the protection of operating room and patient data.

The consortium includes many academic institutions, such as INRIA, Institut Mines-Télécom and Université Paris-Sacaly. It brings together leading researchers in the fields of Virtual Reality, digital twins and Artificial Intelligence.

Language processing to relieve the burden on care teams

Our team offers its technology and expertise in Natural Language Processing (NLP) to solve one of the major challenges of the hospitals of tomorrow: freeing administrative and medical teams from the management of patient reports.

These reports are present at all key moments of the patient journey (pre-, peri- and post-operation) and constitute a real headache for the business teams. Whether they are verbal (surgeon's voice recorder) or written, they constantly change in form and contain technical terms.

For a simple voice dictation, 3 steps are necessary:

• The dictation is transcribed by hand by a secretary.
• The text is then input into a report on the hospital's letterhead by another department.
• This report is then analyzed by a medical secretary to detect key variables about the patient, which will be added to the Electronic Health Record (EHR).

The human time spent on a report is estimated at 25 minutes.

Helping the machine understand medical language

Thanks to speech recognition technologies (or Speech-to-text), it is now possible to automatically transcribe audio and transform it into text. Subsequently, Automatic Language Processing (ALP) technologies are invoked to automatically analyze the text and extract key information, such as important patient variables.

A collaboration with surgeons during the COVID period

It took three months for the surgeons at the Paul-Brousse Hepato-Biliary Center and the Lettria teams to create the language model specific to the healthcare sector. In the midst of the COVID pandemic, this was a real achievement!

Thanks to the unique project management platform, we were able to jointly teach the machine to read medical reports and understand technical terminology.

Next steps in our collaboration

After an initial prototyping and testing phase, the solution was validated by the AP-HP management and will be put into production in the coming months in several hospitals. The expected gains are significant since several tens of thousands of reports could be automatically analyzed each month.

‍