Medical codes tell a story. A story of a patient getting better, a clinician making an intervention, a change in how medicine is practiced, the trajectory of a disease. Staring at a sequence of codes, you're reading the story of a person you've never met. It's intimate. R05, you had a cough. U07, you had COVID. M81, you had a fracture. Multiple M81s, you had multiple fractures. Z00.0, a normal medical visit. Lots of codes in a row give you a visceral feeling of urgency. Long delays and spaces between them are a held breath between chapters, you don't know what's next.
I first learned how deeply medical codes could tell us about patient trajectories during my time at Amgen. My incredibly talented ML team wondered aloud: "could we use something like word2vec to predict the next medical code?" In the story of a patient, could we predict the next sentence, or the next word? And if we could do that, could we predict real patient outcomes?
Using millions of records from a de-identified EHR dataset, we trained what we internally called ICD2Vec; embeddings of ICD10 codes learned the same way word2vec learns word embeddings, from the sequences they appear in. We built that into a model we later published as Crystal Bone, which predicted fracture in the next one to two years, an important outcome for patients with osteoporosis [1].
The initial reaction in early development from clinicians and epidemiologists was both very positive and cautious/skeptical. Traditional fracture risk tools like FRAX use detailed risk factors like how many drinks a week, prior corticosteroid use, bone mineral density. Our model was taking sequences of ICD10 codes and throwing them into a predictive model to get an outcome. Explainability was much harder than with the traditional approaches. But practical implementation in a real EHR was so much simpler. Every EHR has ICD10 codes. You can even get them from claims. Data engineering is dead simple, order the codes in a sequence and input it into a model.
We were also shocked by how the model implicitly learned the topology of ICD10. The figure below shows the code embeddings projected into 2D. The model learned the implicit hierarchy and context of the codes purely from the sequence they appeared in clinical histories without any exposure to the text of the codes themselves. Over time it learned things like ["M81", "I10", "M81.1", …] and inferred that "M81" and "M81.1" were related, without ever seeing the description of either code. Here's a plot from the Crystal Bone paper:
UMAP projection of ICD-10 fracture codes colored by anatomical region. From the Crystal Bone paper (Almog et al., JMIR, 2020).
In the early days of COVID the meaning of codes became visceral. Amgen was an Operation Warp Speed partner, and we were doing COVID research on de-identified EHR data. For the first few weeks the WHO didn't even have an ICD10 code for COVID. We were inferring COVID-19 status from combinations of other codes; J12.81 (SARS-associated coronavirus pneumonia), J12.89 (other viral pneumonia), J80 (ARDS), B97.29 (other coronavirus as cause of disease). The code that's now standard, U07.1, wasn't widely usable until April 2020. So in those early weeks, we were watching patient trajectories not by reading the COVID code, but by reading around the absence of one. The story was in the codes that showed up next to where the COVID code should have been.
Today at PhenoML, our platform is used to extract medical codes for a variety of downstream purposes from clinical decision support to care coordination. One of our customers, a publicly traded lab testing company, uses our APIs to extract codes from completely unstructured data to enrich data on rare disease patients. Codes extracted over time tell the story of a patient with a rare disease. The more of those stories we as a healthcare industry read, the better we learn how to treat the next patient who walks in with the same trajectory, a similar story.
A patient's codes are a record of every time the healthcare system looked at them and wrote something down. A snapshot in time, a caption of a visit, artifacts picked up over years.
[1] In a wonderful example of how small the healthcare AI world is, the lead data scientist and architect of Crystal Bone, Yasmeen Almog now leads product for Probably Genetic, a new customer of ours! Probably Genetic is using these codes to build the data and AI system that supports early diagnosis and treatment of rare disease patients.