Industry best practices, standards, and regulations around AI are still evolving.

The impact of artificial intelligence (AI) on healthcare today is profound, undeniable, and escalating. As the technology empowers healthcare stakeholders and clinicians to address healthcare needs and solutions quickly and accurately, new applications are introduced almost daily. As a result, the healthcare AI market is predicted to grow from approximately $7 billions today to $67 billions in just five years.

In some cases, the rush to apply AI and machine learning (ML) in healthcare has outpaced the capacity to ensure its safe and ethical use to promote positive patient outcomes. Industry best practices, standards, and regulations around AI in healthcare applications and AI continue their rapid evolution. Notably for healthcare applications, the new guidelines were published in Nature Medicine and BMJ in May 2022 that described reporting on AI-based decision support. Regulatory bodies and guideline setting organizations have released seminal documents including the US Food and Drug Administration’s (FDA) Total Product Life Cycle (TPLC) for AI- and ML-based software as a medical device, the FDA’s Good Machine Learning Practices (GMLP) guidelines,  guidelines on clinical decision support systems (CDSS), the UK’s AI Updates to Medical Device Regulation Guidelines, the National Institute of Standards and Technology (NIST) introduced a framework to better manage risk from AI, and most recently, the Coalition for Health AI (CHAI) unveiled the Blueprint for trustworthy AI in healthcare.

In this post, we outline a case study on the realization of the recently released the Blueprint for a US AI Bill of Rights. This document doesn’t include all considerations required to address the principles described in the BOR, but instead focuses on of those of interest to practitioners and developers of machine learning, as well to CMIO (informatics, IT, information) in the process of vetting ML vendors.

Implementation

The principles outlined in the Blueprint for an AI Bill of Rights and the corresponding set of concrete actions that can be taken include the following procedures. These actions include a subset of particular importance to this audience but don’t describe all required components to deliver on the five principles.   

  • Safe and effective systems
  • Algorithmic discrimination protections
  • Data privacy
  • Notice and explanation
  • Human alternatives, considerations, and fallback
Concrete steps toward the Blueprint for an AI Bill of Rights.
The principles outlined in the Blueprint for an AI Bill of Rights.

 

Tackling one of the most complex diseases

Clinicians face challenging questions and important decisions at every step in a patient’s journey. The complex clinical decisions are based on intuition and powered by subpar tools. No two diseases are the same, no two patients are the same, and no two journeys are the same, but often, decisions are made based on population averages or of given static, generic clinical pathways.

Patient journey depicting the challenging questions clinicians face at every step.

Coupling this issue with critical provider pain points, the opportunity for the impact of ML and high-quality data conceptualized and delivered at the right time is enormous. Already spread too thin, clinicians often search the web for answers to help make decisions despite the availability of a growing and overwhelmingly complex variety of data, such as variants, increasing treatment options, and new research findings. Moreover, clinicians spend an inordinate amount of time reviewing the large swaths of data required to describe a patient’s health history. One study found that hospitalists spend more time reviewing and documenting (Approximately 34% of their time) than interacting with patients. This constant hunt for relevant clinical information is often cited as a leading source of clinician burnout.

Ronin is driving positive change in care delivery

Ronin is building a future where every clinical decision is rooted in data, personalized for a given individual, and rendered efficiently with confidence. Ronin offers a comprehensive care solution that includes a patient engagement application and dashboards embedded seamlessly in the clinician’s workflow.

Ronin platform is a comprehensive care solution embedded in the EHR.

 

In collaboration with Oracle, Ronin applies safe and ethical AI practices that already meet or exceed the principles described in the blueprint. The following summary applies to a subset of salient practices and the supporting infrastructure used to achieve this goal.

Oracle Cloud Infrastructure empowers Ronin

Ronin recognized Oracle as the cloud provider best suited to address its technical and organizational needs, and transitioned its existing infrastructure to Oracle Cloud Infrastructure (OCI). OCI includes the services needed to migrate, build, and run sophisticated enterprise machine learning workloads safely and effectively in the cloud with superior price-to-performance ratios.

Safe and effective systems

Proactive and comprehensive monitoring of both the ML model’s health and the health of the data is the first step. Comprehensive model health monitoring is an essential component of a complete ML stack. For ML operations (MLOps) to achieve the same level of platform as a service (PaaS) as they expect from other cloud native managed service solutions, cloud-managed AI tools require mature capabilities in the following areas:

  • Reproducible data transformations: In terms of data changes, models are brittle. Even a small change in ICD-9 codes can cause schematic drift that’s intolerable to a ML model. Ronin gives a comprehensive understanding of the patient’s health, which means performing hundreds of data transformations for each prediction. The quality of those predictions requires reproducible orchestration of all these transformations.
  • Model management: Reproducibility and ease of translation between experimentation and production central location for model lineage, model versioning, stage transitions, such as from staging to production, collaborative model building, and replicable model execution pipelines.
  • Proactive and comprehensive monitoring: Cloud-hosted monitoring and management services should include comprehensive data and model health observability metrics, including data completeness, accuracy, consistency, and  integrity, and performance metrics between actuals and predictions.

Ronin Machine Learning System, including model development and model health monitoring that address the principles of “safe and effective system” and “protections against algorithmic bias.” Quality, safety, and equity were assured with safety audit decision points throughout the end-to-end process.

The following core OCI cloud native services enable the architecture:

  • The OCI Data Science service, the fulcrum of the ML lifecycle, is a fully managed and serverless platform for data science teams to build, train, and manage high-quality machine learning models. Automated ML capabilities rapidly examine the data and recommend the optimal algorithms, while tuning the model and explaining its results. The cloud platform’s security tools and user interfaces enable users with multiple roles to participate in projects and share models. Model-agnostic explanations help data scientists, business analysts, and executives have confidence in the results.

OCI Data Science enables users to manage features in a central repository called a feature store. The feature store addresses the most vexing challenge in the ML lifecycle: Dealing with data or feature engineering. Features and models are the most valuable assets generated during the ML lifecycle, and a feature store is the data warehouse for data science. OCI Data Science’s feature store manages the feature engineering workloads that run in its secured cloud infrastructure, and the platform tracks the code changes, data sources, and ensures that the feature values meet standards of quality, parity, and fairness. The feature store integrates with the rest of the platform, which includes group parity and representation, which enables the system to track which features are used to train which models and which features are used at inference time. This distinction enables data scientists to quickly assess which models are impacted by a change in the feature definition and which models need to be retrained or decommissioned. Data scientists can also use this functionality to address the principles of “safe and effective systems” and “algorithmic discrimination protections” in the US AI bill of rights by preventing the introduction of broken data pipeline and providing transparency in the representation of subgroups within the dataset.

Critical to the principles of “safe and effective systems” and “algorithmic discrimination protections”, model monitoring can enable you to readily monitor data drift and model drift, including data representation, model bias diagnostics, and monitoring subgroups. It also enables you to define your own metrics and use templatized fairness and bias diagnostics that you can monitor beyond drift. 

  • OCI Data Flow is a fully managed big data service that allows you to run Apache Spark applications at any scale with lightweight administration. Spark has become the leading big data processing framework, and OCI Data Flow is the easiest way to run Spark in OCI because developers don’t need to install or manage anything. Data Flow is deeply integrated with OCI Data Science through notebook sessions, where you can connect a notebook session to Data Flow to run Spark jobs. This integration gives you the flexibility to use different scalable options to transform your data, engineer your features, and train your models. It assists you in delivering the safe system at scale.
  • OCI Data Catalog is a metadata management service that helps data professionals discover data and supports data governance. Designed specifically to work with the Oracle ecosystem, it provides an inventory of assets, a business glossary, and a common metastore for data lakes. The Data Catalog Metastore securely stores metadata about tables schema, location, table statistics for query optimization, and partition metadata facilitating expedited data exploration for data scientists.
  • OCI Object Storage enables customers to securely store nearly any type of data in its native format, and data is encrypted at rest and during transmission. It helps ensure that data is secure and supports the data privacy principle.
  • OCI Identity and Access Management (IAM) solutions secure access to enterprise applications for both cloud and on-premises deployments.

Algorithmic discrimination protections

According to the Data Science Oath, in healthcare, data isn’t just numbers without meaning or context, but represents real people and situations. Work in this field can lead to unintended societal consequences, such as inequality, poverty, and disparities because of algorithmic bias. Proactively preventing inequality and disparities because of algorithmic bias and implementing algorithmics objectivity to avoid making dangerous and biased actions is particularly important in healthcare, where inequity can be observed in most health systems’ data.

Algorithmic bias arises from two sources. The first source is biases from the input features or ground truth. Involvement of subject matter experts who can provide the missing link to the data and its significance and nuances can surface these inequalities. The second source is from a lack of representation. If training data lacks representation of all possible patient populations, and the resulting model is applied to another group not represented in the data, the performance of the model in that unrepresented population is inherently inaccurate. In fact, the model can potentially automate or make invisible biases that harm patients within the unrepresented groups. 

Notice and explanation

Ronin’s workflow is designed to assist clinic decisions with the following features:

  • Transparent origin: How the algorithm was developed, the training data, and how it works in the UX are explained succinctly within the workflow.
  • Visualization in the UX: The basis of the prediction is presented in the workflow instead of an external document or publication.
  • Actionable data: The workflow surfaces actionable aspects first to assist with determining the next course of action.
  • Cohort relevance: The relevance of the cohort used to make a prediction to the current patient is immediately available.  

Training and education:

Model interpretability is delivered at the time of clinical decision making personalized to the patient. Interpretability include succinct explanation of how the prediction is made, basis of the prediction, and relevancy.

Similarly, model predictions without context of what drives the prediction are incomplete. Oracle delivers similar insights to ML practitioners and developers, where model deployment allows for the serving of model explanations along with model predictions. The context is automatically part of inference.

Data privacy

Data privacy has taken center stage, thanks to legislation like the European Union’s General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). At minimum, healthcare requires best practices such as informed consent, transparency, lawfulness, a workforce culture of compliance such as SOC2, HIPAA, and HITRUST, and industry standards in data security, access, auditing, encryption, and continuous monitoring. Data privacy is such a large topic that it warrants a full article. In this piece, we discuss only one aspect of notable interest to ML developers. 

Current approaches to AI require complex webs of trust, which necessitate more protections beyond digital tools. Input and output privacy, which guarantee that a user’s input and output data cant be observed by other parties, go together with security. Proper security controls reduce the occurrence of data leaks. However, recent studies have shown the potential for ML system leaks even when the input and output data is deidentified because wide variation exists in what’s considered sufficiently difficult to reidentify data.

Delivering on Privacy by Design—proactive not reactive; preventative not remedial—requires privacy-preserving ML. The building blocks of privacy-preserving ML include federated learning or multiparty computations that use data across tenants, homomorphic encryption, and differential privacy. For example, Ronin utilizes encoded representations of data with representation learning, summary statistics over individual outcomes linked to personal identifiers, and student-teacher models as part of its ML system.

Human considerations

Notable human considerations in healthcare delivery include the following “rights” of clinical decision support:

  • The right information
  • To the right person
  • In the right intervention format
  • Through the right channels
  • At the right time in workflow

Data scientists and users of this technology must look to success metrics that are meaningful. To determine success, implementation of the five rights and other key human considerations must be grounded in real world impact. The clinician using AI results in better outcomes than the clinician alone.

The second tenet is the focus on measuring real-world impact over abstract measures. ML practitioners often explicitly ignore or remove problem-specific details, usually so numbers can be compared across domains. However, most ML metrics reveal nothing about the model’s impact in clinical settings.

Ronin emphasizes human-centric design complemented by well-designed studies to quantify meaningful improvement to cancer patients on specific clinical outcomes, clinical efficiency, patient experience, and healthcare expenditures. When possible, these studies are automated to better understand and properly attribute the clinical impact of a new product feature.

The results: A safe and ethical data and AI-powered platform to improve cancer care

With this infrastructure, Ronin delivers ML and data insights that help clinicians make better-informed treatment decisions, predict, and preempt adverse patient events, and more effectively engage patients. Realization of the five principles allows for AI-based capabilities to advance the quintuple aim of enhanced patient experience and outcomes, improved clinician satisfaction, improved population health, and reduced costs in a way that protects all patients from harm.

Ronin drives advances the quintuple aim using AI, such as a workflow-integrated longitudinal timeline. Today, it can take 30 minutes to get a comprehensive understanding of a patient’s status. Ronin can provide this information in seconds. Ronin’s timeline feature provides an intuitive view of the patient journey, surfacing the information clinicians need to make critical decisions. This view uses natural language processing (NLP) to search through the approximately 80% of electronic health record data stored as unstructured notes and documents. Not only can this process improve clinician efficiency, but it also decreases the chance that critical events are missed, such as an emergency department (ED) visit at another institution, a change in a supplemental medication, or metastasis in a pathology report.

Ronin data discovery delivered within clinical workflow.

Ronin’s solution helps clinicians course-correct at-risk patients with forecasting based on evaluation of more than 220 distinct patient attributes. For example, the platform predicts ED visits to enable early intervention, which is particularly important for cancer patients, who average about two ED visits per year when 43–61% of those visits are potentially preventable. Moreover, the cost of an ED visit can exceed $1,000 with another $10,000 if the patient is admitted to the hospital. From the patient perspective, the experience is highly unpleasant and puts them at risk for acquired infections.

Ronin Data Insights based on outcomes of patients with similar attributes embedded in clinical workflow

Conclusion

The guidelines, best practices, standards, and regulatory landscape of the medical industry are complex and continuously changing. Keeping abreast of it while improving efficiencies in patient care isn’t an easy task. With the right combination of AI and ML solutions, cloud technologies and best practices lead to promising results.

For more information about Ronin solutions and Oracle Cloud Infrastructure, see the following resources: