AI-Driven Automation in Monitoring Post-Operative Complications Across Health Systems

Artificial intelligence systems have been previously used to predict post-operative complications in small studies and single institutions. Here we developed a robust artificial intelligence model that predicts the risk of having cardiac, pulmonary, thromboembolic, or septic complications after elective, non-cardiac, non-ambulatory surgery. We combined structured and unstructured electronic health record data from 3.5 million surgical encounters from 25 medical centers between 2009 and 2017. Our neural network model predicted postoperative comorbidities 15 to 80 times faster than classical models. As such, our model can be used to assess the risk of having a specific complication postoperatively in a fraction of a second. With our model, we believe clinicians will be able to identify high-risk surgical patients and use their good judgment to mitigate upcoming risks, ultimately improving patient outcomes [1].

In today's health systems with an escalating proliferation of electronic medical records, the increasing deployment of mobile care, and the growing demand for better patient outcomes, the need for automated monitoring is compelling. Effective monitoring requires an up-to-date representation that is grounded in active patient data. This is an "intelligent picture" that integrates evolving patient understanding with health knowledge from the evidence base and expert faculty. AI-driven automation in monitoring is the obvious step [2].

For intraoperative decisions in treating chronic surgical wounds, there is, in sharp contrast, a very large evidence base, bolstered by strong clinical faculty experience, for guiding surgery. However, the evidence base is largely silent regarding postoperative decisions that are necessary for achieving healing at the individual patient level. Long-term conditions devolve, evolve, and are expressed—wound by wound—in each patient's home setting. Much of the available evidence is therefore typically out of reach for patient-centric decision support [3].

What must such AI-driven automation do? After a patient undergoes a surgical procedure, identifying clinical complications that represent unacceptable patient conditions, along with the patient monitoring needs that must best drive patient care, is a pressing goal. Fires to be put out are postoperative patient illness risk factors, along with their known signs of acutely threatening vision, lung, heart, metabolic, and other organ functions, which subsequently drive more specific surgical wound healing deterioration [4].

Figure 1

AI-driven tool could improve brain pressure monitoring in intensive care patients

1.1. Background and Rationale

Surgical care is a major component of health services, with an estimated 300 million surgeries performed annually worldwide. Post-operative complications are a serious public health problem. Patients developing complications are at significantly increased risk of mortality and often have prolonged hospitalization, leading to a great loss for the economy [5]. Many complications may result in patients' long-term morbidity and require ongoing health support. Therefore, any adverse postoperative outcomes are undesirable for both the patient and the health system.

Existing manual monitoring of postoperative complications approaches are intensive care unit-based, following vital signs every hour even with healthy patients. They require multiple nurses for every patient for observation each day, resulting in high costs and insufficient human resources, which limit the promotion of these approaches. This leads to a lack of data and visibility for hospital managers, and complications may easily be missed at an early stage, which can be fatal for patients. Among automated approaches, such as tracking vital signs trends to indicate complications, most are limited to certain types of postoperative complications, like respiratory or cardiac complications. Others are complex and not easy to use or understand, making them unsuitable for general ward use where patients are not under continuous professional supervision. Additionally, the manual work on threshold calculations burdens nurses' overtime work. The latest advances in artificial intelligence, particularly deep learning, could make a significant impact by providing automated continuous surveillance and pattern recognition of diverse postoperative complications by tracking a variety of signs and combinations of signals, even with imperfect data quality and specialized expert knowledge.

Equation 1: Risk Prediction for Complications

$\begin{array}{l}\text{where:}\\ R_c:Predicted\text{ risk score for a complication}\text{.}\\ x_i:\text{Input features (e}\text{.g}\text{., patient vitals, demographics)}\text{.}\\ w_i:\text{Model weights for feature }i\text{.}\\ b:Bias\text{ term}\text{.}\\ \sigma :\text{Activation function (e}\text{.g}\text{., sigmoid)}\text{.}\end{array}$

1.2. Research Aim and Objectives

The primary aim of the project is to design, develop, and validate an AI-driven automation system to reliably electro-convert the data from these disparate information sources into a consolidated and coherent review of each patient's surgical departure from optimized recovery trajectories [6]. The goals are to employ these clinical data forms to provide an AI-driven automated submissions system as a basis for inquiry into perioperative information systems by medical practitioners and allied health professionals, with patient consent, at any phase of post-discharge recovery, in total support of ongoing administrative care provisions for which clinical services remain engaged and vigilant. To achieve these primary and instrumental goals, a three-tiered model-based approach will be undertaken: on the admission of the patient for a planned elective surgical procedure, set up and manage identified and agreed clinical data form assessments covering the perioperative admission [7]. Assess the patient as at 'preparedness' for the surgery based on the patient's underlying clinical profile, influencing the objective clinical profile of the surgical process that the patient is about to undertake. Manage escalating patient 'departure from optimized recovery trajectory' alerts that require both intra-operative and immediate postoperative clinical assessment prioritization for attention before increasing departmental budget allocation overwhelms simple service budgetary constraints [8]. Continuous postoperative data form assessment checks for the patient's ongoing trajectory through recovery ensure budgetary responsibility for reassurance is communicated to subacute and domiciliary care staff that the patient's ongoing recovery is proceeding as anticipated. Schedule and conduct appointments and resource personnel for nurse-led practices for the Inpatient Surgical Units. These objectives are determined by a series of questioning, inquiry, and response sub-objectives. Each sub-objective relates to the direct clinical responsibility ownership of the Viewing Call Cycle Management Responsibility No. Conducted when the patient is undergoing a surgical procedure under the primary care of General Surgery, Gyn-Oncology, Orthopedic Surgery, ENT Surgery, and minor surgical procedures.

The application of AI has gained traction in a variety of healthcare systems, and it is tapping into different niche areas. Although AI-driven pathologies exhibit errors, these errors mimic human ones, and they are often not explainable [9]. Given that healthcare systems are complex and are knit into ever-changing disease patterns, AI applications in healthcare should aim to build a 'whole picture analysis' within a systematic approach to decision-making. AI in healthcare should embody a harmonized conceptual framework that can deploy, scale, and entrench safe and effective AI tools. While AI-driven pathology errors are often not interpretable by humans, the outcomes provided by AI-powered healthcare applications need digital ethics approved by human healthcare professionals. Overall, AI applications in healthcare should be human-centric, allowing us to create digital twins of the professionals themselves. Together, AI applications in healthcare should support continuous data liquidation, affirming the importance of addressing long-term sustainability to protect privacy and confidentiality. A survey among healthcare executives predicted that a significant percentage of healthcare providers will be using predictive and AI-driven tools shortly [10]. A subsequent survey showed that AI technology implementation was in its infancy across healthcare, with a large percentage of healthcare organizations reporting that they have some sort of AI strategy. It pointed out that only a small percentage of organizations have rolled out an AI project. The survey affirms that the key enabling foundations require further maturation.

Figure 2

The Role of AI in Hospitals and Clinics

2.1. Overview of AI in Healthcare

Artificial intelligence (AI) in health care has been gaining significant momentum in the past decade, with grand expectations to transform care delivery [11]. From an operational standpoint, the advent of AI and machine learning (ML) in health care has allowed rapid digitization of medical data, automated routine tasks, improved diagnostic accuracy, and personalized treatment. However, despite the rapid pace of growth, several barriers exist to integrating AI tools in clinical workflows. A significant challenge is tied to the generalizability of models in AI. That is, training a machine learning model on data from a single context and attempting to use it in another context often results in suboptimal performance. Clinical data is highly contextualized, and generalization is a daunting challenge, particularly when leveraging clinical data from multiple institutions [12].

Despite these challenges, we have progressed in developing methods to train models on multi-site retrospective data using federated learning approaches, wherein models are trained across sites without sharing the data. However, these techniques are limited by the need to develop distributed models across multiple sites, the need for an application programming interface to query the health system’s electronic health records, and delaying models’ deployment into clinical practice before the need to train on data. One common problem in developing distributed models that often limits accuracy is the inability to standardize and validate medical record formats across locations. For example, clinical notes can be categorized into three formats: out of distribution, where the notes are dissimilar to the other institution’s notes but still English sets of characters; out of vocabulary, where the note text format is similar to other sites but was not seen in training; in-vocab, where the note text was present in the training data. A study was conducted to characterize the out-of-vocabulary and out-of-distribution variation across different institutions using tools for natural language processing. It shows that while natural language processing task performance in one institution is relatively stable over time, the variation across institutions is striking. The results imply that clinical natural language processing models are more brittle, and hence further work is needed to address multi-site translation of narratives into standard, structured data [13].

2.2. Benefits and Challenges of AI Integration

The aim of this text is to highlight the benefits and challenges involved in the integration of AI models into a practical healthcare application using the example of a large-scale automatic monitoring system for post-operative complications. We also explore the potential of such a system beyond the case study and will prepare the ground for the clinically valid large-scale validation of the model performance [14].

Benefits and Challenges of AI Integration The implementation of AI-powered health applications can bring significant benefits to the current healthcare system, including higher productivity gains, cost efficiency, accuracy, and enhanced patient care and medical staff focus. Several implementation attempts of AI technologies have been reported. However, according to the current number of successful implementations and their impact on the healthcare system, most academic achievements still seem to be experimental applications. High-standard limitations and complexity of healthcare data, ethical requirements, data privacy issues, data imbalances, interpretability, and clinical validation of the model can hinder the implementation of machine learning technologies [15].

The intrinsic lack of interpretability of machine learning models, which does not guarantee human-understandable and actionable outputs, can be a major issue for a clinical practitioner who is not an AI expert. Especially with high-discrepancy applications like the integration of AI into clinical settings, model interpretability is an open area of research. If the developed models are not capable of being explainable, the created disparity as a result of the recommendation can affect expert trust and adoption, decrease AI utility resulting in unnecessary care redundancy, and increase administrative burden [16]. Furthermore, it is becoming increasingly evident that AI should not be limited to black-box statistical tools when implemented in clinical tasks. Despite these challenges, successful AI models integrated with feedback in clinical decision support pipelines have been introduced for different tasks.

Figure 3

Benefits of AI in Healthcare

Post-operative complications impose considerable stress on patients and lead to higher healthcare costs for providers, resulting in an increased length of stay in the hospital and a reduction in overall quality of life. Such complications may become costly burdens for the provider and potentially life-threatening for the patient [17]. Complications may have a mean cost for high specialty surgical groups, with some complications reaching significant cost levels. While the majority of post-operative complications are recognized within the initial hospitalization, some may appear after discharge from the hospital or following transfer to a post-acute care institution, home health agency, or residential setting, resulting in increased costs and worsened outcomes. It is currently difficult to track these patients effectively and continuously following initial discharge, given care is more fragmented, so the identification of post-discharge complications can take some time and may not be communicated back effectively to the provider, significantly affecting hospital performance data and star ratings [18].

Automating the detection of complications and creating interventions may directly or indirectly improve the way that health systems are compensated and evaluated in terms of their quality of care and patient safety. However, creating interventions based on continuous and detailed surveillance for all patients is unattainable in real-world settings given the manpower, costs, and resources required to review each patient continuously throughout their stay and after discharge. Expanding the perioperative clinical decision support system to include post-discharge complication recognition could allow the potential to provide these surveillance capabilities for an entire health system or network, helping providers bridge the gap between inpatient and ambulatory settings. There is also a benefit to vendors through increased sales of products for remotely monitoring perioperative conditions and at-risk incision sites [19].

Figure 4

Postoperative Remote Automated Monitoring

3.1. Common Complications After Surgery

The postoperative period is riddled with potential complications. Some common complications include surgical site infections, surgical bleeding, venous thromboembolisms resulting from deep vein thrombosis or pulmonary emboli, urinary tract infections, pneumonia, delirium, and unplanned intubations. Patients with one of these post-operative complications are likely to experience other complications [20]. For example, patients who develop a surgical site infection face nearly a 300% increase in the odds of experiencing additional post-operative complications, while diagnoses of surgical site infections, pneumonia, unplanned intubations, and urinary tract infections all have a significantly greater than 2-fold influence on developing other post-operative complications [21]. Such a high correlation of risks likely reflects the physiological toll stopping normal organ function and cutting and suturing the body that surgery inherently exerts. While some of these complications are inevitable due to the high volume of surgeries performed.

Each year, more than 60 million inpatient surgeries are conducted, and more than 130 million are conducted worldwide each year. Such a high volume makes reducing unnecessary complications a key cost-containment goal across health systems [22]. Hospital costs of surgical site infections alone total about $3.5 billion per year. Added to this are the non-trivial personal costs that many of these patients and their families have to bear, which are incurred from lost wages, additional treatments, and even increases in the risk of mortality. The costs of developing a healthcare-associated infection can vary significantly depending on the type of infection, but most infections for hospital-specific data range from $1,200 to $32,600. Preventing these post-surgical complications is frequently an economic necessity; being able to predict their onset will provide significant clinical and financial benefits.

Equation 2: Alert Generation Threshold

$\begin{array}{l}A:Alert\text{ status (1 = alert generated, 0 = no alert)}\text{.}\\ R_c:Risk\text{ score}\text{.}\\ T:\text{Predefined threshold for alert generation}\text{.}\end{array}$

3.2. Importance of Timely Detection

Timely detection of postoperative complications is paramount because both inpatient hospital stays and, wherever feasible, general anesthesia are rapidly becoming exceptions rather than the norm in surgery. This trend suggests that post-operative care and management should increasingly be focused, integrated, and embedded inside near-native physiologic homeostasis and lower levels of healthcare [23]. Active prevention and mitigation of predictable post-operative complications and morbidity through personalized optimization and precision management of autoregulatory aspects of organ and systemic blood flow, tissue-level metabolic support, real-time remote feedback on organ and wound status, patient mobility, and a seamless transition home should be the new standard of care. However, there is a paucity of suitable algorithm-based touch-free adaptive medical devices that can accomplish such continuous longitudinal monitoring of the surgical patient [24].

I have created an algorithm that is designed to learn to identify postoperative patients for whom providing care would be clinically relevant in the next 24 to 48 hours, based on historical data. It is deployed via the system used by surgeons across hospitals to monitor the clinical status of post-operative cardiac surgical patients in the days following their discharge from the coordinating hospital to their homes. Using this algorithm, surgeons and clinical teams at remote hospitals share in real-time with their home hospital teams all the data of the post-operative patients, as well as direct electronic clinical communications, care and medication plans, and educational surgery re-entry, patient follow-up, and trust-bridging materials needed to enable them to respond knowledgeably on their behalf. This has introduced a model of shared records, collaborative clinical care, and shared responsibility for postoperative patients that can help reduce the burden of big city hospital traffic [25].

While an expansion of monitoring at home is ideal after surgery, in reality, monitoring after discharge from the hospital is infrequent. Most patients are seen in the surgeon’s office once every two weeks after surgery, and the patient’s vitals are taken at that time. Also, a caller may inquire about how the patient is doing. Once patients are discharged, they are, in most cases, no longer physically followed by the surgeon during the critical weeks after their operation. Likewise, outside of military settings, data from ambulatory remote monitoring are not currently used to inform ongoing management of critical complications. Our experience has also revealed an asymmetry in data. Data in the hospital, even in the post-discharge observation units, are plentiful. Vital signs are taken every 4 to 8 hours or can be continuously monitored. In contrast, vital signs from patients at home are sparser [26].

The patient’s electronic medical record is another important source of data. Information about the patient can be provided by the patient or caretakers, but this verbal data may be less reliable. Appointment check-in and check-out information can be a provider-entered description of the trip to the surgeon’s office to help reduce a patient’s activity. Information about prescriptions filled is a measure of home wound care. Other important data capsules include notes from the surgeon, indications of ER or urgent care visits, a description of the duration of the stay, and whether the patient was readmitted or taken back to surgery. These all serve as quality checks on the data. Finally, data on the initial presentation, including the medication being prescribed and the post-operative recovery, can be taken from the patient directly [27].

Figure 5

Vital Sign Monitoring of In-Hospital Patients: A Review of the Current Evidence

4.1. Manual Monitoring Processes

Traditionally, in-hospital post-operative complications have been monitored by the inpatient care team—led by surgeons—for all patients diagnosed with surgical site infections, urinary tract infections, pneumonia, or sepsis. These infections are identified by surveillance tests and are subsequently diagnosed by clinical cultures of surges or body fluids within the recommended duration after a procedure [28]. Advanced infections that develop in a patient should be quickly reported to the care team by the patient or the patient's family upon recognizing the symptoms. Surveillance tests are conducted if the patient's in-hospital chart shows early warning signs. Surveillance tests were manually scheduled in most hospital units until very recently. More advanced inpatient telemonitoring devices were used on a smaller scale, and high-risk surgery patients were telemonitored in a specialized step-down unit at academic centers. These highly specialized monitoring units are, however, not typically available at community and most academic centers. The same was observed for in-hospital observation which is filled with routine check-ins by the nurses and care team and is typically performed for all patients after a stressful surgery [29].

4.2. Limitations of Current Systems

Several limitations may hinder the current approaches detailed in the previous sections. First, there is a delay in reporting event rates via traditional scientific methods. The systems typically must wait for the article to be completed before sending the manuscript to the editorial board. After acceptance, the timing of publication is scheduled to fit the journal’s calendar and related policies. This typically results in lead times on the order of several weeks to months after the article's completion. Whether due to the time taken to conduct meta-analyses, provide coverage, or produce evidential claims, those timeframes may lag the pace of development of systems with real-world health applications [30]. By some estimates, less than 5% of 110,000 already publicly available natural language processing tools deal with health and medicine. Second, the noise involved in getting to a valid claim in clinical scientific expository writing is followed by clinical scientific research because it requires optimizing a very specific and relatively patchy pattern in noisy data, often at small sample sizes with few occurrences of the target pattern. Third, epistemic uncertainty abounds and involves:

• The inherent uncertainty in doing science about rare or noisy events in complex systems planted the seeds for building the models but has no direct influence over their sizing or building [31].
• Uncertainty in claiming evidence about real-world performance through meta-analyses and narrative reviews.
• Uncertainty in the true value of the scientific claim reported.

Fourth, and relatedly, the literature is yet to provide generalizability of predictive performance across different types of health systems or estimates of whether the performance of predictive tools is the same as the comparative estimates for the scientific claim that is derived from them [32]. This can be quite relevant in some settings, especially when the gain in accuracy and speed of these types of tools is relatively modest. For example, when the use of C-reactive protein and procalcitonin for sepsis diagnosis provides a narrow AUC, the marginal clinical utility of automating the prediction of sepsis onset and providing confidence intervals for scientific inferences of evidence of sepsis may not be at the point where the exact configuration process should be embarked upon or when the tools should be used in practice [33].

The objective of AI-driven modeling in healthcare is to facilitate automated decision-making at the point of care. In pursuit of reducing dependence on human physicians to predict, intervene in, and treat acute medical scenarios, companies and researchers have partnered with hospitals to provide AI-driven solutions across a spectrum of challenges, from evaluating metrics from stroke patient CT scans to predicting the onset of complications in cirrhosis patients. In the postoperative period, complications often manifest over a wide range of time and vary in severity, making the problem particularly well-suited for AI-driven models to encourage early intervention. The first step in developing machine learning models for post-operative monitoring is to define a target sample for model evaluation. The model user interacts with the machine learning model by inputting features of the patient, along with the desired prediction type, the desired model output, and potentially some other pieces of information that can assist the model in its decision-making process. Model development then involves defining the input feature set, as well as the output feature set [34].

Figure 6

Patient Satisfaction in Postoperative Care

5.1. Machine Learning Algorithms

We operationalize several machine learning algorithms to build models that predict eleven physiological postoperative complications based on input demographic and admission characteristics. To implement machine learning-based algorithms, we have used a visual platform. Based on the literature review and our expertise in healthcare analytics and AI algorithms, we considered the following algorithms: Neural Network, CHAID, Decision Tree, Logistic Regression, Random Forest, k-nearest Neighbor, and Support Vector Machine. These models are implemented in a package that facilitates statistical tools for model training, evaluation, and prediction.

The aforementioned models are implemented in a visual platform and a statistical package. Each modeling technique is employed to predict worse outcomes from patient-level demographic and admission characteristics independently. We use standard steps in AI algorithm implementation to build and assess the performance of these classifiers. In this context, the predictive power of algorithms varies based on the nature of the models, data distributions, sample sizes, and algorithms. However, in this study, we implement a set of well-known models, and our study sample's related intrinsic size differences across these models are comparatively minor, which provides the likelihood of generating a consistent set of outputs. Thus, they render an equal model comparison. Additionally, we will run our algorithm to implement these models for training, cross-validation, and testing, and select the best model by fine-tuning their parameters. Upon the selection of our best model, we will run it on a testing dataset to guarantee the model's predictive power.

5.2. Natural Language Processing (NLP)

NLP supports the progression down the levels of the data pyramid by allowing the machine to understand words and sentences like a human. An important part of language understanding is named entity recognition (NER)—identifying key clinical phrases that describe an in-depth clinical concept such as symptoms, procedures, and diseases. Automated NER systems make it possible to process thousands of MR reports efficiently, providing immediate opportunities to automate screening activities for various medical conditions. The program analyzes and assigns each word of a given text to a certain category; words that do not fit into any of the existing entity categories are labeled as 'unknown'. The system pays attention to the world's current construction and the surrounding context to tag the most appropriate label. Machine learning can play a predominant role in these developable systems by enabling the algorithm to learn from massive datasets [35].

NLP areas chiefly developed to date in the clinical literature involve question answering, document classification, information extraction, and the development of more accurate specialized vocabularies. The patient's history is then further utilized to obtain other categories of notes, tests, and reports. While most adult patients remain healthy during their post-operative period, adults who require hospital readmission are quite significant. Automated NLP algorithms that process the free text in patient records have shown useful results for transitional care purposes. Documents are part of the data pyramid that contains significant clinical information that an NLP algorithm needs to learn from to efficiently process subsequent, smaller levels of clinical data.

Given the rapid development and implementation of AI systems in the healthcare domain, it is also important to outline the challenges associated with their deployment. Indeed, as the use cases and thereby the convenience and benefits of AI models may seem overwhelmingly strong and the realities of AI capabilities perhaps blurry to many, we would like to address potential implementation challenges and frame some of the ethical issues that the introduction of AI to hospitals needs to address. Currently, the hybrid model of AI and clinical experts is perhaps the most transparent way of using these new technologies, as human experts can oversee the outputs and guide further decision-making processes. This, however, introduces the necessity of understanding and managing the sources of errors both from the side of AI and human practitioners, which is necessary for lowering potential over-reliance on a partially understood system.

It is essential to understand that AI is not a direct plug-in to existing operational models, as the implementation and actualization of potential benefits require significant developmental effort and dedicated ethical consideration. Moreover, as AI works with data, it is essential to understand existing limitations and biases in this data, whether it originates from small or large groups. For example, countries vary in the content that is readily available as part of their electronic health records, with variations regarding the content, such as admission reasons and disease-specific information, which may be key to understanding data completeness or bias. Furthermore, the successful implementation of AI models relies on model availability, the speed of model training on new data, the local health system dynamics, and the associated deployment of risk models, due to exclusive historical data use, open-source model development, or because the model has yet to be validated on the local data cohorts. This can contribute to ethical challenges and considerations – from health informatics to broader socio technical contributions, preventing bias, reinforcing existing inequalities, or posing a threat to transparently explainable models. Regardless of these potential implementation roadblocks, policymakers are under pressure to perceive AI-driven systems development and law regulation from the innovation standpoint while minimizing the associated risks.

Figure 7

Remote patient monitoring using artificial intelligence

6.1. Data Privacy and Security

Data privacy and security are the most significant concerns associated with making data from an EHR available for training and testing an AI model. Privacy is a concept concerning the confidentiality of patients, and security deals with the reliability of data from tampering, repudiation, or disclosure without the patient’s consent. Adherence to regulations and implementation of de-identification techniques can help alleviate privacy and security concerns. In addition to de-identification methods, ensuring data access on a need-to-know basis using secure software and hardware solutions can reduce the risk of an unauthorized user breaching the database.

A few recommended best practices to ensure data privacy and security include deploying and executing AI models within the hospital. The hospital retains data in perpetuity, also known as unsupervised AI training. That way, patient data is never shared or converted into an external database. When the training data from a hospital are combined with fresh patient data to provide a prediction, or when the combination happens outside the hospital, that is considered supervised AI training. Selected employees of the hospital medical staff require individual authorization and authentication to use the models. AI models are used only on resources within the hospital, and authenticated connections to data sources, servers, and devices outside the hospital are tightly controlled. Devices that store EHR data must be physically protected within the hospital. Access to medical records of patients not currently being treated is disabled, retention of all EHR data is whitelisted entirely within the hospital, and appropriate logging of all access to high-quality hospital EHR must be conducted. These practices will maintain data security and privacy and are consistent with differential privacy principles. In performing this assessment, it is important to recognize organizational capabilities and constraints. If necessary, organizations have the option of enhancing policies, processes, and technology that currently exist.

6.2. Bias and Fairness in AI Algorithms

One interpretation is that, when we apply AI to build algorithms that support clinical decision-making, we must think not only of the performance of such algorithms in terms of predictive accuracy but also of bias and fairness. Such considerations must feature more often in the discussion when AI and machine learning models are being considered as solutions to choices made by health systems. Clinical AI developers must adhere to strict regulatory guidelines that ensure their tools behave safely, and guidance from professional bodies on fairness, bias, transparency, and explainability could be equally well designed and enforced. These challenges widen when predictive and decision-making algorithms are used further downstream to automate patient management in policy development.

In the academic setting, where there is not yet widespread clinical AI deployment, the potential for harm is rarely discussed and bias and fairness are seldom considered. While performance is evaluated in the context of real-world data, which can reveal discrimination against certain groups, typically algorithms are validated by standardized performance measures, with biases assessed in an ad hoc manner if at all. Conversely, data has inherent biases given how collection is influenced by existing societal structures and policies; consequently, health system AI development must encompass ongoing and complex discussions around data representative of the underlying target population to ensure such biases are avoided. The requirement should be for robust models that maintain similar levels of accuracy across different clinical and demographic groups. When biased predictions do occur, suboptimal patient care can have harmful, long-term consequences [36].

Equation 3: Efficiency of Prediction Model

$\begin{array}{l}E:\text{Efficiency ratio of the AI model}\text{.}\\ T_c:\text{Time taken by the AI model to predict complications}\text{.}\\ T_m:\text{Time taken by classical models for the same task}\text{.}\end{array}$

I illustrate the general system in the context of two case study scenarios from the field of healthcare: sepsis and post-surgery complications following a colonoscopy. Because we leverage event prediction models that are designed to not just predict an event, but also say when this will occur, we can offer the possibility of automated patient triaging in any automated system. We begin by explaining this specific system in the context of our two healthcare case studies. For both scenarios, the event prediction part of our approach uses a Temporal Convolution Network optimized for the event with hyperparameter optimization. It is the spring of 2018, and in a large diverse hospital in the northeast of the US, our AI-driven system kicks into action when a patient arrives at the waiting area of the hospital alongside the 10th patient on the waiting list for his procedure, a colonoscopy. The colonoscopy is a minor procedure, but the patient is getting in a large hospital with advanced multi-modality services, and several cohorts of patients will be undergoing surgeries utilizing different modalities and specialties. In particular, the hospital administration implements a package of clinical services and discounts for the tourist-friendly spring-summer months, and this generates both more routine requests as well as several other minor and major surgeries that stretch operating room capacity during sunny weather. The system monitors the colonoscopies for post-procedural complications through an automated procedure that commonly dismisses patients and embraces patients with probable complications requiring further diagnostics [37].

Figure 8

Artificial Intelligence in Critical Care Medicine

7.1. Successful Implementations in Healthcare Systems

The global scale and cost burden of health system complications will likely continue to widen gaps between the needs and capacities of individual systems. Failure to act with sufficient force or speed on these threats could bring devastating consequences to millions of individuals, and much more broadly to societies and economies as a whole. While the dramatic scenarios make compelling stories, they are not the reason that system leaders are so focused on detecting and containing complications earlier and more effectively than they have done in the past. Many problems in health systems exist, but monitoring is a first step toward mitigation.

In contrast to all the other terrible things that can happen to people, complications reflect weaknesses or breakdowns in the capabilities and functions of a defined system and its social purpose. There are data and measures-led processes underway at multiple health systems worldwide. Approaches involve a combination of significant planned investment in new governance, technology, workforce, and community responses to complications, based on collective learning from and actively involving patients. Underlying these ongoing transformations and reflections is an appreciation that unwelcome outcomes have multiple antecedents, progress, and sustain them. As well as the challenge of timely and accurate detection and diagnosis, the best responses for health systems that are increasingly sophisticated in the care they provide to patients are essential for ensuring that it is implemented in a safe and patient-centered manner that achieves desired outcomes within available resources.