Health care organizations, like many other enterprises, face steep challenges in their attempt to maximize operational efficiency in the face of resource constraints. Whether it is a hospital’s attempt to optimize staffing or a government trying to fairly allocate and distribute limited doses of Covid-19 vaccines, these tasks can be formidable. A promising way to manage the complexity is to enlist data-driven analytics and artificial intelligence (AI).
However, such techniques, while powerful, can also mask problematic underlying ethical assumptions or lead to morally questionable outcomes. Consider a recently published study about models used by some of the most technologically advanced hospitals in the world to help prioritize which patients with chronic kidney disease should receive kidney transplants. It found that the models discriminated against black patients: “One-third of Black patients … would have been placed into a more severe category of kidney disease if their kidney function had been estimated using the same formula as for white patients.” While it is just the latest of many studies to show the deficiencies of such models, it is unlikely to be the last.
Can AI and analytics be used in a way that improves operational efficiency without jeopardizing our ethical principles? The answer is “yes” — if moral objectives and constraints, now often treated as an afterthought, are considered from the outset when designing models. We will discuss a recent attempt to combine ethics, analytics, and operational efficiency in the world of organ allocation and examine the lessons it holds for other areas of health care and beyond.
Insight Center
A key challenge marrying ethics and efficiency is to have the underlying models strike a balance across multiple objectives. Some of these objectives pertain to efficiency alone (e.g., years of life expectancy, lives saved, quality of life improvements, total costs) and often can be formulated mathematically. Others relate to ethical dimensions of the allocation decisions and can be either hard to describe or defined in multiple, mutually inconsistent ways. In the case of allocating organs for transplantation, besides different efficiency metrics that policymakers care about, there is a multitude of possibly conflicting fairness considerations that need to be accounted for so that patients are not discriminated based on a long list of factors, such as race, sex, age, and geographic location.
Current practice handles allocation by employing a scoring formula that awards points to candidates based on numerous different (permissible) criteria such as wait time, medical urgency, proximity to the donor, and so on. Deciding on how many points to award per criterion, however, is a daunting task: Tweaking the number of points for one criterion might improve outcomes along some dimensions but worsen them among many others, often in unpredictable ways. To come up with a model that carries out the desired policy, organizations usually produce a number of scoring formulas and test them by conducting simulations. The most promising are then deployed in the real world and if found to disadvantage a patient group, are refined. In other words, fairness analysis is done ex post, and discriminatory effects are identified on the go.
However, considering ethical obligations so late in the game makes it harder to achieve morally acceptable outcomes for three reasons:
- The execution of decisions, such as organ allocation, entails processes, people, and other resources that may be difficult to alter once deployed.
- Since people may have little tolerance for discrimination or other unfair outcomes, ex-post ethical assessment may lead to the outright rejection of models and health care solutions.
- Not considering ethics upfront limits aspirations from the outset — or, to put it another way, reviewing implementations ex post facto can bias perceptions about what is achievable in practice.
An ideal process starts with understanding all of the dimensions of the goal and then trying to find the best way to achieve all of them to the greatest degree possible. It is therefore imperative to develop models and data analytics processes that, from the start, are trying to balance all the objectives, including the ethical considerations.
For the allocation of organs for transplants, this would mean before even beginning to build the AI algorithms or analytics models, you would give ethicists a “clean slate” to articulate what allocation outcomes are considered fair. For example, what percent of organs shall be offered to Black or female patients? Once these questions are addressed and there is a North Star on the horizon, then leverage data, AI, and analytics to design a policy that hits or at least comes as close as possible to these targets.
For example, one of us has worked on an online tool that uses this approach to facilitate policy design for the allocation of lungs. The interface of the tool utilizes “sliders” that allow the parties responsible for designing the policy to specify how the important efficiency and ethics metrics shall be changed vis-à-vis the status-quo. Then, by employing simulation, machine learning, and mathematical optimization, the tool produces a conforming allocation policy — i.e., how many points to award for each day of being on the waiting list, for each extra life year saved, for each mile that the candidate is closer, and so on.
This “ethics by design” type of approach we’ve described, attempts to strike a kind of “reflective equilibrium” (to use a term associated with the philosopher John Rawls) in which ethicists and those who lead institutions begin by identifying a set of candidate considerations that appear to matter (in the case of organs, this may be prognosis, life expectancy, and so on). Analytics are then used to model different ways of trading off these considerations and what the results would be. The result of that analysis is presented and considered by all relevant stakeholders, which leads to revision of either the list of considerations or their weighting, which leads to a new round of analysis, and so on. When stakeholders says, “I think you are putting too much of a premium on age,” the “sliders can be adjusted” to show them exactly what would happen if the trade-off is changed, which may prompt more discussion and (hopefully) more understanding.
The organ allocation example also holds lessons for the rollout of approved Covid-19 vaccines over the coming months. Authorities will face an important allocation and distribution challenge: Given that it will take months for production and distribution capacity to be able to meet the huge demand, countries around the world will need to decide who can be vaccinated first. The kinds of tools and principles we’ve discussed can assist.