Artificial intelligence has been used to spot bias in healthcare, such as a lack of darker skin tones in dermatologic educational materials, but AI has been the cause of bias itself in some cases.

When AI bias occurs in healthcare, the causes are a mix of technical errors as well as real human decisions, according to Dr. Marshall Chin, professor of healthcare ethics in the Department of Medicine at the University of Chicago. Chin co-chaired a recent governmentpanelon AI bias.

This is something that we have control over, Chin tells InformationWeek. It's not just a technical thing that is inevitable.

In 2023, a class action lawsuit accused UnitedHealth of illegally using an AI algorithm to turn away seriously ill elderly patients from care under Medicare Advantage. The lawsuit blamed naviHealths nH Predict AI model for inaccuracy. UnitedHealth told StatNews last year that the naviHealth care-support tool is not used to make determinations. The lawsuit has no merit, and we will defend ourselves vigorously, the company stated.

Other cases of potential AI bias involved algorithms studying cases of heart failure, cardiac surgery, and vaginal birth after cesarean delivery (VBAC), in which an AI algorithm led Black patients to get more cesarean procedures than were necessary, according to Chin. The algorithm erroneously predicted that minorities were less likely to have success with a vaginal birth after a C-section compared with non-Hispanic white women, according to the US Department of Health and Human Services Office of Minority Health. It inappropriately had more of the racial minority patients having severe cesarean sections as opposed to having the vaginal birth, Chin explains. It basically led to an erroneous clinical decision that wasn't supported by the actual evidence base.

Related:Why AIs Slower Pace in Healthcare Is as It Should Be

After years of research, the VBAC algorithm was changed to no longer consider race or ethnicity when predicting which patients could suffer complications from a VBAC procedure, HHS reported.

When a dataset used to train an AI system lacks diversity, that can result in misdiagnoses, disparities in healthcare, and unequal insurance decisions on premiums or coverage," explains Tom Hittinger, healthcare applied AI leader at Deloitte Consulting.

If a dataset used to train an AI system lacks diversity, the AI may develop biased algorithms that perform well for certain demographic groups while failing others, Hittinger says in an email interview. This can exacerbate existing health inequities, leading to poor health outcomes for underrepresented groups.

Related:Metaverse: The Next Frontier in Healthcare?

Although AI tools can cause bias, they also bring more diversity to drug development. Companies such as BioPhy study patterns in patient populations to see how people respond to different types of drugs.

The challenge is to choose a patient population that is broad enough to offer a level of diversity but also bring drug efficacy. However, designing an AI algorithm to predict patient populations may result in only a subset of the population, explains Dave Latshaw II, PhD, cofounder ofBioPhy.

If you feed an algorithm that's designed to predict optimal patient populations with only a subset of the population, then it's going to give you an output that only recommends a subset of the population, Latshaw tells InformationWeek. You end up with bias in those predictions if you act on them when it comes to structuring your clinical trials and finding the right patients to participate.

Therefore, health IT leaders must diversify their training sets when teaching an AI platform to avoid blindness in the results, he adds.

The dream scenario for somebody who's developing a drug is that they're able to test their drug in nearly any person of any background from any location with any genetic makeup that has a particular disease, and it will work just the same in everyone, Latshaw says. That's the ideal state of the world.

Related:Connected Healthcare Takes Huge Leap Forward

IT leaders should involve a diverse group of stakeholders when implementing algorithms. That involves tech leaders, clinicians, patients, and the public, Chin says.

When validating AI models, IT leaders should include ethicists and data scientists along with clinicians, patients, and associates, which are nonclinical employees, staff members, and contractual workers at a healthcare organization, Hittinger says. When multiple teams roll out new models, that can increase the time required for experimentation and lead to a gradual rollout along with continuous monitoring, according to Hittinger.

That process can take many months, he says.

Many organizations are using proprietary algorithms, which lack an incentive to be transparent, according to Chin. He suggests that AI algorithms should have labels like a cereal box explaining how algorithms were developed, how patient demographic characteristics were distributed, and the analytical techniques used.

That would give people some sense of what this algorithm is, so this is not a total black box, Chin says.

In addition, organizations should audit and monitor AI systems for bias and performance disparities, Hittinger advises.

Organizations must proactively search for biases within their algorithms and datasets, undertake the necessary corrections, and set up mechanisms to prevent new biases from arising unexpectedly, Hittinger says. Upon detecting bias, it must be analyzed and then rectified through well-defined procedures aimed at addressing the issue and restoring public confidence.

Organizations such as Deloitte offer frameworks to provide guidance on how to maintain ethical use of AI.

One core tenet is creating fair, unbiased models and this means that AI needs to be developed and trained to adhere to equitable, uniform procedures and render impartial decisions, Hittinger says.

In addition, healthcare organizations can adopt automated monitoring tools to spot and fix model drift, according to Hittinger. He also suggests that healthcare organizations form partnerships with academic institutions and AI ethics firms.

Dr. Yair Lewis, chief medical officer at AI-powered primary-care platform Navina, recommends that organizations establish a fairness score metric for algorithms to ensure that patients are treated equally.

The concept is to analyze the algorithms performance across different demographics to identify any disparities, Lewis says in an email interview. By quantifying bias in this manner, organizations can set benchmarks for fairness and monitor improvements over time.

Original post:
How AI Bias Is Impacting Healthcare - InformationWeek

Theres a new startup in San Francisco, and theyre into marketing. Alone, that wouldnt be so interesting, but this company has made its business about algorithmic marketing. This is 2006, in the days when Facebook was still new and generative artificial intelligence was a conversation among science fiction fans. But this startup is sure that machine learning is the future.

The startup secures a major client, a local tourist board, and is eager to turn this into an early win. So they put their heads together and burned the midnight oil. Why would you come to San Francisco? Who should they target, and how? The team came up with the old, familiar answers: Alcatraz, Golden Gate Bridge, and wine country. But then they turned to their algorithm to measure the decision-making behaviors of potential San Francisco visitors.

Their systems buzzed and whirred and they came back with dentistry. At first, the CEO thought it was a mistake. Dentistry? San Franciscans have nice teeth, no doubt, but they arent famous for them. Then the team realized something: the annual Dentistry Association conference was about to be held in San Francisco. No one had known. No one had even thought of that. But the machine did. The team whipped up a presentation for their client, and everyone thought it was great.

Try BigThink+ for your business

Engaging content on the skills that matter, taught by world-class experts.

Thats the true story of one of Quantcasts earliest successes. Today, Quantcast is one of the biggest AI-driven advertising technology companies in the world. They have offices in ten countries around the world with hundreds of employees. They were one of the few to recognize the power of AI before the rest of the world did. Last month, Big Think talked with Quantcast founder and CEO, Konrad Feldman, to make sense of how AI has changed marketing and will continue to revolutionize business.

Almost everyone in the world uses some kind of bank. There are billions of transactions occurring every second, and each of those transactions represents a behavior. A boy is buying a chocolate bar from a vending machine. A pensioner is moving money between pension accounts. A young couple is putting down a deposit on a house. A businessman is giving a generous tip. Money moves, and it tells us a story. We can often tell more about peoples behavior from their spending habits than from any self-recorded account.

The problem, then, is how to make sense of those stories. How can anyone hope to read the habits of billions of microtransactions? I started my career in research in neural networks at UCL, Feldman tells Big Think. I started a business using machine learning techniques to find patterns predominantly in the financial service industry. I ended up working in lots of different business domains where it was really hard for people to interpret all of the data and find the patterns of activity that they needed to address particular business problems. We ended up building a lot of systems in the banking sector to help the big banks detect things like money laundering and terrorist financing.

Having fine-tuned various algorithms in the financial sector, Feldman looked to move into the online space; Quantcast was focused on dot-com entrepreneurs just starting out. But now there was a problem. In the days before cloud computing and the internet of everything, the internet was a web of tiny pockets and gated communities. Quantcast Measure is the product we launched in 2006 to help websites of any size, explains Feldman. It was a classic sort of disruptive innovation initially. These algorithms have the ability to provide much broader visibility to better understand audiences. Over time, larger and larger sites realized the richness of audience information they could derive, and participate in machine learning. That was the first use of Quantcast Measure.

Marketers need data. But everyone has their own data sets, often very small. Quantcast offered the opportunity to combine them and analyze them. We had to solve some really thorny technical issues, says Feldman, both in terms of the sort of data processing architectures we needed and because the data volumes involved are just absolutely immense.

Its often misunderstood that machine learning and AI are some kind of data-processing panacea. A computer can do billions of calculations per nanosecond, so surely they can make light work of all that data? But an algorithm is only as good as the humans who engineer it. As Feldman puts it, A lot of people, when they think about AI or machine learning, think of it as some sort of holistic algorithm that does everything. In fact, even though we have algorithms now in large language models (LLMs) that can actually do a number of different things, fundamentally, most specific problems require specific algorithms and specific tuning and application of those algorithms.

The various explosions of GPTs are a good example of this. OpenAI has produced one of the most powerful AIs in the history of civilization, but after a few months, its usage slipped, and people started to see it as a time waste. Then, at the end of 2023, OpenAI introduced custom-made GPTs. Essentially, anyone could fine-tune and create an algorithm of their own using OpenAIs LLMs. Users started to craft prompts in an algorithmic style: Imagine youre a movie critic, In the style of a romantic poet, You are Albert Einstein, Explain like Im five years old, and so on. This is an example of the fine-tuning Feldman talks about to turn machine learning into something practical because machine learning varies.

We believe it will be possible to build machine learning models that could actually infer audience characteristics.

The machine learning we use to measure audiences is different from the machine learning were using to build predictive audience models for advertising delivery, explains Feldman. Its different from the machine learning thats been used to ensure brand safety or to price individual impressions. Weve developed different types of machine learning approaches to solve some of these fundamental problems. Quantcasts marketing algorithms will capture information in real time [and] then make adjustments as they go along. They are interactive tools that will optimize for some specified goal as theyre working.

The clear implication here is that the next stage of AI will be more flexible, fine-grained and fine-tuned. For AI to become really useful and for the conversation to become really exciting AI has to become personalized AI.

We believe it will be possible to build machine learning models that could actually infer audience characteristics, says Feldman. Humans are really good [at] pattern matching thats how weve evolved. But machines also have flexibility because they can deal with so many dimensions of data. If youre going adjust and tailor the content, if youre going to modify content for each individual recipient, well then, people cant do that. You have to use a computer to do it.

Read the original post:
Why the CEO of Quantcast is betting on "personalized AI" - Big Think

Netflix recently published how its Machine Learning Platform (MLP) team provides an ecosystem around Metaflow, an open-source machine learning infrastructure framework. By creating various integrations for Metaflow, Netflix is able to support hundreds of Metaflow projects maintained by multiple engineering teams.

Metaflow's integrations with Netflix's production systems enable projects to move from prototype to production without incurring unsustainable operational overhead. The engineering team explains their key to success:

Given the very diverse set of ML and AI use cases we support [...], we don't expect all projects to follow the same path from prototype to production. Instead, we provide a robust foundational layer with integrations to our company-wide data, compute, and orchestration platform, as well as various paths to deploy applications to production smoothly. On top of this, teams have built their own domain-specific libraries to support their specific use cases and needs.

One integration example provided is the "Fast Data" library for Metaflow. Netflix hosts its main data lake on S3 as Apache Iceberg tables and uses Apache Spark for ETL. The Fast Data library enables fast, scalable, and robust access to the Netflix data warehouse by leveraging high-performance components from the Python data ecosystem. This library allows Netflix toprocessterabytes of data collectively and encode complex relationships between titles, actors, and other film attributes, supporting the company's broad business applications.

The Fast Data Library for Metaflow (Source)

Netflix's production workflow orchestrator,Maestro, plays a critical role in managing Metaflow projects in production. It supports scalability and high availability and enables seamless integration of Metaflow flows with other systems through event-triggering. Using this integration, Netflix engineers can support content decision-making and answer "what content Netflix should bring to the service".

Finally, for deployments that require an API and real-time evaluation, Netflix provides an integrated model hosting service, Metaflow Hosting. Metaflow Hosting "provides an easy to use interface on top of Netflix's existing microservice infrastructure, allowing data scientists to quickly move their work from experimentation to a production grade web service that can be consumed over a HTTP REST API with minimal overhead."

Using this integration, Netflix hosts and scales a model to compute various media asset features. Once it finds the features, a consuming service can store them in a store for future use. An old talk provides an overview and further details about this service.

Hosting and Consuming the Media Feature Computation Model (Source)

Netflix implemented the integrations using Metaflow's extension mechanism, "which is publicly available but subject to change and hence not part of Metaflow's stable API yet." They invite engineers to contact them on the Metaflow community Slack to discuss building additional extensions.

Read more:
Netflix Uses Metaflow to Manage Hundreds of AI/ML Applications at Scale - InfoQ.com

To a human ear the songs of all male zebra finches sound more or less the same. But faced with a chorus of this simple song, female finches can pick the performer who sings most beautifully.

Zebra finches are found in Australia, and they usually mate monogamously for life making this a high-stakes decision for the female finches. The zebra finch is among about a third of songbirds who learn a single song from their fathers early in life, and sing it over and over, raising the question of how female songbirds distinguish between them to choose a mate.

Listen to the song of a male zebra finch:

Scientists believe that most male songbirds evolved to sing a variety of songs to demonstrate their fitness. Under that theory, the fittest songbirds will have more time and energy to work on their vocal stylings and attract females with their varied vocal repertoire.

New research using machine learning shows finches may be sticking to one tune, but how they sing it makes a big difference. Published Wednesday in the journal Nature, the study reveals the complexity of a single zebra finch song and what female songbirds might be hearing in their prospective mates seemingly simple songs.

When researchers analyze birdsongs, theyre often not listening to them but rather looking at spectrograms, which are visualizations of audio files.

So I put together that, Hey, what humans are doing is looking at images of these audio files. Can we use machine learning and deep learning to do this? said Danyal Alam, the lead author on the new study and a postdoctoral researcher at the University of California at San Francisco.

Alam, along with Todd Roberts, an associate professor at UT Southwestern Medical Center and another colleague, used machine learning to analyze hundreds of thousands of zebra finch songs to figure out how they were different from each other and which variations were more attractive to female songbirds.

The researchers found one metric that seemed to get females attention: the spread of syllables in the song. The females seemed to prefer longer paths between syllables. This isnt something humans can easily pick up by listening to the songs or looking at the spectrograms but based on how these algorithms mapped the syllables, the researchers were able to see them in a new way.

To check their hypothesis, the researchers brought the findings back to the birds.

They generated synthetic bird songs to see if females preferred those with a longer path and they did, suggesting the birds intended audience picked up on the same pattern as the researchers computers.

Listen to see if you can tell the difference between a synthetic finch song that doesnt spread out its syllables:

Alam and his colleagues also found that baby birds had a harder time learning the long-distance song patterns than the shorter ones which suggests fitter birds would be more able to learn them, the researchers said.

The studys finding is consistent with whats been shown in other species: The more complexity or difficulty in a song, the more appealing it is to female birds.

A lot of signals in animal communication are meant to be an honest signal of some underlying quality, said Kate Snyder, a researcher at Vanderbilt who wasnt involved in the new paper.

For example, she said, if you look at a peacock, you see the male birds with the longer and more beautiful tails are better at attracting mates. Maintaining a tail like that is expensive for the bird so it must be good at finding food and surviving in its environment to have the time to devote to keeping its tail nice.

Learning takes a lot of time, energy, brain space, Snyder said. Only the fittest male birds will have the time and energy to devote to learning to sing.

Among finches, that work has just been harder to spot until now.

We used to think of this single song repertoire as perhaps a simple behavior, said Roberts. But what we see is that its perhaps much more complicated than we previously appreciated.

Read more:
AI reveals the complexity of a simple birdsong - The Washington Post

Posted by Yossi Matias, VP Engineering & Research, and Grey Nearing, Research Scientist, Google Research

Floods are the most common natural disaster, and are responsible for roughly $50 billion in annual financial damages worldwide. The rate of flood-related disasters has more than doubled since the year 2000 partly due to climate change. Nearly 1.5 billion people, making up 19% of the worlds population, are exposed to substantial risks from severe flood events. Upgrading early warning systems to make accurate and timely information accessible to these populations can save thousands of lives per year.

Driven by the potential impact of reliable flood forecasting on peoples lives globally, we started our flood forecasting effort in 2017. Through this multi-year journey, we advanced research over the years hand-in-hand with building a real-time operational flood forecasting system that provides alerts on Google Search, Maps, Android notifications and through the Flood Hub. However, in order to scale globally, especially in places where accurate local data is not available, more research advances were required.

In Global prediction of extreme floods in ungauged watersheds, published in Nature, we demonstrate how machine learning (ML) technologies can significantly improve global-scale flood forecasting relative to the current state-of-the-art for countries where flood-related data is scarce. With these AI-based technologies we extended the reliability of currently-available global nowcasts, on average, from zero to five days, and improved forecasts across regions in Africa and Asia to be similar to what are currently available in Europe. The evaluation of the models was conducted in collaboration with the European Center for Medium Range Weather Forecasting (ECMWF).

These technologies also enable Flood Hub to provide real-time river forecasts up to seven days in advance, covering river reaches across over 80 countries. This information can be used by people, communities, governments and international organizations to take anticipatory action to help protect vulnerable populations.

The ML models that power the FloodHub tool are the product of many years of research, conducted in collaboration with several partners, including academics, governments, international organizations, and NGOs.

In 2018, we launched a pilot early warning system in the Ganges-Brahmaputra river basin in India, with the hypothesis that ML could help address the challenging problem of reliable flood forecasting at scale. The pilot was further expanded the following year via the combination of an inundation model, real-time water level measurements, the creation of an elevation map and hydrologic modeling.

In collaboration with academics, and, in particular, with the JKU Institute for Machine Learning we explored ML-based hydrologic models, showing that LSTM-based models could produce more accurate simulations than traditional conceptual and physics-based hydrology models. This research led to flood forecasting improvements that enabled the expansion of our forecasting coverage to include all of India and Bangladesh. We also worked with researchers at Yale University to test technological interventions that increase the reach and impact of flood warnings.

Our hydrological models predict river floods by processing publicly available weather data like precipitation and physical watershed information. Such models must be calibrated to long data records from streamflow gauging stations in individual rivers. A low percentage of global river watersheds (basins) have streamflow gauges, which are expensive but necessary to supply relevant data, and its challenging for hydrological simulation and forecasting to provide predictions in basins that lack this infrastructure. Lower gross domestic product (GDP) is correlated with increased vulnerability to flood risks, and there is an inverse correlation between national GDP and the amount of publicly available data in a country. ML helps to address this problem by allowing a single model to be trained on all available river data and to be applied to ungauged basins where no data are available. In this way, models can be trained globally, and can make predictions for any river location.

Our academic collaborations led to ML research that developed methods to estimate uncertainty in river forecasts and showed how ML river forecast models synthesize information from multiple data sources. They demonstrated that these models can simulate extreme events reliably, even when those events are not part of the training data. In an effort to contribute to open science, in 2023 we open-sourced a community-driven dataset for large-sample hydrology in Nature Scientific Data.

Most hydrology models used by national and international agencies for flood forecasting and river modeling are state-space models, which depend only on daily inputs (e.g., precipitation, temperature, etc.) and the current state of the system (e.g., soil moisture, snowpack, etc.). LSTMs are a variant of state-space models and work by defining a neural network that represents a single time step, where input data (such as current weather conditions) are processed to produce updated state information and output values (streamflow) for that time step. LSTMs are applied sequentially to make time-series predictions, and in this sense, behave similarly to how scientists typically conceptualize hydrologic systems. Empirically, we have found that LSTMs perform well on the task of river forecasting.

Our river forecast model uses two LSTMs applied sequentially: (1) a hindcast LSTM ingests historical weather data (dynamic hindcast features) up to the present time (or rather, the issue time of a forecast), and (2) a forecast LSTM ingests states from the hindcast LSTM along with forecasted weather data (dynamic forecast features) to make future predictions. One year of historical weather data are input into the hindcast LSTM, and seven days of forecasted weather data are input into the forecast LSTM. Static features include geographical and geophysical characteristics of watersheds that are input into both the hindcast and forecast LSTMs and allow the model to learn different hydrological behaviors and responses in various types of watersheds.

Output from the forecast LSTM is fed into a head layer that uses mixture density networks to produce a probabilistic forecast (i.e., predicted parameters of a probability distribution over streamflow). Specifically, the model predicts the parameters of a mixture of heavy-tailed probability density functions, called asymmetric Laplacian distributions, at each forecast time step. The result is a mixture density function, called a Countable Mixture of Asymmetric Laplacians (CMAL) distribution, which represents a probabilistic prediction of the volumetric flow rate in a particular river at a particular time.

The model uses three types of publicly available data inputs, mostly from governmental sources:

Training data are daily streamflow values from the Global Runoff Data Center over the time period 1980 - 2023. A single streamflow forecast model is trained using data from 5,680 diverse watershed streamflow gauges (shown below) to improve accuracy.

We compared our river forecast model with GloFAS version 4, the current state-of-the-art global flood forecasting system. These experiments showed that ML can provide accurate warnings earlier and over larger and more impactful events.

The figure below shows the distribution of F1 scores when predicting different severity events at river locations around the world, with plus or minus 1 day accuracy. F1 scores are an average of precision and recall and event severity is measured by return period. For example, a 2-year return period event is a volume of streamflow that is expected to be exceeded on average once every two years. Our model achieves reliability scores at up to 4-day or 5-day lead times that are similar to or better, on average, than the reliability of GloFAS nowcasts (0-day lead time).

Additionally (not shown), our model achieves accuracies over larger and rarer extreme events, with precision and recall scores over 5-year return period events that are similar to or better than GloFAS accuracies over 1-year return period events. See the paper for more information.

The flood forecasting initiative is part of our Adaptation and Resilience efforts and reflects Google's commitmentto address climate change while helping global communities become more resilient. We believe that AI and ML will continue to play a critical role in helping advance science and research towards climate action.

We actively collaborate with several international aid organizations (e.g., the Centre for Humanitarian Data and the Red Cross) to provide actionable flood forecasts. Additionally, in an ongoing collaboration with the World Meteorological Organization (WMO) to support early warning systems for climate hazards, we are conducting a study to help understand how AI can help address real-world challenges faced by national flood forecasting agencies.

While the work presented here demonstrates a significant step forward in flood forecasting, future work is needed to further expand flood forecasting coverage to more locations globally and other types of flood-related events and disasters, including flash floods and urban floods. We are looking forward to continuing collaborations with our partners in the academic and expert communities, local governments and the industry to reach these goals.

Excerpt from:
Using AI to expand global access to reliable flood forecasts - Google Research