Why the FDA Is Getting Strict on AI

Last week, the Food and Drug Administration (FDA) released its first artificial intelligence/machine learning action plan to provide oversight to this type of growing technology in the medical field.

While we’ve seen AI used in many different sectors, such as the food industry, the security space, and the retail industry, the technology is still largely unregulated. However, experts think that this rapidly evolving technology will provide massive benefits to both doctors and patients—if regulated under guidelines. 

"Regulating AI is not just a good thing, but essential to protect patients and build trust in this promising technology," wrote Sam Surette, head of regulatory affairs and quality assurance at Caption Health, to Lifewire in an email.

"FDA oversight ensures that products in the field have demonstrated safety and effectiveness through rigorous performance testing."

FDA’s Approach To AI

The FDA’s plan centers around transparency and performance monitoring to provide AI medical devices an assurance of safety and effectiveness. 

Some of the action plan's key components include supporting regulatory science efforts to develop a methodology for evaluating and improving machine learning algorithms (including identifying and eliminating bias).

The plan also lays out how the FDA intends to figure out appropriate machine learning development practices, work with stakeholders who are piloting the process, and more. 

"The plan outlines a holistic approach based on total product lifecycle oversight to further the enormous potential that these technologies have to improve patient care while delivering safe and effective software functionality that improves the quality of care that patients receive," Bakul Patel, director of the Digital Health Center of Excellence in the Center for Devices and Radiological Health, said in the FDA’s press release. 

Patel added that the action plan would change over time, as new applications for the technology are released and improved upon.

"To stay current and address patient safety and improve access to these promising technologies, we anticipate that this action plan will continue to evolve over time," he said.

Surette said that perhaps the most important aspect of the FDA's blueprint for regulating AI is this "predetermined change control plan," which will allow for modifications over time. 

"The use of these predetermined change control plans allows the FDA to review possible modifications ahead of time, obviating the need for a new premarket review before each significant algorithm update," he wrote. 

Joe Mullings, chairman and CEO of The Mullings Group, said the FDA's AI framework couldn't come at a better time, since AI algorithms are always changing.

"AI uses algorithms, and algorithms are still a black art of sorts, so having a gatekeeper like the FDA is critical," he told Lifewire in a phone interview. "[AI] used in the proper ways will impact healthcare at a dramatic scale that we’ve never seen before."

Impacts Of More AI In Healthcare

Experts say there are pros and cons to AI in healthcare, and that the FDA needs to factor in both to ensure proper regulation. 

As far as benefits go, Mullings said regulations of AI in med-tech would allow solutions to individual patient care to happen even quicker. 

"We are taking these world-class clinicians, doctors, and surgeons, and we are able to give them a higher level of information for them to make their decisions; taking an already great clinician, doctor, or surgeon and making him or her even greater," he said. 

He added that doctors could decide on patients' behalf with a higher accuracy rate than they otherwise would. 

With the FDA providing guidance, that means it’ll be easier to do things like use AI to read radiology images, acquire and interpret ultrasound exams, and even speed the process of detecting COVID-19 in hospital patients by analyzing routine blood samples. 

However, there are other implications when it comes to AI and its human-created algorithms. Mullings pointed to AI’s problems with bias, especially towards people of color. 

"We're running into issues in clinical trials that are not taking the full complete look at people of color and are over-indexing towards the white population," Mullings said.

"Algorithms are equations written by a human being that will have an unconscious bias in it. That built-in bias then amplifies that potential fault, and after a while, a doctor is making blind decisions with it." 

Still, the hope is that this bias and other hurdles of AI, such as the long involved process of collecting and labeling data, will decrease significantly with these new regulations. 

"The digital health community has been appreciative of FDA's proactive approach to AI," Surette wrote. "This AI Action Plan released by FDA represents the next step in right-sizing regulatory oversight of this relatively nascent field, which should be encouraging to those in the med-tech community."