Why smarter technology doesn’t automatically mean better healthcare

Your medical device now has AI. But is it actually any better?

It’s a question more MedTech leaders should be asking.

Artificial intelligence is rapidly becoming part of the medical device development conversation. From connected diagnostics and remote monitoring to surgical systems and personalised therapies, the potential is considerable.

But there’s a danger.

In the rush to integrate AI, are we making medical devices genuinely better, or simply making them more complicated?

A poorly designed device doesn’t become a good one because an algorithm has been added.

A confusing interface is still confusing.

An uncomfortable wearable is still uncomfortable.

A complicated workflow is still complicated.

And a device that clinicians don’t want to use won’t suddenly become indispensable because it can generate predictions.

AI can make a medical device more intelligent. It cannot automatically make it more useful.

For MedTech leaders, that distinction could determine which AI-enabled products succeed and which become expensive development exercises.

The AI feature trap

There is growing pressure on MedTech companies to demonstrate an AI strategy.

Investors want to see innovation. Competitors are announcing AI-enabled products. Development teams are exploring new capabilities, and commercial teams are looking for differentiation.

The temptation is understandable.

If AI can analyse more data, recognise more patterns and generate more recommendations, surely that makes the product better?

Not necessarily.

Consider a connected patient monitoring device.

It already collects heart rate, activity and physiological data. Adding AI could allow it to identify patterns associated with patient deterioration.

That sounds valuable.

But what happens if it generates frequent alerts that clinicians cannot interpret or act upon?

What happens if patients receive notifications that create anxiety without providing meaningful guidance?

What happens if the algorithm performs well, but the device itself is uncomfortable, difficult to position or unreliable in everyday use?

We’ve improved the technology.

But have we improved the healthcare experience?

The critical question isn’t whether AI can be integrated.

It’s whether it should be.

1. Start with the clinical problem, not the algorithm

The most successful medical devices generally begin with a clearly defined problem.

A clinical task that takes too long.

A procedure that introduces unnecessary risk.

A patient who struggles to manage treatment.

A diagnostic process that is too expensive or inaccessible.

A clinician who needs better information to make a decision.

AI should be evaluated against those problems, not treated as the starting point.

The World Health Organization has emphasised that AI in healthcare should be designed around clearly defined purposes, with appropriate attention to safety, effectiveness, human autonomy and accountability.

That is an important distinction.

Rather than asking, “Where can we use AI?”, development teams should ask:

What problem are we trying to solve?

Who experiences that problem?

How significant is it?

What would a genuinely better outcome look like?

Is AI actually the best way to achieve it?

Sometimes the answer will be yes.

Sometimes a simpler interface, better ergonomics or a redesigned workflow will deliver greater value at considerably lower cost.

And sometimes the best innovation decision is not to use AI at all.

2. More information doesn’t necessarily mean better decisions

One of AI’s greatest strengths is its ability to process large volumes of information.

But healthcare professionals rarely suffer from a shortage of data.

They suffer from a shortage of time and attention.

A device that generates more information without making it easier to interpret can simply add to the problem.

Research into clinical decision support has repeatedly identified alert fatigue as a challenge, particularly where systems generate excessive or poorly targeted notifications.

The issue is not simply whether an algorithm can identify something important.

It’s whether the information reaches the right person, at the right time, in a form that supports an appropriate action.

Consider two approaches to an AI-enabled diagnostic device.

The first provides a detailed dashboard containing multiple scores, probability estimates, graphs and trend indicators.

The second presents a clear recommendation, relevant supporting information and an appropriate indication of uncertainty.

Both may use the same underlying algorithm.

But the experience is very different.

Which is more likely to be adopted in a busy clinical environment?

The answer depends on the clinical task and the level of oversight required, but the principle remains.

Good design translates complexity into clarity without concealing information users need to make safe decisions.

AI should reduce cognitive workload, not redistribute it.

3. Human factors become even more important when AI is involved

Medical device usability has always been critical.

With AI, the challenge becomes more complex.

Traditional devices generally respond to relatively predictable inputs and user actions.

AI-enabled systems may produce outputs based on probabilistic models, changing data and complex relationships that are not immediately apparent to the user.

That introduces important questions.

How does a clinician know when to trust a recommendation?

How should the system communicate uncertainty?

When should a human intervene?

Can users recognise an incorrect output?

What happens when the system encounters unfamiliar conditions?

How easily can a recommendation be challenged or overridden?

The FDA’s guidance on applying human factors and usability engineering to medical devices focuses on reducing use-related risks and ensuring devices can be used safely and effectively by their intended users.

For AI-enabled products, these principles become particularly important.

The interface must support appropriate reliance on the system.

Too little trust and users may ignore valuable information.

Too much trust and they may accept recommendations without sufficient scrutiny.

Neither is desirable.

The objective isn’t to make AI appear infallible.

It’s to help people understand what the system can do, where its limitations lie and when their judgement remains essential.

That requires human factors expertise from the beginning of development, not simply usability testing before launch.

4. AI cannot compensate for poor physical product design

Much of the AI conversation focuses on software.

But many medical devices are physical products operating in demanding real-world environments.

Wearables must be comfortable.

Diagnostic equipment must be intuitive.

Home-use devices must be easy to operate correctly.

Rehabilitation technologies must accommodate different physical capabilities.

Connected devices must integrate into existing clinical workflows.

AI cannot solve every underlying physical design problem.

Imagine a sophisticated AI-enabled wearable capable of detecting clinically relevant changes in physiological signals.

The algorithm performs exceptionally well in development.

But patients find the device uncomfortable and stop wearing it.

Or the sensor is difficult to position correctly.

Or battery life is inadequate.

Or the device generates inconsistent data because of movement or poor contact.

The result?

An intelligent system that cannot reliably deliver its intended benefit.

If the physical product fails, the intelligence behind it becomes irrelevant.

This is why industrial design, mechanical engineering, electronics, software and human factors need to be developed as an integrated system.

Not as separate workstreams brought together at the end.

5. Regulation is asking increasingly difficult questions

AI-enabled medical devices introduce regulatory considerations that go beyond those associated with many conventional products.

The FDA’s January 2025 draft guidance on AI-enabled device software functions adopts a Total Product Lifecycle perspective, addressing issues including development, validation, transparency, performance monitoring and change management.

Its guidance on Predetermined Change Control Plans also provides a framework through which manufacturers can seek authorisation for specified future modifications to AI-enabled device software.

In Europe, the AI Act introduces additional requirements for certain high-risk AI systems, including those associated with regulated medical devices where the relevant conditions apply.

These developments reinforce an important point.

Regulatory compliance is not simply about demonstrating that an algorithm performs accurately.

Manufacturers must consider the safety and performance of the complete device within its intended use.

That includes how people interact with the technology.

Can users understand the output?

Can they recognise when intervention is required?

Can the system be used safely under foreseeable conditions?

How will performance be monitored once the product is in use?

The answers can influence everything from interface architecture to product documentation, risk management and post-market processes.

The earlier these questions are addressed, the easier it becomes to design a coherent product.

6. Adoption is the ultimate test of innovation

A medical device can be technically impressive, clinically promising and appropriately regulated.

But if it doesn’t fit into the way healthcare is delivered, adoption may still be limited.

Healthcare professionals are already operating under significant pressure.

New technologies compete for their time, attention and budgets.

Introducing AI may create additional demands.

Training.

Workflow changes.

IT integration.

Data governance.

Cybersecurity.

Procurement.

Ongoing maintenance.

Clinical accountability.

These are not peripheral considerations.

They influence whether a product is adopted and whether its benefits can be realised.

The NHS AI Lab’s work on AI deployment and the NICE Evidence Standards Framework for Digital Health Technologies both reinforce the importance of evaluating technologies in their intended healthcare context.

For MedTech leaders, this raises an uncomfortable commercial question.

Are we developing something that healthcare professionals genuinely need, or something that we believe they should want?

There is a difference.

A successful AI-enabled product must do more than demonstrate technical performance.

It must earn its place in clinical practice.

7. The biggest opportunity may be making AI invisible

Perhaps the most interesting future for AI in MedTech is one in which users barely notice it.

Not because the technology is unimportant.

But because it has been integrated so effectively that the product simply becomes easier to use.

A rehabilitation device that automatically adjusts to a patient’s progress.

A diagnostic system that highlights the most relevant information without overwhelming the clinician.

A connected monitoring device that identifies meaningful changes while minimising unnecessary alerts.

A home-use device that guides patients through a procedure and helps prevent foreseeable errors.

In each case, AI is valuable because it improves the experience or outcome.

Not because it is visible.

The best AI-enabled medical device may be the one that doesn’t feel like an AI product at all.

It simply works better.

8. Five questions MedTech leaders should ask before adding AI

Before committing significant resources to AI integration, leadership teams should challenge their development programmes with five questions.

1. What specific clinical or user problem will AI solve?

Can the benefit be clearly articulated and measured?

2. Will it make the product simpler or more complicated?

Are we reducing workload, improving decisions or merely adding functionality?

3. Have we designed for real-world users and environments?

Do we understand the physical, cognitive and practical challenges associated with using the device?

4. Can we demonstrate meaningful value?

Will the technology improve outcomes, efficiency, accessibility or another benefit that customers recognise?

5. Would the product still be compelling without AI?

If the answer is no, is AI genuinely creating the value, or are we using it to compensate for weaknesses elsewhere?

These questions may appear straightforward.

But answering them early could save considerable development time and investment.

Better products, not just smarter technology

At Maddison, we believe the most exciting opportunities in MedTech emerge when new technologies are combined with a deep understanding of people, products and the environments in which they operate.

That means challenging assumptions.

Exploring alternative approaches.

Building and testing ideas quickly.

Bringing together industrial design, engineering, software and human factors expertise.

And understanding how an innovative concept becomes a practical, manufacturable and commercially viable medical device.

For established MedTech organisations, that sometimes requires a different way of working.

A focused team of experienced specialists who can step outside existing product architectures, explore difficult problems and develop solutions without being constrained by the demands of day-to-day programmes.

Not innovation for its own sake.

Innovation that delivers something useful.

Because the next generation of successful medical devices won’t necessarily be the ones with the most sophisticated AI.

They’ll be the ones that make healthcare work better.

One final question

If you removed the AI from your latest medical device concept, would you still have a product that clinicians and patients wanted to use?

If not, perhaps the first challenge isn’t artificial intelligence.

It’s product design.

Maddison. Design That Means Business.

References

1. World Health Organization, Ethics and Governance of Artificial Intelligence for Health (2021).
Principles covering human autonomy, safety, transparency, accountability and equity in healthcare AI.
https://www.who.int/publications/i/item/9789240029200

2. FDA, Applying Human Factors and Usability Engineering to Medical Devices (2016).
Guidance on reducing use-related risks and designing devices for safe and effective use.
https://www.fda.gov/regulatory-information/search-fda-guidance-document…

3. FDA, Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations (2025, draft guidance).
Recommendations for managing AI-enabled medical devices throughout their lifecycle.
https://www.fda.gov/regulatory-information/search-fda-guidance-document…

4. FDA, Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions (2025).
Guidance on planned modifications to AI-enabled device software.
https://www.fda.gov/regulatory-information/search-fda-guidance-document…

5. European Union, Regulation (EU) 2024/1689, Artificial Intelligence Act.
Regulatory requirements for AI systems, including certain high-risk applications associated with medical devices.
https://eur-lex.europa.eu/eli/reg/2024/1689/oj

6. NICE, Evidence Standards Framework for Digital Health Technologies.
Framework for evaluating evidence relating to digital health technologies.
https://www.nice.org.uk/corporate/ecd7

7. Ancker JS et al. Effects of Workload, Work Complexity, and Repeated Alerts on Alert Fatigue in a Clinical Decision Support System. BMC Medical Informatics and Decision Making. 2017;17:36.
Research examining the relationship between clinical workload, repeated alerts and alert fatigue.
https://doi.org/10.1186/s12911-017-0430-8

8. ISO 14971:2019, Medical Devices: Application of Risk Management to Medical Devices.
International standard covering medical device risk management.

9. IEC 62366-1:2015+A1:2020, Medical Devices: Application of Usability Engineering to Medical Devices.
International standard covering usability engineering and use-related risk.