How Physical AI, augmented reality and surgeon-guided autonomy could redefine the relationship between human expertise and intelligent machines.

Imagine an operating room where the surgeon is supported by an interconnected network of intelligent systems.

A surgical robot assists with precision-intensive tasks. An AI system continuously interprets the surgical environment. Wearable displays provide contextual information without interrupting the surgeon's workflow. Intelligent instruments communicate their position and operational status.

The system recognises the stage of the procedure, anticipates relevant transitions and makes information available when it matters.

The surgeon remains in control, but no longer has to manage every aspect of the technological environment independently.

This is one possible vision of the intelligent operating room.

And it raises a fundamental question:

What should the relationship between human expertise and machine intelligence look like in the future of surgery?

Beyond the replacement narrative

Discussions around artificial intelligence and robotics frequently revolve around automation and replacement.

Surgery deserves a different conversation.

Surgical expertise is not simply a sequence of mechanical movements.

It involves judgment, adaptability, interpretation, decision-making and the ability to respond to unexpected situations.

Even highly experienced surgeons continuously navigate uncertainty.

The challenge is therefore not simply to build machines that can replicate human movements.

It is to develop intelligent systems that can complement human expertise.

The most meaningful applications of surgical AI may initially involve reducing cognitive and operational burdens rather than replacing the surgeon.

This could include:

• Contextual information retrieval.

• Anatomical recognition and visualisation.

• Procedural awareness.

• Instrument and workflow monitoring.

• Surgical documentation.

• Predictive alerts.

• Decision-support capabilities.

Over time, carefully validated systems could progress towards performing increasingly complex and well-defined tasks under appropriate human oversight.

The emergence of surgeon-guided autonomy

An interesting development in this direction is the emergence of surgeon-guided autonomy.

In November 2026, Zimmer Biomet introduced its Surgeon-Guided Autonomy vision, organised around three connected capabilities:

Predict. Perform. Recover.

The framework connects patient-specific planning, execution of defined surgical tasks and postoperative recovery into a continuous learning loop.

Its underlying principle is particularly interesting: transfer the burden of surgery without transferring the surgeon's control.

This represents a broader shift in how the industry thinks about autonomy.

Rather than viewing autonomy as a binary condition in which a machine either operates independently or remains entirely human-controlled, it can be approached as a progression of capabilities.

Different tasks can have different levels of automation, depending on their complexity, risk and clinical validation.

A system might autonomously process information, recommend an action, assist with execution or perform a narrowly defined task under supervision.

The surgeon's role evolves alongside these capabilities.

Physical AI: Bringing intelligence into the operating room

Physical AI extends artificial intelligence beyond screens and digital interfaces into systems that interact with the physical world.

Surgical robotics is a natural domain for this development.

However, physical interaction requires more than the ability to execute movements.

An intelligent system must understand its surroundings, interpret instructions, account for changing conditions and respond appropriately.

In surgery, this involves several interconnected capabilities:

1. Perception: Recognising anatomy, instruments, people and the operative environment.

2. Contextual understanding: Interpreting procedural stages, actions and relationships.

3. Prediction: Anticipating changes and evaluating possible consequences.

4. Planning: Translating clinical objectives into defined actions.

5. Execution: Performing actions within validated operational boundaries.

6. Feedback: Monitoring results and adapting within permitted limits.

This progression is particularly important because the operating room is not a static environment.

Anatomy varies. Tissue deforms. Visibility changes. Instruments interact with complex biological structures.

Reliable physical AI must account for these conditions.

XR: The interface between human and machine intelligence

Another important component of the future operating room could be extended reality.

While surgical robotics changes how physical tasks are performed, XR has the potential to change how surgeons perceive and interact with information.

Imagine a surgeon having access to relevant patient information, anatomical references, procedural guidance and contextual intelligence within their field of view.

Rather than repeatedly looking away from the surgical field to consult external displays, surgeons could potentially interact with information through hands-free interfaces.

Possible applications include:

• Contextual surgical visualisation.

• Patient-specific anatomical overlays.

• Access to imaging and clinical information.

• Remote expert collaboration.

• Procedural guidance.

• Real-time visual assistance.

The distinction is important.

XR should not simply introduce another display into an already crowded operating room.

Its value lies in making relevant information accessible at the point of need.

Combined with multimodal surgical intelligence, XR could become an important interface between the surgeon and the intelligent operating room.

The operating room as a continuous learning system

Perhaps the most transformative possibility is the creation of a continuous learning loop.

Today, surgical planning, intraoperative execution and postoperative recovery are often supported by different systems and information sources.

Connecting these stages could create opportunities to understand the relationship between surgical decisions, procedural execution and patient outcomes.

Consider the potential:

Before surgery

Patient-specific information informs planning and preparation.

During surgery

Intelligent systems capture and interpret procedural activity, supporting the surgeon with relevant contextual information.

After surgery

Clinical outcomes and recovery information contribute to evaluating the procedure and informing future planning.

Over time, this could create a richer feedback loop between surgical experience and future clinical practice.

However, establishing such a system requires careful attention to data quality, clinical validation, privacy, interoperability and accountability.

The objective should always be improved patient care rather than automation for its own sake.

What will the surgeon of tomorrow look like?

The surgeon of tomorrow may operate in an environment where many activities that currently demand manual attention are supported by intelligent systems.

Documentation may become increasingly automated.

Anatomical information may be contextualised and visualised dynamically.

Procedural deviations may be recognised earlier.

Robotic systems may undertake defined tasks with appropriate supervision.

XR interfaces may provide access to information without disrupting surgical workflow.

But the surgeon's expertise will remain central.

In fact, as surgical systems become more capable, the importance of defining clinical intent, supervising machine behaviour and exercising judgment may become even greater.

The relationship will evolve from human operators using technological instruments towards humans and intelligent systems working together.

Building towards this future

At Nuevata, we see this transition as an opportunity to rethink how surgical intelligence is developed.

We believe that capturing the surgeon's perspective, structuring multimodal surgical information and developing systems capable of understanding procedural context are important foundations for this evolution.

Our work with IRIS and MIRA begins with the challenge of making surgical activity more observable, structured and understandable.

The broader vision extends towards contextual intelligence, predictive capabilities and interfaces that can bring relevant information closer to the surgeon.

We do not believe the future of surgery will be defined by a single device, algorithm or robotic platform.

It will emerge through the convergence of surgical expertise, multimodal AI, robotics, connected medical devices and human-machine interfaces.

The objective is not to build technology that takes the surgeon out of the operating room.

It is to build technology that enables surgeons to do more, understand more and focus their expertise where it matters most.

The future of surgery is not human versus machine. It is human intelligence amplified by machine intelligence.