Using AI to get augmented reality ready for prime time

Using AI to get augmented reality ready for prime time

UROtech24 is a meeting that showcases the latest in technology in urology in its widest sense. By combining advanced 3D-reconstruction with imaging technology and AI, surgeons will soon be able to see hidden anatomical and tumour details, projected onto the organs they are operating on and constantly corrected for manipulation or positioning.

Prof. Francesco Porpiglia (Turin, IT) has been working in this field in recent years, and foresees a bright future for this kind of innovation. At UROtech24, there will be several demonstrations of this technology. Prof. Porpiglia will be chairing and speaking in a session on imaging and navigation in urology, together with ESUT Chair Prof. Ali Gözen (Ostfildern, DE).

We spoke to Prof. Porpiglia about the latest developments in augmented reality and how artificial intelligence will further aid surgeons in their operations.

UROtech24 will be held in Vienna on 25-27 January 2024 and is a joint meeting of the EAU Sections of Uro-Technology and Urolithiasis (ESUT and EULIS) in collaboration with the EAU Robotic Urology Section (ERUS) and the EAU Working group on Paediatric Urology (EWPU). Registration is currently open, with the early fee available until 26 October 2023.

Augmented reality

“I certainly think augmented reality (AR) will be next big thing, the next big technique in urology’s future,” says Prof. Porpiglia. “It allows us to better navigate inside the body than with the imaging technology we have now. Highly-detailed tumour models can be projected inside the body, allowing us to see the hidden anatomy.”

“My team and I started working with high-fidelity 3D reconstruction, this is the very important first step. Only with a highly-accurate reconstruction can you use superimposition. Overlapping anatomy allows you to better explore the vascular anatomy and the tumour, especially when it’s located inside the kidney.”

“This also means that we don’t need to use ultrasound. We can use this technology to help us remove tumours but also for things like selective clamping of arteries.”

Augmented reality overlays will integrate with existing and future robotic systems, acting as an additional software interface between the surgeon and the robot. A second surgeon currently helps with the projection. The technology can also be used in a laparoscopic context.

Improvements in AI

Will the use of augmented reality become a popular addition to Europe’s operating theatres?

“It certainly has a lot of potential but there are some limitations. There’s the cost of modelling, and when surgical teams first start using it, there is a bit of a learning curve. You need a team that knows how to work with the technology, and you need a surgeon that’s dedicated to developing this technology inside the department. You also need an assistant to guide the surgery, for as long as the projection is a manual process.”

“Improvements in AI will automate the overlapping. We are already working in this direction, there are several studies with automatic overlapping for robotic-assisted radical prostatectomy. I anticipate that this product will become available in only a matter of months. From there, we expect to see more diffusion as costs come down and as we complete studies and degrees of evidence go up.”

Urology and technology

UROtech is a meeting that brings together the EAU’s most technologically-minded Sections: ESUT and EULIS (technology and urolithiasis, respectively), supported by ERUS (robotics) and EWPU (paediatric urology). The meeting takes a wide look at technology in urology, incorporating imaging, surgery and most recently AI.

“I think this is an essential meeting for urologists,” says Prof. Porpiglia. “We have complex and sophisticated technology at our disposal and I think it’s in the nature of urologists to push this technology. Just as an example, in my department in Turin we currently have four different robotic platforms: two for surgery, and two more for BPH and stones. If we start combining surgical robots with energy-based instruments for stone treatment, and even automatisation through AI, that opens up a lot of new applications.”

We spoke to Prof. Porpiglia in Florence, Italy, at the occasion of ERUS23, the EAU’s robotic urology meeting. Prof. Porpiglia’s insights are reflected in the words of ERUS Chair Prof. Alberto Breda (Barcelona, ES) on the role of robotics in general and ERUS’s involvement in UROtech24:

“Of course the EAU Robotic Urology Section will be contributing to UROtech24. Urologists, particularly those working in urolithiasis are dealing more and more with robotics. ERUS will be providing some speakers and sharing our expertise in Vienna. We are naturally complementary and it’s a good collaboration between the Sections.”