Platform for investigating continuum manipulator behavior in orthopedics

Introduction

Use of a robotic continuum manipulator (CM) has been proposed for minimally invasive orthopedic surgical procedures such as those in the pelvis [1], femur [2], and knee [3, 4]. Features such as physical compliance and increased dexterity make CMs favorable, but can also present localization challenges during interactions with the surgical environment. As orthopedic procedures often involve direct manipulation of bone, robust control strategies need to be developed. A simulator can facilitate the development of such strategies as early testing on physical hardware can be slow to iterate and may damage equipment. Two key gaps were identified in the direct application of existing tools to investigate the use of CMs in orthopedic procedures. First, the representation of a CM and patient bone together in a physics environment where they can interact was not straightforward. Second, the existing implementations of volumetric drilling did not generalize to representing non-homogeneous materials or interaction with non-rigid drills. By addressing these gaps, we present a tool that can simulate key behaviors of CM-to-bone interactions and intra-operative X-ray images of the surgical scene while simultaneously interacting with existing robot control software infrastructure.

There are simulation tools that have previously been used to model CM behavior or CM interactions with an environment. However, these implementations do not lend themselves well to orthopedic applications as they often utilize mesh-based collision or are focused on intrinsically soft robotics interacting with deformable material (e.g., [5–7]). In orthopedics, the relevant patient anatomy is typically more rigid and requires substantial manipulation and removal. This makes volumetric representations more natural and efficient for orthopedics applications than the mesh representations often used for soft tissue applications. Additionally, volumetric representations have strong synergy with radiological images, facilitating paired generation of intra-operative images with the physics simulation.

Prior works have established a general framework for simulation of surgical robotic systems [8], which has previously been used for simulation of volumetric drilling in the skull base [9]. Our tool is provided as an open-source plugin that extends this framework. New capabilities include a real-time physics-based simulation of CM behavior and significant enhancements to existing volumetric drilling behavior. Interactions between tools and bone volume models are now done using a physics-based method, and the voxel-level representation of these interactions better represents a continuum of bone material properties derived from patient computed tomography (CT) images. Together, these contributions and improvements allow for representation of collision, drilling, and tunneling in bone by CMs and enable several emergent behaviors seen when manipulating real materials such as the ‘skiving’ behavior of a burr off of less machinable materials. We further augment physical simulation with generation of digitally reconstructed radiograph (DRR) images (i.e., simulated intra-operative X-ray) of the scene, which can be used to improve or validate strategies for registration (e.g., [10]) and intra-operative image-based control.

Methods

The simulation platform is developed as a plugin to the asynchronous multi-body framework (AMBF) [8]. A tendon-driven notched nitinol CM with planar bending as used in [10] is modeled by segmenting the structure at each flexure element center, creating 27 links connected by 6DOF spring joints with stiffness set to match finite element estimates (Fig. 1). The CM is mounted to a UR5 (Universal Robots, Denmark) to perform gross kinematic movement and insertion. Actuation of the CM is achieved via a force at the tendon attachment. The entire system can be controlled using the popular robot operating system (ROS) [11] middleware and as such can be transparently operated by the same control code used for a matching physical system setup.

The CM can interact with any volume model. Here, a bone segmentation volume was generated from a CT image via an intensity threshold. Extending capabilities in [12], this CT image and the simulation state are used to produce a DRR of the surgical scene. Unlike previous implementations which have used a position-set algorithm to achieve these interactions [9], here the physics-based sequential impulse method is used to simulate collisions between the tool and anatomy. This is a critical contribution as the position-set algorithms are only effective for interactions between a volume model and a single rigid-body. Trying to combine these position-set algorithms with force-driven behaviors (e.g., a robot holding a drill or the internal forces in a CM) results in numerical instability that can render the simulation unusable. By implementing the interactions using an impulse based approach, we remove the instabilities associated with these position-set algorithms. Impulse-based approaches were introduced to physics simulations in [13] and have been used in other applications to solve these instability issues. We encourage interested readers to find detailed mathematical descriptions of the method at this reference. Tool-to-voxel interactions occur between the volume and proxy contact spheres that are located on the CM including a bounding sphere around each of the CM segments, one coincident with the burr, and several spaced along the rigid shaft. The volume contact surfaces can be chosen by the user as either bounding spheres around each voxel in the volume, or a plane defined by the instantaneous contact normal. Behavior demonstrated here uses the sphere-to-plane contact.

In the naive implementation of volumetric drilling, all voxels in collision with the burr at a given timestep will be removed. To allow for differences in burr speed to impact material clearance rate, [9] allowed for the number of voxel removals to be capped within a given timestep and modulated this value proportionally to burr speed. However, this implementation still limited interactions to occur with a homogeneous material. To maintain the tool-driven behavior while further capturing the material variability within bone as well as patient-specific properties, we introduce a new method for representing low-level volumetric drilling behavior that uses two parameters: (1) initial voxel machinability and (2) a running value for the ‘strength’ of each voxel. Here, machinability is taken to be proportional to bone density which has previously been shown to correlate to the Hounsfield unit values in a CT [14]. Initial voxel ‘strength’ is set lower for voxels with higher machinability (i.e., the easier it is to drill, the lower this value is initially set). When the burr is in contact with a voxel, that voxel will have its ‘strength’ value reduced at each physics update by a value proportional to the current burr speed. When this ‘strength’ value reaches zero, the voxel is considered drilled and will no longer generate collisions or appear in DRRs. The removal of drilled voxels from DRRs allows cut paths to be visible as areas of lower intensity (Fig. 2).

Critically, this implementation results in high-level material-driven and tool-driven behaviors that emerge directly from low-level properties and interactions, rather than by direct manual programming. As an example, we show replication of burr skiving behavior that occurs in real systems (Fig. 3). Burr skiving is when a burr travels laterally from its insertion direction and may occur during drilling if the insertion rate is faster than the material clearance rate and lateral material is absent or more easily machined. For traditional drilling applications, this is often more relevant at surfaces, but it becomes important to model throughout when using CMs as lateral deviations of the tip during insertion can lead to large deviations in trajectory. As such, it is important to be able to capture this type of behavior in simulations that will be used to develop control algorithms. This behavior cannot be demonstrated using the naive implementation of volumetric drilling or the method in [9] as demonstrated in a simple planar representation in Fig. 4.

Results

Simulated curved drilling of a femoral bone was performed with a physics rate of 1000Hz and rendered in real time with a graphics rate of 120Hz using a Precision 3650 Tower (Dell, USA) with a Xeon W-1370P CPU (Intel, USA) and a RTX A4000 (Nvidia, USA) GPU. DRR images were generated and displayed at a rate of 1.5 Hz. Comparison of real CM behavior during curved drilling in phantom bone (Sawbones, USA) under fluoroscopy was compared to behavior in the simulation (Fig. 3). In a limited sample, tip location deviation was within 3 mm after insertion. Critically, the high-level behavior of the interaction (e.g., tunneling and skiving) matched that of the real system.

Discussion and conclusion

The behavior of the simulated CM corresponds to the behavior of a real CM interacting with phantom bone material. Modeling interaction with volumetric elements of varying machinability allows for stable simulation of cutting curved paths and burr skiving. Generation of simulated X-ray images that update based on the bone’s manipulation shows promise for developing image-guided control methods.

In development of such a platform, the ability to generate high-level behaviors such as tunneling and skiving is of more initial interest than the numerical fidelity of the simulation to a real setup as tuning of simulation parameters for specific tools or procedures is common practice. However, 3-mm tip deviation would be appropriate for targeting of osteonecrotic lesions in the femoral head, which was the clinical task that inspired the curved drilling demonstration. Further, as these high-level behaviors emerge implicitly from the newly defined low-level improvements, the proposed enhancements provide useful modes for users to get the fidelity they require for a given tool–procedure pair. The impact of tool-specific and material-specific properties on drilling behavior operate cohesively, but can now be modulated and tuned separately by the user. Further, as the material-specific properties are directly inferred from patient images, any tuning or adjustments are made en masse and automatically incorporate patient-specific properties.

In future, researchers may be able to further support this effort by considering the use of system identification or learning-based methods to best set or adapt these parameters given a set of empirical data from phantoms and cadaver models as well as clinical datasets. An exceptionally well-tuned model, sometimes referred to as a ‘digital twin’ may be useful for intra-operative visualization or control of the real system. Further, improvements in volumetric drilling behavior can be directly included into works such as [9] to investigate the impact on simulation fidelity in haptics-enabled scenarios. In its current form, this tool will be used to develop image-guided autonomous control for curved drilling in orthopedic procedures. The tool is provided open source for the community at (https://github.com/htp2/continuummanip-volumetric-drilling-plugin).