The NVIDIA Clara developer kit, NVIDIA Clara Holoscan, and us4us front end help build AI models on streaming data for ultrasounds, to remove artifacts like aliasing.
At RSNA 2021, there are dedicated tracks on ultrasound imaging, which is a cost-effective way to see what is going on inside a patient’s body without exposure to radiation or the need for injections and surgeries.
Ultrasound imaging is typically done by trained sonographers and needs special expertise to interpret. The probe is a small transducer to both transmit sound waves into the body and record the waves that echo back. It is placed on the skin and as it moves, waves bounce off your blood cells, organs, and other body parts, and then back to the device. A computer then takes all the sound waves and turns them into moving images that you visualize on a screen.
The LITMUS group (Laboratory on Innovative Technology in Medical Ultrasound) at the University of Waterloo, Canada is working on making ultrasound color doppler imaging (CDI) easier to visualize. They used the NVIDIA Clara Holoscan platform, including the NVIDIA Clara AGX Developer Kit and the NVIDIA Clara Holoscan SDK, along with frontend us4us, to remove aliasing artifacts and increase the frame rate 12-fold- from 2 fps to 30 fps.
- Clara Holoscan is an AI platform that includes strong deep learning compute ability that can run a model at high frame rates. The Clara Holoscan SDK is designed to facilitate the creation of AI pipelines for the processing of real-time streaming medical data for ultrasound, video, and other imaging applications.
- The Clara AGX Developer Kit combines the power of an NVIDIA RTX 6000 GPU controlled by an NVIDIA AGX Xavier SoC, with external connectivity provided by two PCIe Gen4 x 8 slots, and a NVIDIA ConnectX-6 SmartNIC with a 100 GbE port.
Color Doppler imaging
Color Doppler imaging (CDI) is a non-invasive way to see blood flow in arteries and veins. It is used to identify a blockage, blood clot, or narrowing of the arteries that can lead to deadly clinical outcomes such as a stroke or heart attack. These blockages can occur in a variety of arteries in the body and significantly alter the properties of blood flow. The flow alterations can be captured by CDI and used in the identification and monitoring of diseased conditions. CDI can also be used for detecting aneurysms, where swollen artery walls can also impact blood flow.
Figure 2 shows a typical CDI sequence obtained from a carotid artery model where flow comes in from the left side of the image, then branches out into the upper and lower branches. Flow speed in the artery is shown in shades of blue or red, depending on its direction relative to the probe. The surrounding grayscale image shows the tissue structure. The CDI sequence also shows how blood flow dynamics can change throughout the cardiac cycle, which is typically less than a second long.
Aliasing problems in CDI
One recurring issue in CDI is the presence of so-called aliasing artifacts that hinder the visualization of blood flow. Aliasing artifacts occur when blood flow exceeds the maximum flow speed measurable by the CDI system.
For example, Figure 3 shows that flow in the upper branch is fast and exceeds the maximum measurable flow speed on the color scale (25 cm/s). The color chosen for this region is therefore picked from the opposite end of the color scale and incorrectly indicates that flow is going in the opposite direction. The maximum measurable speed stems from underlying system limitations and imaging considerations.
Aliasing is most problematic in tortuous vasculature such as bifurcations and in conditions where a wide range of multidirectional velocities are encountered. CDI in such conditions can become difficult to interpret.
Novel deep learning–based solution
The LITMUS group devised a new deep learning–based solution to address these aliasing artifacts in CDI for the femoral artery bifurcation. The femoral artery bifurcation in the thigh was chosen due to its diverse flow properties, including a wide range of flow speeds and multidirectional flow. The artery can be a site of blockage in conditions of peripheral artery disease and would be susceptible to aliasing in the bifurcation, even in healthy conditions.
To address aliasing artifacts in CDI, the LITMUS group devised a two-step process:
- Aliasing artifacts in CDI are segmented using a convolutional neural network (CNN) model.
- The segmented aliasing artifacts are subsequently removed by an adaptive technique.
For the aliasing segmentation, a U-Net CNN was trained to detect aliasing artifacts using several relevant ultrasound features that are often computed in typical CDI pipelines and can contain features that are relevant for aliasing detection. The network was trained on 1,136 frames obtained from three real femoral artery bifurcation acquisitions using a us4us ultrasound frontend. The aliasing artifacts in CDI were manually labelled for training and validation. The model definition and training were done in TensorFlow.
The segmentation maps were then leveraged by an adaptive phase unwrapping algorithm that reverses the aliasing artifact according to flow continuity criteria so that a smooth aliasing-free flow profile is achieved. The framework was then evaluated on a new acquisition from an unseen femoral artery bifurcation acquisition, where it was shown to deal with multidirectional and excessive aliasing.
The framework was computationally demanding, requiring more than 500 ms per frame for simple de-aliasing, and even slower for excessive aliasing cases.
Clara Holoscan for real-time de-aliasing in bedside applications
CDI is widely expected to be a point-of-care modality that can be used to gain quick and immediate insights into blood flow conditions in patients. Offline processing would disrupt this utility of CDI, so it is important that the aliasing removal framework be run in real time.
NVIDIA and the LITMUS group collaborated to accelerate the de-aliasing framework to achieve real-time performance that would be suitable in a bedside application, using the NVIDIA Clara Holoscan SDK and the NVIDIA Clara AGX Developer kit.
A GPU-accelerated CDI platform was implemented on the NVIDIA Clara AGX Developer Kit using a CUDA-based framework previously reported by LITMUS. For more information, see Live Ultrasound Color-Encoded Speckle Imaging Platform for Real-Time Complex Flow Visualization In Vivo.
Raw sensor data is continuously copied to the NVIDIA RTX 6000 GPU in the Clara AGX developer kit where custom CUDA-built kernels perform the necessary processing for image formation. The pretrained U-Net TensorFlow model was implemented using the Tensor RT API and the adaptive phase unwrapping algorithm was accelerated using the CUDA-NPP library. Further CUDA and OpenGL functions were used for display. The result was a complete raw-sensor-data-to-de-aliased-CDI package that was run on the Clara AGX Developer Kit with demonstrated real-time performance.
Figure 5 shows the aliasing resistant CDI framework in action on the Clara AGX developer kit, processing raw sensor data from a femoral bifurcation model to aliasing resistant CDI in live mode. The raw data was acquired using the us4us frontend, which gives researchers access to all the fundamental signals as they arrive from the probe:
- Left: The aliased CDI sequence is obtained using a conventional processing pipeline. At systole (peak of the cardiac cycle, frozen frame), flow is moving away from the probe and should all be blue. In the bottom branch, however, the flow speed in the direction of the probe exceeds the maximum measurable and therefore appears as a red/orange shaded region that incorrectly suggests flow is going up.
- Middle: Aliasing segmentation is obtained by the integrated U-Net model during the live imaging session. You can see how the aliasing artifact in the systolic frame is correctly captured on-site.
- Right: The CDI sequence has the aliasing removed. The maximum measurable speed is increased and the visualization of blood flow is made more intuitive.
The processing time of the de-aliasing module was improved to 30 fps, a 12x improvement from the previous 2-2.5 fps. In building up to this, CuPy was used to prototype and get quick GPU acceleration, giving an intermediate ~15 fps.
Conclusion
The LITMUS group’s workflow showed how the NVIDIA Clara AGX Developer Kit and the NVIDIA Clara Holoscan SDK can resolve aliasing artifacts in CDI, in real time. Removing aliasing makes image visualization and interpretation easier by removing the ambiguity about the blood flow direction. This makes the most impact in tortuous vasculature where flow direction can be difficult to guess by the sonographer.
For more information, see the following resources: