Categories
Misc

Jumpstarting Link-Level Simulations with NVIDIA Sionna

Sionna simulates physical layer and link-level research.Sionna is a GPU-accelerated open-source library for link-level simulations.Sionna simulates physical layer and link-level research.

Even while 5G wireless networks are being installed and used worldwide, researchers in academia and industry have already started defining visions and critical technologies for 6G. Although nobody knows what 6G will be, a recurring vision is that 6G must enable the creation of digital twins and distributed machine learning (ML) applications at an unprecedented scale. 6G research requires new tools.

Holographic mimo, reconfigurable intelligent surfaces and AI-native air interface are key emerging technologies in 6G.
Figure 1. 6G key technologies

Some of the key technologies underpinning the 6G vision are communications at the high frequencies known as the Terahertz band. In this band, more spectrum is available by orders of magnitude. Technology examples include the following:

  • Reconfigurable intelligent surfaces (RIS) to control how electromagnetic waves are reflected and achieve the best coverage.
  • Integrated sensing and communications (ISAC) to turn 6G networks into sensors, which offers many exciting applications for autonomous vehicles, road safety, robotics, and logistics. 

Machine learning is expected to play a defining role for the entire 6G protocol stack, which may revolutionize how we design and standardize communication systems.

Addressing the research challenges of these revolutionary technologies requires a new generation of tools to achieve the breakthroughs that will define communications in the 6G era. Here is why:

  • Many 6G technologies require the simulation of a specific environment, such as a factory or cell site, with a spatially consistent correspondence between physical location, wireless channel impulse response, and visual input. This can currently only be achieved by either costly measurement campaigns, or by efficient simulation based on a combination of scene rendering and ray tracing.  
  • As machine learning and neural networks become increasingly important, researchers would benefit tremendously from a link-level simulator with native ML integration and automatic gradient computation.
  • 6G simulations need unprecedented modeling accuracy and scale. The full potential of ML-enhanced algorithms will only be realized through physically-based simulations that account for reality in a level of detail that has been impossible in the past. 

Introducing NVIDIA Sionna

To address these needs, NVIDIA developed Sionna, a GPU-accelerated open-source library for link-level simulations. 

Sionna enables rapid prototyping of complex communication system architectures. It’s the world’s first framework that natively enables the use of neural networks in the physical layer and eliminates the need for separate toolchains for data generation, training, and performance evaluation. 

Sionna implements a wide range of carefully tested, state-of-the-art algorithms that can be used for benchmarking and end-to-end performance evaluation. This lets you focus on your research, making it more impactful and reproducible while you spend less time implementing components outside your area of expertise. 

Sionna is written in Python and based on TensorFlow and Keras. All components are implemented as Keras layers, which lets you build sophisticated system architectures by connecting the desired layers in the same way you would build a neural network. 

Apart from a few exceptions, all components are differentiable so that gradients can be back-propagated through an entire system. This is the key enabler for system optimization and machine learning, especially the integration of neural networks. 

NVIDIA GPU acceleration provides orders-of-magnitude faster simulations and scaling to large multi-GPU setups, enabling the interactive exploration of such systems. If no GPU is available, Sionna even runs on the CPU, though more slowly.

Sionna comes with rich documentation and a wide range of tutorials that make it easy to get started. 

Forward error correction, channel models, multiuser mimo and OFDM are features of Sionna at release.
Figure 2. Features of Sionna’s first release

The first release of Sionna has the following major features:  

  • 5G LDPC, 5G polar, and convolutional codes, rate-matching, CRC, interleaver, scrambler 
  • Various decoders: BP variants, SC, SCL, SCL-CRC, Viterbi 
  • QAM and custom modulation schemes 
  • 3GPP 38.901 Channel Models (TDL, CDL, RMa, UMa, Umi), Rayleigh, AWGN 
  • OFDM 
  • MIMO channel estimation, equalization, and precoding 

Sionna is released under the Apache 2.0 license, and we welcome contributions from external parties.

Hello, Sionna!

The following code example shows a Sionna “Hello, World!” example in which the transmission of a batch of LDPC codewords over an AWGN channel using 16QAM modulation is simulated. This example shows how Sionna layers are instantiated and applied to a previously defined tensor. The coding style follows the functional API of Keras. You can open this example directly in a Jupyter notebook on Google Collaboratory.

batch_size = 1024
n = 1000 # codeword length
k = 500 # information bits per codeword
m = 4 # bits per symbol
snr = 10 # signal-to-noise ratio

c = Constellation("qam",m,trainable=True)
b = BinarySource()([batch_size, k])
u = LDPC5GEncoder (k,n)(b)
x = Mapper (constellation=c)(u)
y = AWGN()([x,1/snr])
11r = Demapper("app", constellation=c)([y,1/snr])
b_hat = LDPC5GDecoder(LDPC5GEncoder (k, n))(11r)

One of the key advantages of Sionna is that components can be made trainable or replaced by neural networks. NVIDIA made Constellation trainable and replaced Demapper with a NeuralDemapper, which is just a neural network defined through Keras.

c = Constellation("qam",m,trainable=True)
b = BinarySource()([batch_size, k])
u = LDPC5GEncoder (k,n)(b)
x = Mapper (constellation=c)(u)
y = AWGN()([x,1/snr])
11r = NeuralDemapper()([y,1/snr])
b_hat = LDPC5GDecoder(LDPC5GEncoder (k, n))(11r)

What happens under the hood is that the tensor defining the constellation points has now become a trainable TensorFlow variable and can be tracked together with the weights of NeuralDemapper by the TensorFlow automatic differentiation feature. For these reasons, Sionna can be seen as a differentiable link-level simulator.

Looking ahead

Soon, Sionna will allow for integrated ray tracing to replace stochastic channel models, enabling many new fields of research. Ultra-fast ray tracing is a crucial technology for digital twins of communication systems. For example, this enables the co-design of a building’s architecture and the communication infrastructure to achieve unprecedented levels of throughput and reliability. 

Pseudo-code blocks surrounding a simulated image.
Figure 3. Access the power of hardware-accelerated ray tracing from within a Jupyter notebook

Sionna takes advantage of computing (NVIDIA CUDA cores), AI (NVIDIA Tensor Cores), and ray tracing cores of NVIDIA GPUs for lightning-fast simulations of 6G systems.

We hope you share our excitement about Sionna, and we look forward to hearing about your success stories!

For more information, see the following resources:

Categories
Misc

New Sensor Partners Expand Surgical, Ultrasound, and Data Acquisition Capabilities in the NVIDIA Clara Holoscan Platform

NVIDIA Clara Holoscan offers an expanded selection of third-party interface options for video capture, ultrasound research, data acquisition, and connection to legacy medical devices.

New advances in computation make it possible for medical devices to automatically detect, measure, predict, simulate, map, and guide clinical care teams. NVIDIA Clara Holoscan, the full-stack AI computing platform for medical devices, has added new sensor front-end partners for video capture, ultrasound research, data acquisition, and connection to legacy-medical devices. 

Clara Holoscan currently consists of developer kits with an accompanying Clara Holoscan SDK for developing AI models. Announced today at GTC, Clara Holoscan MGX, the medical-grade platform for building software-defined medical devices, will be available in 2023 for production-ready deployment

With nine front-end partners now supported on Clara Holoscan, medical device developers can add AI capabilities that augment human interpretation, maximize efficiency, and reduce error.

Powering low latency streaming for surgical video applications

NVIDIA has partnered with several leading video capture card manufacturers to provide the software driver support for these cards to be inserted into the PCI Express slots in the Clara AGX and Clara Holoscan Developer Kits. In addition, these capture cards will support the NVIDIA GPUDirect technology, which uses remote direct memory access to transfer video data directly from the capture card to GPU memory.

AJA Video Systems provides high-quality video I/O devices for professional video applications. The Corvid and Kona series of SDI and HDMI video capture cards are supported on Clara Developer Kits. The partnership between NVIDIA and AJA has led to the addition of Clara AGX Developer Kit support in the AJA NTV2 SDK and device drivers as of the NTV2 SDK 16.1 release. 

KAYA Instruments offers CoaXPress and CameraLink video capture cards for connecting a wide array of scientific camera solutions and electron microscopes. KAYA Instruments capture cards are supported on the Clara AGX Developer Kit with an upcoming version of the Kaya Instruments driver software.

Deltacast designs and produces a range of cost-effective video capture cards for use in the broadcast video, industrial, aerospace, and medical markets. Deltacast video interface cards support a variety of protocols including 12G-SDI and HDMI 2.0, offering reliability, low latency, and high quality.  Deltacast will support the Clara AGX Developer Kit in their upcoming VideoMaster 6.20 SDK and driver software release.

Blackmagic Design is one of the world’s leading innovators and manufacturers of creative video technology. Their DeckLink series of SDI and HDMI video capture cards support resolutions up to 8k. An upcoming release of their desktop video ecosystem, including the DeckLink driver and desktop video SDK, will support the Clara AGX Developer Kit.

YUAN High Tech offers a wide variety of video capture cards for HDMI, SDI, DVI, IP, and analog video.  Yuan has over 10 years of experience supporting the medical device industry with their video capture solutions and will support the Clara AGX Developer Kit in an upcoming release of their driver software.

Magewell produces a line of Pro Capture PCIe cards supporting SDI, HDMI, DVI, and analog video formats for reliable, high-quality video applications in broadcast, media, and medical applications. Magewell will support the Clara AGX Developer Kit in an upcoming release of their driver software.

Real time, high-performance compute for ultrasound imaging

Ultrasound imaging is another application where real-time, high-performance compute is crucial at all points in the processing pipeline. The NVIDIA Clara Holoscan SDK can support ultrasound imaging at all stages of the processing pipeline from beamforming to image reconstruction to post-processing and rendering. For developers designing the next generation of software-defined ultrasound devices, NVIDIA has partnered with two leading ultrasound research platform providers for Clara Holoscan.

Ultrasound R&D company us4us provides a range of cost-effective ultrasound front-end research platforms. When connected to the Clara AGX Developer Kit by PCI Express, these can be used to prototype a software-defined ultrasound system. Beamforming, image processing, AI image analysis, and rendering are all done on an NVIDIA GPU. 

This provides developers with maximum flexibility in developing, testing, and modifying their processing pipelines, on a platform similar to one they would deploy in a production medical device. Direct access to raw ultrasound data from up to 1024 Tx and up to 256 Rx transducer channels opens up exciting possibilities for AI algorithm development at much higher accuracy and resolution than available from processed video output. See the NVIDIA Ultrasound NGC container for more information.

Verasonics offers the world-leading Vantage ultrasound research platform, a powerful development system offering up to 256 Tx and 256 Rx channels and a long list of features, which can be flexibly configured.  The Vantage system operates with the powerful MATLAB scripting environment and connects to the Clara AGX Developer Kit using an Ethernet connection, for maximum flexibility in data connectivity.    

Supporting analog data acquisition

Finally, for applications that require analog data, analog waveform generation, or general-purpose I/O, NVIDIA is partnering with Spectrum Instrumentation.   

Spectrum Instrumentation produces a diverse range of PCI Express data acquisition cards, offering the Clara AGX Developer Kit the ability to both sample and produce multiple analog signals, interact with medical devices and sensors using simple control signals, or control power relays or other system components. 

This rapidly growing interface ecosystem is currently supported on the Clara AGX Developer Kit and will be supported on the future Clara Holoscan Developer Kit. With nine sensor frontends supporting a range of modalities, the Clara Holoscan ecosystem will continue to provide flexibility and speed to sensing instruments.

Access the Clara Holoscan NGC Collection for a growing collection of AI frameworks, reference applications, and AI models built for Clara Developer Kits and medical device development, including containers for streaming video, ultrasound, metagenomics, and dermatology.

Categories
Misc

Major Updates to NVIDIA AI Software Advancing Speech, Recommenders, Inference, and More Announced at NVIDIA GTC 2022

At GTC 2022, NVIDIA announced Riva 2.0, Merlin 1.0, new features to NVIDIA Triton, and more.

At GTC 2022, NVIDIA announced major updates to its suite of NVIDIA AI software, for developers to build real-time speech AI applications, create high-performing recommenders at scale and optimize inference in every application, and more. Watch the keynote from CEO, Jensen Huang, to learn about the latest advancements from NVIDIA.


Announcing NVIDIA Riva 2.0

Today, NVIDIA announced Riva 2.0 in general availability. Riva is an accelerated speech AI SDK which provides models, tools, and fully-optimized speech recognition and text-to-speech pipelines for real-time applications.

Highlights include:

  • World class automatic speech recognition in seven languages.
  • Neural-based text to speech, generating high-quality human-like voices. 
  • Domain-specific customization with TAO Toolkit and NeMo.
  • Support to run in cloud, on-prem, and on embedded platforms.

NVIDIA also announced Riva Enterprise, providing enterprises with large-scale deployments access to speech experts at NVIDIA. Enterprises can try Riva with guided labs on ready to run infrastructure in LaunchPad.

Add this GTC session to your calendar to learn more:


Announcing NVIDIA Merlin 1.0 Hyperscale ML, DL Recommender Systems on CPU, GPU

Today, NVIDIA announced NVIDIA Merlin 1.0, an end-to-end framework designed to accelerate recommender workflows, from data preprocessing, feature transforms, training, optimization, and deployment. With this latest release of NVIDIA Merlin, data scientists and machine learning engineers can scale faster with less code. The new capabilities offer quick iteration over features, models, as well as deployment of fully trained recommender pipelines with feature transforms, retrieval, and ranking models as an inference microservice.

Highlights include:

  • Merlin Models, a new library for data scientists to train and deploy recommender models in less than 50 lines of code.
  • Merlin Systems, a new library, for machine learning engineers to easily deploy recommender pipelines as an ensembled Triton microservice.
  • Support for large scale multi-GPU, multinode inference, and less compute intensive workloads.  

For more information about the latest release, download and try NVIDIA Merlin.

Add these GTC sessions to your calendar to learn more:


Announcing new features in NVIDIA Triton

Today, NVIDIA announced new key updates to NVIDIA Triton. Triton is an open-source inference-serving software that brings fast and scalable AI to every application in production. 

Highlights include:

  • Triton FIL backend: Model explainability with Shapley values and CPU optimizations for better performance.
  • Triton Management Service to simplify and automate setting up and managing a fleet of Triton instances on Kubernetes. Alpha release is targeted for the end of March.
  • Triton Model Navigator to automate preparing a trained model for production deployment with Triton.
  • Fleet Command integration for edge deployment.
  • Support for inference on AWS Inferentia and MLFlow plug-in to deploy MLFlow models. 
  • Kick-start your Triton journey with immediate, short-term access in NVIDIA LaunchPad without needing to set up your own Triton environment.

You can download Triton from the NGC catalog, and access code and documentation on GitHub.

Add these GTC sessions to your calendar to learn more:


Announcing new updates to NVIDIA NeMo Megatron

Today NVIDIA announced the latest version of NVIDIA NeMo Megatron, a framework for training large language models (LLM.) With NeMo Megatron research institutions and enterprises can achieve the fastest training for any LLM. It also includes the latest parallelism techniques, data preprocessing scripts, and recipes to ensure training convergence.

Highlights include:

  • Hyper parameter tuning tool that automatically creates recipes based on customers’ needs and infrastructure limitations. 
  • Reference recipes for T5 and mT5 models.
  • Cloud support for Azure.
  • Distributed data preprocessing scripts to shorten end-to-end training time.

Click here to apply for early access. 

Add these GTC sessions to your calendar to learn more:


Announcing new Features in NVIDIA Maxine

Today NVIDIA announced the latest version of NVIDIA Maxine, a suite of GPU-accelerated SDKs that reinvent audio and video communications with AI, elevating standard microphones and cameras for clear online communications. Maxine provides state-of-the-art real-time AI audio, video, and augmented reality features that can be built into customizable, end to end deep learning pipelines. 

Highlights include:

  • Audio super resolution: Improves real-time audio quality by upsampling the audio input stream from 8kHz to 16kHz and from 16kHz to 48kHz sampling rate.
  • Acoustic echo cancellation: Cancels real-time acoustic device echo from input audio stream, eliminating mismatched acoustic pairs and double-talk. With AI-based technology, more effective cancellation is achieved than with traditional digital signal processing.
  • Noise removal: Removes several common background noises using state-of-the-art AI models while preserving the speaker’s natural voice.
  • Room echo cancellation: Removes reverberations from audio using state-of-the-art AI models, restoring clarity of a speaker’s voice.

Download NVIDIA Maxine now.  

Add these GTC sessions to your calendar to learn more:

Register for GTC now to learn more about the latest updates to GPU-accelerated AI technologies.

Categories
Misc

Supercharge AI-Powered Robotics Prototyping and Edge AI Applications with the Jetson AGX Orin Developer Kit

A rendering of the now available NVIDIA Jetson AGX Orin Developer Kit.The Jetson AGX Orin Developer Kit offers 8X the performance of the last generation, offering the most powerful AI supercomputer for advanced robotics, and embedded and edge computing.A rendering of the now available NVIDIA Jetson AGX Orin Developer Kit.

Availability of the the NVIDIA Jetson AGX Orin Developer Kit was announced today at NVIDIA GTC. The platform is the world’s most powerful, compact, and energy-efficient AI supercomputer for advanced robotics, autonomous machines, and next-generation embedded and edge computing.

Jetson AGX Orin delivers up to 275 trillion operations per second (TOPS). It gives customers more than 8X the processing power of its predecessor Jetson AGX Xavier, while maintaining the same small form factor and pin compatibility. It features an NVIDIA Ampere Architecture GPU, Arm Cortex-A78AE CPU, next-generation deep learning and vision accelerators, high-speed interfaces, faster memory bandwidth, and multimodal sensor support to feed multiple, concurrent AI application pipelines.

The NVIDIA Jetson AGX Orin Developer Kit is perfect for prototyping advanced AI-powered robots and edge AI applications for manufacturing, logistics, retail, agriculture, healthcare, and more.

“As AI transforms manufacturing, healthcare, retail, transportation, smart cities, and other essential sectors of the economy, demand for processing continues to surge,” said Deepu Talla, vice president and general manager of embedded and edge computing at NVIDIA. “A million developers and more than 6,000 companies have already turned to Jetson. The availability of Jetson AGX Orin will supercharge the efforts of the entire industry as it builds the next generation of robotics and edge AI products.”

Jetson AGX Orin Developer Kit features:

  • Up to 275 TOPS and 8X the performance of the last generation, plus high-speed interface support for multiple sensors.
  • An NVIDIA Ampere Architecture GPU and 12-core Arm Cortex-A78AE 64-bit CPU, together with next-generation deep learning and vision accelerators.
  • High-speed I/O, 204.8GB/s of memory bandwidth, and 32GB of DRAM capable of feeding multiple concurrent AI application pipelines.

The Jetson AGX Orin Developer Kit has the computing capability of more than eight Jetson AGX Xavier systems. It integrates the latest NVIDIA GPU technology with the world’s most advanced deep learning software stack, delivering the flexibility to create sophisticated AI solutions now and well into the future. The developer kit can emulate all the production Jetson AGX Orin and Orin NX modules, set for release Q4 2022.

Customers using the Jetson AGX Orin Developer Kit can leverage the full NVIDIA CUDA-X accelerated computing stack. This suite includes pretrained models from the NVIDIA NGC catalog and the latest NVIDIA application frameworks and tools for application development and optimization, such as Isaac, Metropolis, TAO, and Omniverse.

These tools reduce time and cost for production-quality AI deployments. Developers can access the largest, most complex models needed to solve robotics and edge AI challenges in 3D perception, natural language understanding, multisensor fusion, and more.

Developer kit pricing and availability

The NVIDIA Jetson AGX Orin Developer Kit is available now at $1,999. Production modules will be available in Q4 2022 starting at $399.

Learn more about this new Jetson offering and attend an upcoming dedicated GTC session.

Downloadable documentation, software, and other resources are available in the Jetson Download Center.

Categories
Misc

Create 3D Virtual Worlds with New Releases, Expansions, and Toolkits from NVIDIA Omniverse 

Learn about all of the new applications, features, and functions released for developers to build, extend, and connect 3D tools and platforms to the Omniverse ecosystem seamlessly.

Developers, creators, and enterprises around the world are using NVIDIA Omniverse—the real-time collaboration and simulation platform for 3D design—to enhance complex workflows and develop for 3D worlds faster. At NVIDIA GTC, we showcased how the platform’s ecosystem is expanding, from new Omniverse Connectors and asset libraries to updated Omniverse apps and features.

With these latest releases and capabilities, Omniverse developers can build, extend, and connect 3D tools and platforms to the Omniverse ecosystem more easily than ever before.

Omniverse Kit 

Omniverse Kit is a powerful toolkit on which Omniverse applications are built.

The latest version includes core functionality and rendering improvements for 3rd party renderer integrations and Arbitrary Output Variables (AOV) support.

Key updates:

  • Manipulator Framework: Create 3D manipulators within the viewport.
  • Hydra Delegates: Add hydra-based renderers to Kit based applications.
  • WARP in Omnigraph: Create GPU-accelerated Omnigraph compute nodes using Python syntax.
  • MDL Python Support Added: Enhance ability to integrate MDL through Python.
Figure 1. Hydra Support in NVIDIA Omniverse.

Omniverse Nucleus

Omniverse Nucleus Workstation and Enterprise Nucleus Server were updated with a new navigator and several features requested from the community.

Key feature:

  • DeepSearch, a new AI-based search microservice for Nucleus. Users can search through massive, untagged databases of assets using natural language or images. Now available for early access to Omniverse Enterprise customers.
A screen shot of the new connect and multiple view Nucleus instance in a shared Navigator.
Figure 2. A new ability to connect and view multiple Nucleus instances in a shared Navigator makes it easier to find and search for projects.

Omniverse Cloud

At GTC, we announced Omniverse Cloud, a suite of cloud services that provide instant access to the NVIDIA Omniverse platform. Omniverse Cloud encompasses several services, including Nucleus Cloud, now in early access, plus Omniverse Create and View apps streaming from the cloud.

Figure 3. Until now, individual or teams of designers, creators, engineers, and researchers have hosted NVIDIA Omniverse on their RTX-powered laptops, desktops, or data centers.

Omniverse Apps 2022.1 updates

Omniverse Create 

With Omniverse Create advanced scene composition users can interactively simulate, and render scenes in Pixar USD in real time.

Key updates:

  • General animation keyframing and curve editing.
  • ActionGraph based on Omnigraph to map keyboard shortcuts and your own user interface buttons. 
  • New PhysX GPU accelerated softbody support.
  • vMaterials 2.0 additions. Access a massive library of over 900 physical materials.
  • Animation Graph (beta): Bring characters to life with our new node-based graph editor for runtime animation blending and playback.
Figure 4. Omniverse Create 2022.1 overview.

Omniverse View

With Omniverse View, project reviewers can collaboratively review 3D design projects in stunning photorealism. 

Key updates releasing March 30:

  • Markup: Add editable text, shapes, and scribbles to convey comments. 
  • Measure: A variety of snapping methods are available to measure between entities and create desired dimensions.
  • Turntable: Rotate objects on a virtual turntable to see how light affects the model.
  • Teleport: Move around large scenes by jumping great distances to place the camera in an area.
  • Waypoints 2.0: Edit or reorder waypoints, and easily create a 3D slide deck walkthrough.
An immersive VR experience showing the teleport tool in Omniverse View
Figure 5. Create an immersive VR experience with the teleport tool in Omniverse View.

Omniverse Machinima

With Omniverse Machinima, users can collaborate in real time to animate and manipulate characters and their virtual world environments. 

Key updates:

  • Track and capture motion in real time using a single camera, with live conversion from 2D camera capture to 3D model in the Maxine Pose Estimation workflow.
  • Faster and easier to use sequencer with tools such as splitting, looping, hold, and scale, and even more “drag and drop” functionality.
  • Animation Retargeting (beta): Easily remap skeletal animations from one biped to another, with presets for popular character systems like Reallusion Character Creator 3.
Figure 6. Join the Omniverse Machinima challenge.

Omniverse XR App

Coming soon to beta, the new Omniverse XR App lets you bring your 3D work into human scale. View, review, and annotate Omniverse scenes, manipulate 3D objects, control the lighting all in VR. 

Figure 7. Bring your 3D work into human scale using Omniverse scenes.

New developer tools

Omniverse Code

Omniverse Code is the integrated development environment that helps developers build Kit-based extensions, apps, and microservices easier than ever.  Key features, releasing March 30, include new interactive documentation and templates along with Omniverse Replicator extensions.

A snapshot of Omniverse Code (left) alongside Microsoft Visual Studio (right).
Figure 8. Omniverse Code (left) alongside Microsoft Visual Studio (right).

Omniverse Replicator

NVIDIA Omniverse Replicator is a framework built on the Omniverse platform that enables physically accurate 3D synthetic data generation to accelerate training and accuracy of perception networks. Omniverse Replicator is now available as a set of Omniverse Extensions, with content, and examples within Omniverse Code App in the Omniverse Launcher.

An image of Toy Jensen “Where’s Waldo” style hiding in a stack of Marbles assets, but being found by Omniverse Replicator.
Figure 9. Images from Omniverse Marbles.

Omniverse Avatar

NVIDIA Omniverse Avatar, a technology platform for generating interactive AI avatars, connects NVIDIA technologies in speech AI, computer vision, natural language understanding, recommendation engines, and simulation technologies. While currently under development, developers and artists can get started with one piece of the Avatar technology, Omniverse Audio2Face

Figure 10. NVIDIA CEO Jensen Huang kicks off GTC with a keynote that presents the latest breakthroughs including Omniverse Avatar.  

Partner ecosystem updates

New support for the Omniverse ecosystem provided by leading 3D marketplaces and digital asset libraries gives creators an even easier way to build their scenes. TurboSquid by Shutterstock, Sketchfab, Twinbru, and newly added Reallusion’s ActorCore assets are searchable directly inside the Omniverse App asset browser. Joining our catalog of Omniverse-ready assets alongside CGTrader is A23D, bringing 40,000 Omniverse-ready assets to users.

New Omniverse connections

  • Adobe Substance 3D Material Extension: Import Substance 3D asset files into any Omniverse App.
  • Adobe Substance 3D Painter Connector: Apply textures, materials, and masks or UV mapping onto 3D assets with Adobe Substance 3D Painter, releasing March 28.
  • Unreal Engine 5: Send and sync model data and export Nanite Geometry to Omniverse Nucleus.
  • e-on VUE: Create beautiful CG environments including skies, terrains, roads, and rocks.
  • e-on PlantCatalog: Export a plant, enable live-sync, and edit in real time. 
  • e-on PlantFactory: Create ultra-realistic, high polygon plants.
  • Maxon Cinema 4D: USD is now supported. Use the app in a connected workflow with OmniDrive. 
  • Ipolog: Perform material provisioning and production logistics for manufacturing planners.  
  • LumenRT for NVIDIA Omniverse, powered by Bentley iTwin: Allows engineering-grade, millimeter-accurate digital content to be visualized on multiple devices and form factors.
Simulation of ITER Fusion Reactor Model from LumenRT for NVIDIA Omniverse.
Figure 11. ITER fusion reactor model rendered in LumenRT for NVIDIA Omniverse, powered by Bentley iTwin. Image courtesy of Bentley Systems and ITER.

Supported Hydra renderers

Creators, designers, and developers can now integrate their favorite Hydra delegate renderers directly into their Omniverse workflows by building a Hydra render delegate themselves or using NVIDIA-built Hydra render delegates for Pixar HDStorm, Maxon RedShift, and OTOY Octane, with Blender Cycles, Chaos V-Ray, and Autodesk Arnold coming soon.

Image displaying Omniverse Create UI and a scene within it taking advantage of the Hydra Render Delegate for Cycles X.
Figure 12. Integrate Hydra-delegate renderers directly into an Omniverse workflow with Hydra render delegates for Blender Cycles X (shown above.)

New CAD Importers

Manufacturing and product development workflows just became easier with the introduction of 26 CAD Importers to Omniverse. Developers, designers, and engineers can now easily bring in common CAD formats from leading ISV applications directly into Omniverse.

Image displaying Omniverse UI and CAD software side by side, to exemplify the import capabilities within Omniverse to import CAD files and render in high-fidelity.
Figure 13. Omniverse CAD Importer.

Explore more at GTC  

Join Omniverse Developer Days at GTC, which will showcase the many ways power users and developers can build extensions and apps on the platform. Hear from Omniverse engineering leaders and industry experts as they share new insights about 3D virtual world building, simulation, rendering, and more.

Additional resources 

Learn more by diving into the Omniverse Resource Center, which details how developers can build custom applications and extensions for the platform. 

For additional support, explore the Omniverse forums and Medium channel, tutorials on Twitter and YouTube, and join our Discord server to chat with the community.

Categories
Misc

Boosting Ultra Rapid Nanopore Sequencing Analysis on NVIDIA DGX A100

Ultra rapid nanopore sequencing is bringing us one step closer to same-day whole genome genetic diagnosis.

Fast and cost-effective whole genome sequencing and analysis can be critical in a diagnostic setting. Recent advances in accelerated clinical sequencing, such as the world-record-setting DNA sequencing technique for rapid diagnosis, are bringing us one step closer to same-day, whole-genome genetic diagnosis.

A team led by Stanford University School of Medicine, NVIDIA, Google, UCSC and Oxford Nanopore Technologies (ONT) recently used this technique to identify disease-associated genetic variants that resulted in a diagnosis in just over 7 hours, with results published earlier this year in the New England Journal of Medicine.

This record-beating end-to-end genomic workflow is reliant on innovative technology and high-performance computation. Nanopore sequencing was implemented across 48 flow cells, with optimized methods enabling pore occupancy at 82%, rapidly generating 202 Gb in just a couple of hours. Analysis of the output was distributed across a Google Cloud computing environment, including basecalling and alignment across 16 instances of 4 x V100 GPUs (64 GPUs total), and variant calling across 14 instances of 4 x P100 GPUs (56 GPUs total).

Since the January NEJM publication, the NVIDIA Clara team has been optimizing the same whole genome workflow for the DGX-A100, giving clinicians and researchers the ability to deploy the same analysis as the world record approach on just eight A100 GPUs, and in just 4h10m for a 60x whole genome (Figure 1; benchmarked on the HG002 reference sample).

Visual representation of the GPUs used for the original record, including 64 GPUs for basecalling and alignment, and 56 GPUs for the multiple variant calling steps. Diagram also shows the workflow optimized for the DGX-A100, using just 8 GPUs. Benchmark times displayed for HG002 whole genome reference sample, sequencing to a similar depth of coverage as Patient 11 (60x).
Figure 1. Optimized nanopore sequencing workflow on NVIDIA DGX-A100

Not only does this enable fast analysis in a single server (8-GPU) framework, but it also lowers the cost per sample by two-thirds, from $568 to $183.

Basecalling and alignment

Basecalling is the process of classifying raw instrument signals into the bases A, C, G, and T of the genome. It is a computationally critical step in ensuring accuracy in all downstream analysis tasks. It is also an important data reduction step, reducing resultant data by approximately 10x.

At ~340 bytes per base, a single 30x coverage whole genome can easily be multiple terabytes in raw signals, as opposed to hundreds of gigabytes when processed. As such, it is beneficial for compute speed to rival sequencing output speed, which is non-trivial at a pace of ~450 bases per second through the 128,000 pores across 48 flow cells.

ONT’s PromethION P48 sequencer can generate as much as 10 terabases in a 72-hour run, equivalent to 96 human genomes (30x coverage).

The rapid classification task required for this already benefits from deep learning innovation and GPU acceleration. The core data processing toolkit for this purpose, Guppy, uses a recurrent neural network (RNN) for basecalling, with the option of two different architectures of either smaller (faster) or larger (higher accuracy) recurrent layer sizes.

The main computational bottleneck in basecalling is the RNN kernel, which has benefited from GPU integration with ONT sequencers, such as the desktop GridION Mk1 that includes a single V100 GPU and the handheld MinION Mk1C that includes a Jetson Edge platform.

Alignment is the process of taking the resultant basecalled fragments of DNA, now in the form of character strings of As, Cs, Gs, and Ts, and determining the genome location where those fragments originated, assembling a full genome from the massively parallelized sequencing process. This essentially rebuilds the full length genome from many 100-100,000bp long reads. For the world-record-setting sample, this totaled around 13million reads.

For the original world-record analysis, basecalling and alignment were run on separate instances of Guppy and Minimap2, respectively. In migrating this to a single-server DGX-A100 solution, and using Guppy’s integrated minimap2 aligner, you immediately save time on I/O. Through balancing of basecalling and alignment across the DGX eight A100 GPUs and 96 CPU threads, respectively, the two processes can be overlapped perfectly to align reads concurrently with basecalling, resulting in no impact on total runtime (

This brings the runtime of the basecalling and alignment step on the DGX-A100 to 2h 46m, which can also be overlapped with the sequencing itself. It is similar to the sequencing time expected for a 60x sample.

Variant calling

Variant calling is the portion of the workflow designed to identify all of the points in the newly assembled individual’s genome that differ from expected, compared to a reference genome. This involves scanning the full breadth of the genome to look for different types of variation. For example, this might include small single-base-pair variants all the way to large structural variants covering thousands of base-pairs. The world record pipeline used PEPPER-Margin-DeepVariant for small variants, and Sniffles for structural variants.

The PEPPER-Margin-DeepVariant approach is designed to optimize small variant calling for the long reads produced by nanopore sequencing.

  • PEPPER identifies candidate variants through an RNN consisting of two bidirectional, gated, recurrent unit layers and a linear transformation layer.
  • Margin then uses a hidden Markov model approach for a process called haplotyping, determining which variants have been inherited together from the maternal or paternal chromosomes. It passes this information to Google DeepVariant to use for maximum heterozygous variant calling accuracy.
  • DeepVariant classifies final variants through a deep convolutional neural network, which is built on the Inception v2 architecture adapted specifically for DNA read pile-up input images.

Overall, PEPPER-Margin-DeepVariant allows the faster neural network of PEPPER to scan the whole genome for candidates, and then uses the larger neural network of DeepVariant for high accuracy variant calling of those candidates. To accelerate this pipeline, the world-record workflow used Parabricks DeepVariant, a GPU-accelerated implementation providing >20x faster runtimes than the open-source version on CPU (Figure 2).

The Clara team took this acceleration further by modifying PEPPER-Margin to run in an integrated fashion, splitting the data by chromosome and running the programs concurrently on GPU. PEPPER was also optimized for pipeline parameters such as batch sizes, number of workers, and number of callers, as well as upgrading PyTorch to enable support for NVIDIA Ampere Architecture acceleration of the RNN inference bottleneck.

For structural variant calling, Sniffles was upgraded to the recently released Sniffles 2, which is considerably more efficient, at 38x acceleration on CPU alone.

All these improvements put the runtime of the multiple variant calling stage at 1h 24m on the DGX-A100.

Benchmarks here include open-source DeepVariant on a 32-thread CPU and Parabricks DeepVariant on 4 GPUs. Data used was an ONT-sequenced 50x whole genome reference sample. GPU runtime was benchmarked on a p3.8xlarge AWS instance.
Figure 2. Parabricks DeepVariant enables fast runtimes on ONT data

Powering real-time sequencing with NVIDIA DGX-A100

By bringing the world-record DNA sequencing technique for critical care patients to the DGX A100, the NVIDIA Clara team is powering real-time sequencing, simplifying a complex workflow on a single server, and cutting analysis costs by two-thirds. These improvements better enable long-read whole genome sequencing in a clinical care setting and help rare and undiagnosed disease patients get answers faster.

This pipeline will be available free for researchers. Request access today.

Featured image courtesy of Oxford Nanopore Technologies

Categories
Misc

Creating a custom model for image recognition with my own images

I’m currently doing a tutorial showing me how to use TensorFlow in a general sense but for the project I’m working on I have images that are originating from my project. How do I create a custom model to train the neural network on with my images instead of one of the preset ones in most tutorials?

submitted by /u/Kapooshi
[visit reddit] [comments]

Categories
Misc

Driving on Air: Lucid Group Builds Intelligent EVs on NVIDIA DRIVE

Lucid Group may be a newcomer to the electric vehicle market, but its entrance has been grand. The electric automaker announced at GTC that its current and future fleets are built on NVIDIA DRIVE Hyperion for programmable, intelligent capabilities. By developing on the scalable, software-defined platform, Lucid ensures its vehicles are always at the cutting Read article >

The post Driving on Air: Lucid Group Builds Intelligent EVs on NVIDIA DRIVE appeared first on NVIDIA Blog.

Categories
Misc

NVIDIA Launches AI Computing Platform for Medical Devices and Computational Sensing Systems

NVIDIA today introduced Clara Holoscan MGX™, a platform for the medical device industry to develop and deploy real-time AI applications at the edge, specifically designed to meet required regulatory standards.

Categories
Misc

NVIDIA Announces Availability of Jetson AGX Orin Developer Kit to Advance Robotics and Edge AI

NVIDIA today announced the availability of the NVIDIA® Jetson AGX Orin™ developer kit, the world’s most powerful, compact and energy-efficient AI supercomputer for advanced robotics, autonomous machines, and next-generation embedded and edge computing.