Cloud Hosting

GTCNVIDIA today launched dozens of enterprise-grade generative AI microservices that businesses can use to create and deploy custom applications on their own platforms while retaining full ownership and control of their intellectual property.

Built on top of theNVIDIA CUDAplatform, the catalog of cloud-native microservices includesNVIDIA NIMmicroservices for optimized inference on more than two dozen popular AI models from NVIDIA and its partner ecosystem. In addition, NVIDIA accelerated software development kits, libraries and tools can now be accessed asNVIDIA CUDA-X microservices for retrieval-augmented generation (RAG), guardrails, data processing, HPC and more. NVIDIA also separately announced over two dozenhealthcare NIM and CUDA-X microservices.

The curated selection of microservices adds a new layer to NVIDIAs full-stack computing platform. This layer connects the AI ecosystem of model developers, platform providers and enterprises with a standardized path to run custom AI models optimized for NVIDIAs CUDA installed base of hundreds of millions of GPUs across clouds, data centers, workstations and PCs.

Among the first to access the new NVIDIA generative AI microservices available inNVIDIA AI Enterprise 5.0are leading application, data and cybersecurity platform providers includingAdobe,Cadence,CrowdStrike, Getty Images,SAP,ServiceNow, and Shutterstock.

Established enterprise platforms are sitting on a goldmine of data that can be transformed into generative AI copilots, said Jensen Huang, founder and CEO of NVIDIA. Created with our partner ecosystem, these containerized AI microservices are the building blocks for enterprises in every industry to become AI companies.

NIM Inference Microservices Speed Deployments From Weeks to Minutes NIM microservices provide pre-built containers powered by NVIDIA inference software including Triton Inference Server and TensorRT-LLM which enable developers to reduce deployment times from weeks to minutes.

They provide industry-standard APIs for domains such as language, speech and drug discovery to enable developers to quickly build AI applications using their proprietary data hosted securely in their own infrastructure. These applications can scale on demand, providing flexibility and performance for running generative AI in production on NVIDIA-accelerated computing platforms.

NIM microservices provide the fastest and highest-performing production AI container for deploying models from NVIDIA,A121, Adept,Cohere, Getty Images, and Shutterstock as well as open models from Google,Hugging Face, Meta, Microsoft, Mistral AI and Stability AI.

ServiceNowtoday announced that it is using NIM to develop and deploy new domain-specific copilots and other generative AI applications faster and more cost effectively.

Customers will be able to access NIM microservices fromAmazon SageMaker,Google Kubernetes EngineandMicrosoft Azure AI, and integrate with popular AI frameworks likeDeepset,LangChainandLlamaIndex.

CUDA-X Microservices for RAG, Data Processing, Guardrails, HPC CUDA-X microservicesprovide end-to-end building blocks for data preparation, customization and training to speed production AI development across industries.

To accelerate AI adoption, enterprises may use CUDA-X microservices includingNVIDIA Rivafor customizable speech and translation AI,NVIDIA cuOpt for routing optimization, as well asNVIDIA Earth-2for high resolution climate and weather simulations.

NeMo Retriever microservices let developers link their AI applications to their business data including text, images and visualizations such as bar graphs, line plots and pie charts to generate highly accurate, contextually relevant responses. With these RAG capabilities, enterprises can offer more data to copilots, chatbots and generative AI productivity tools to elevate accuracy and insight.

AdditionalNVIDIA NeMo microservicesare coming soon for custom model development. These include NVIDIA NeMo Curator for building clean datasets for training and retrieval, NVIDIA NeMo Customizer for fine-tuning LLMs with domain-specific data, NVIDIA NeMo Evaluator for analyzing AI model performance, as well asNVIDIA NeMo Guardrailsfor LLMs.

Ecosystem Supercharges Enterprise Platforms With Generative AI Microservices In addition to leading application providers, data, infrastructure and compute platform providers across the NVIDIA ecosystem are working with NVIDIA microservices to bring generative AI to enterprises.

Top data platform providers includingBox, Cloudera, Cohesity,Datastax, Dropbox andNetAppare working with NVIDIA microservices to help customers optimize their RAG pipelines and integrate their proprietary data into generative AI applications.Snowflakeleverages NeMo Retriever to harness enterprise data for building AI applications.

Enterprises can deploy NVIDIA microservices included with NVIDIA AI Enterprise 5.0 across the infrastructure of their choice, such as leading cloudsAmazon Web Services (AWS),Google Cloud,AzureandOracle Cloud Infrastructure.

NVIDIA microservices are also supported on over 400 NVIDIA-Certified Systems, including servers and workstations from Cisco,Dell Technologies,Hewlett Packard Enterprise (HPE), HP,Lenovoand Supermicro. Separately today, HPE announced availability of HPEs enterprise computing solution for generative AI, with planned integration of NIM andNVIDIA AI Foundation modelsinto HPEs AI software.

NVIDIA AI Enterprise microservices are coming to infrastructure software platforms includingVMware Private AI Foundationwith NVIDIA.Red HatOpenShift supports NVIDIA NIM microservices to help enterprises more easily integrate generative AI capabilities into their applications with optimized capabilities for security, compliance and controls.Canonicalis adding Charmed Kubernetes support for NVIDIA microservices through NVIDIA AI Enterprise.

NVIDIAs ecosystem of hundreds of AI and MLOps partners, including Abridge, Anyscale, Dataiku,DataRobot,Glean, H2O.ai,Securiti AI,Scale AI,OctoAIandWeights & Biases, are adding support for NVIDIA microservices through NVIDIA AI Enterprise.

Apache Lucene,Datastax, Faiss, Kinetica, Milvus, Redis, and Weaviate are among the vector search providers working with NVIDIA NeMo Retriever microservices to power responsive RAG capabilities for enterprises.

Availability Developers can experiment with NVIDIA microservices atai.nvidia.comat no charge. Enterprises can deploy production-grade NIM microservices with NVIDIA AI Enterprise 5.0 running on NVIDIA-Certified Systems and leading cloud platforms.

For more information, watch the replay ofHuangs GTC keynoteand visit the NVIDIA booth at GTC, held at the San Jose Convention Center through March 21.

View post:

NVIDIA Launches Generative AI Microservices for Developers to Create and Deploy Generative AI Copilots Across ... - NVIDIA Blog