Google Cloud is revamping its AI Hypercomputer architecture with significant enhancements across the board to support rising demand for generative artificial intelligence applications that are becoming increasingly pervasive in enterprise workloads.
At Google Cloud Next 24 today, the company announced updates to almost every layer of the AI Hypercomputer cloud architecture, with new virtual machines powered by Nvidia Corp.s most advanced graphics processing units one of the most significant revelations. In addition, it unveiled enhancements to its storage infrastructure for AI workloads, plus the underlying software for running AI models, and more flexible consumption options with its Dynamic Workload Scheduler service.
The updates were announced by Mark Lohmeyer, vice president and general manager of Compute and ML Infrastructure at Google Cloud. He explained that generative AI has gone from almost nowhere just a couple of years ago to becoming widespread across a wide range of enterprise applications encompassing text, code, videos, images, voice, music and more, placing incredible strains on the underlying compute, networking and storage infrastructure that supports it.
To support the increasingly powerful generative AI models being adopted across the enterprise today, Google Cloud has announced the general availability of what it says is its most powerful and scalable tensor processing unit to date. Its called the TPU v5p, and it has been designed with a single purpose in mind to train and run the most demanding generative AI models.
TPU v5p is built to deliver enormous computing power, with a single pod containing 8,960 chips running in unison, which is more than twice as many as the number in a TPU v4 pod. According to Lohmeyer, the TPU v5p delivers some impressive performance gains, with twice as many floating point operations per second and three-times more high-bandwidth memory on a per-chip basis, resulting in vastly improved overall throughput.
To enable customers to train and serve AI models running on large-scale TPU clusters, Google is adding support for the TPU v5p virtual machines on Google Kubernetes Engine, its cloud-hosted service for running software containers.
As an alternative, customers can also use the latest hardware from Nvidia to train their generative AI models on Google Cloud. Besides its TPU family, its also providing access to Nvidias H100 GPUs through its new A3 family of VMs. The A3 Mega VM will become generally available from next month, and one of its main advantages will be support for confidential computing, which refers to techniques that can protect the most sensitive data from unauthorized access even while its being processed. This is a key development, Lohmeyer said, as it will provide a way for generative AI models to access data that was previously deemed too risky for them to process.
Character.AI is using Google Clouds Tensor Processor Units and A3 VMs running on Nvidias H100 Tensor Core GPUs to train and infer LLMs faster and more efficiently, said Character Technologies Inc. Chief Executive Noam Shazeer. The optionality of GPUs and TPUs running on the powerful AI-first infrastructure makes Google Cloud our obvious choice as we scale to deliver new features and capabilities to millions of users.
More exciting, perhaps, is what Google Cloud has in store for later in the year. Though it hasnt said when, the company confirmed that its planning to bring Nvidias recently announced but not yet released Blackwell GPUs to its AI Hypercomputer architecture. Lohmeyer said the Blackwell GPUs will be made available in two configurations, with VMs powered by both the HGX B200 and GB200 NVL72 GPUs. The former are designed for the most demanding AI workloads, while the latter is expected to support a new era of real-time large language model inference and massive-scale training for trillion-parameter scale models.
More powerful compute is just one part of the infrastructure equation when it comes to supporting advanced generative AI workloads. In addition, enterprises also need access to more capable storage systems that keep their data as close as possible to the compute instances that power them. The idea is that this reduces latency to train models faster, and with todays updates, Google Cloud claims its storage systems are now among the best in the business, with improvements that maximize GPU and TPU utilization, resulting in superior energy efficiency and cost optimization.
Todays updates include the general availability of Cloud Storage FUSE, a new file-based interface for Google Cloud Storage that enables AI and machine learning applications to tap into file-based access to its cloud storage resources. According to Google Cloud, GCS FUSE delivers an increase in training throughput of 2.9 times compared with its existing storage systems, with model serving performance showing a 2.2-times improvement.
Other enhancements include support for caching in preview within Parallelstore, a high-performance parallel file system thats optimized for AI and high-performance computing workloads. With its caching capabilities, Parallelstore enables up to 3.9 times faster training times and 3.7 times superior training throughput, compared to traditional data loaders.
The company also announced AI-focused optimizations to the Filestore service, which is a network file system that enables entire clusters of GPUs and TPUs to simultaneously access the same data.
Lastly, theres the new Hyperdisk ML service, which delivers block storage, available now in preview. With this, Google Cloud claims it can accelerate model load times by up to 12-times compared to alternative services.
A third part of the generative AI equation is the open-source software thats used to support many of these models, and Google Cloud hasnt ignored these either. Its offering a range of updates across its software stack that it says will help simplify developer experiences and improve performance and cost efficiencies.
The software updates include the debut of MaxDiffusion, a new high-performance and scalable reference implementation for diffusion models that generate images. In addition, the company announced a range of new open models available now in MaxText, such as Gemma, GPT3, Llama 2 and Mistral.
The MaxDiffusion and MaxTest models are built on a high performance numerical computing framework called JAX, which is integrated with the OpenXLA compiler to optimize numerical functions and improve model performance. The idea is that these components ensure the most effective implementation of these models, so developers can focus on the math.
In addition, Google announced support for the latest version of the popular PyTorch AI framework, PyTorch/XLA 2.3, which will debut later this month.
Lastly, the company unveiled a new LLM inference engine called Jetstream. Its an open-source offering thats throughput- and memory-optimized for AI accelerators such as Google Clouds TPUs. According to Lohmeyer, it will provide three-times higher performance per dollar on Gemma 7B and other open AI models.
As customers bring their AI workloads to production, theres an increasing demand for a cost-efficient inference stack that delivers high performance, he explained. JetStream helps with this need and offers support for models trained with both JAX and PyTorch/XLA, and includes optimizations for popular open models such as Llama 2 and Gemma.
The final ingredient for running generative AI on Googles cloud stack is the Dynamic Workload Scheduler, which delivers resource management and job scheduling capabilities to developers. The main idea is that it improves access to AI computing capacity while providing tools to optimize spending on these resources.
With todays update, Dynamic Workload Scheduler now provides two starting modes flex start mode for enhanced obtainability with optimized economics, and calendar mode, for more predictable job start times and durations. Both modes are now available in preview.
According to Lohmeyer, flex start jobs will be cued to run as soon as possible, based on resource availability. This will make it easier for developers to access the TPU and GPU resources they need for workloads with more flexible start times. As for calendar mode, this provides short-term reserved access to AI compute resources including TPUs and GPUs. Users will be able to reserve co-located GPUs for a period of up to 14 days, up to eight weeks in advance. Reservations will be confirmed, and the capacity will come available on the requested start date.
Dynamic Workload Scheduler improved on-demand GPU obtainability by 80%, accelerating experiment iteration for our researchers, said Alex Hays, a software engineer at Two Sigma Inc. Leveraging the built-in Kueue and GKE integration, we were able to take advantage of new GPU capacity in Dynamic Workload Scheduler quickly and save months of development work.
THANK YOU
Originally posted here:
Google Cloud's AI Hypercomputer cloud infrastructure gets new GPUs, TPUs, optimized storage and more - SiliconANGLE News
- CTERA Networks Partners with SYNNEX Corporation to Drive Market Demand for Hybrid Cloud Storage, Collaboration and ... [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Cloud storage exempt from Ninefold's uptime boost [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Virsto Named Finalist of 2012 Storage Virtualization & Cloud Awards [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Innovative Hybrid Cloud Storage Solutions Now Available From PROMISE Technology [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Box Talks Integration with BlackBerry 10 and Cloud Storage for Business - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- AG112's Weekly Technology Tutorials Ep.7 Cloud Storage - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Cloud Storage - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Google Cloud Storage Office Hours - 9/5/2012 - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- IBM Cloud Storage -- Future Directions - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Working with best FREE Cloud storage solution - MediaFire - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Best Cloud Storage | How Nate Made $450 His First Hour... - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Cloud Storage Services: Comparison - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Top 10 Free Cloud Storage Services of 2012 - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Cloud Storage Wars - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Secure and Comprehensive Cloud Storage for Health IT - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Free Cloud Storage! - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Microsoft SkyDrive Cloud Storage - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Top 16 Android Cloud Storage Apps Quick Breakdown - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Up to 48GB of FREE Cloud Storage, 14GB Guaranteed - Video [Last Updated On: October 5th, 2012] [Originally Added On: October 5th, 2012]
- Nasuni's CEO To Speak At Interop On The Secure Use Of Cloud Storage [Last Updated On: October 6th, 2012] [Originally Added On: October 6th, 2012]
- Oracle vs Amazon Cloud Storage: OpenWorld 2012 - Video [Last Updated On: October 6th, 2012] [Originally Added On: October 6th, 2012]
- Apple extends iCloud storage for another year [Last Updated On: October 7th, 2012] [Originally Added On: October 7th, 2012]
- Interush Introduces Convenient Cloud-Based Storage Service with Release of PHYTTER DOCK Application [Last Updated On: October 9th, 2012] [Originally Added On: October 9th, 2012]
- Get a free 15GB cloud-storage account from 4Sync [Last Updated On: October 9th, 2012] [Originally Added On: October 9th, 2012]
- Cloud Solutions Increase Customer Engagement and Retention [Last Updated On: October 9th, 2012] [Originally Added On: October 9th, 2012]
- Pogoplug offering 100GB of cloud storage to UK users for just £19.99 a year [Last Updated On: October 10th, 2012] [Originally Added On: October 10th, 2012]
- New vFoglight Storage 2.0 Provides Integrated Application to Disk Performance Monitoring [Last Updated On: October 10th, 2012] [Originally Added On: October 10th, 2012]
- Lunacloud Deploys Cloudian® To Grow Business, Offer S3 Compatible Cloud Storage [Last Updated On: October 11th, 2012] [Originally Added On: October 11th, 2012]
- New Cloud Storage Company, ZapDrive, Launches Today Offering 100 GB for $19.99/year. [Last Updated On: October 11th, 2012] [Originally Added On: October 11th, 2012]
- Otixo Adds Ubuntu One to Aggregated Cloud Storage Lineup [Last Updated On: October 11th, 2012] [Originally Added On: October 11th, 2012]
- Cloud Storage Reviews Announcement Video - Video [Last Updated On: October 11th, 2012] [Originally Added On: October 11th, 2012]
- Cloud storage outage strikes Macquarie Telecom [Last Updated On: October 11th, 2012] [Originally Added On: October 11th, 2012]
- Online-Storage.com is Now SIO.CO [Last Updated On: October 11th, 2012] [Originally Added On: October 11th, 2012]
- C2C Maximizes eMail Archiving Flexibility and Control With Support for the Hybrid Cloud [Last Updated On: October 11th, 2012] [Originally Added On: October 11th, 2012]
- OwnCloud: Build your own or manage your public cloud storage services [Last Updated On: October 12th, 2012] [Originally Added On: October 12th, 2012]
- Ubuntu's cloud storage service hits Mac in beta, with 5GB free [Last Updated On: October 12th, 2012] [Originally Added On: October 12th, 2012]
- Akitio Cloud Hybrid Review: Convenient NAS and USB Storage in One [Last Updated On: October 13th, 2012] [Originally Added On: October 13th, 2012]
- Symform Hires Senior Sales Executive to Build Global Partnerships as Distributed Cloud Storage Network Surpasses 5.5 ... [Last Updated On: October 15th, 2012] [Originally Added On: October 15th, 2012]
- Get an extra 25GB of storage in the Dropbox Great Space Race [Last Updated On: October 16th, 2012] [Originally Added On: October 16th, 2012]
- Microsoft Acquires StorSimple To Increase Cloud Storage Capabilities [Last Updated On: October 17th, 2012] [Originally Added On: October 17th, 2012]
- Inktank-Metacloud Partnership Enhances Fully Managed Private Cloud Solution With Enterprise-Class Storage [Last Updated On: October 17th, 2012] [Originally Added On: October 17th, 2012]
- Citrix and NetApp Collaborate to Simplify Cloud Storage [Last Updated On: October 17th, 2012] [Originally Added On: October 17th, 2012]
- Microsoft Acquires Leader In Cloud-integrated Storage [Last Updated On: October 17th, 2012] [Originally Added On: October 17th, 2012]
- Microsoft Buys StorSimple for Enterprise Cloud Storage [Last Updated On: October 18th, 2012] [Originally Added On: October 18th, 2012]
- FreedomPACS, Radiology PACS and Cloud Image Storage Provider, Releases Results of County Hospital Case Study ... [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Nirvanix Selects Brocade as Networking Backbone for Global Cloud Expansion [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Pogoplug offers unlimited cloud storage for $5 a month [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- NTT Communications Chooses Cloudian® S3 compatible Object Storage Platform for Multi Petabyte Cloud Storage as a Service [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- TwinStrata and Google to Host "Beyond Disaster Recovery: Integrating Cloud Storage into Your IT Strategy" Seminar [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Cloud Storage Reviews Outlines "How SugarSync Works" In Latest Guide [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Symform Challenges Users to Think Beyond Centralized Data Centers With Its 'Byte Me' Promotion [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Avere to tart up FTX with cloud storage gateway, mutterings foretell [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Deals WD My Book Live Personal Cloud Storage 2 TB Network Attached Best Price 2012 - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Create and Manage Your Own Cloud Storage Free - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Free Cloud Space 100GB - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- DuraCloud Brown Bag Series: How DuraCloud is Different From Amazon - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- PocketCloud Explore - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Free 1TB Cloud storage - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Store your files on WEB for free - Unlimited and better than dropbox - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- CloudBackupNow - Retention Policy (with audio) - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- CloudBackupNow - Retention Policy - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- CloudBackupNow - Primer II - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- ERP Data Capture animation - Video [Last Updated On: November 1st, 2012] [Originally Added On: November 1st, 2012]
- Cash rains DOWN on the Cloud - Nasuni trousers $20m [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- My PC Backup Review The Cloud Storage Service For You - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Samsung ATIV S Review - Phones 4u - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Trust Me mv - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Product Webinar: Collaborating and Exchanging Large Data at Distance with Faspex 3.0 - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- DT Daily: Facebook takes aim at Craigslist, Halo 4 reviews a - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- 2 MCSE Private Cloud Storage Basics - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Gladinet Cloud Enterprise Quick Start Guide - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Installing OfficeDrop Mac File Sync - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- OfficeDrop Mac File Sync - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Secure Cloud Storage - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Windows Phone 8: Lenese integrates apps in the camera app - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Graphic Video on Wuala Secure Cloud Storage from Paula Hansen and Chart Magic - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Hurricane Sandy Cheat Meal Run to Tastee Diner - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- Cloud Zow Review - Cloudzow Review | Marketing Secret Revealed - Video [Last Updated On: November 3rd, 2012] [Originally Added On: November 3rd, 2012]
- What is Cloud Storage? - Video [Last Updated On: November 4th, 2012] [Originally Added On: November 4th, 2012]
- Perfume - Chocolate Disco [ hide@BSB Battle In Feb. Remix ] - Video [Last Updated On: November 4th, 2012] [Originally Added On: November 4th, 2012]