Cloud Hosting

Cloud computing is powering a new age of data and AI by democratizing access to scalable compute, storage, and networking infrastructure and services. Thanks to the cloud, organizations can now collect data at an unprecedented scale and use it to train complex models and generate insights.

While this increasing demand for data has unlocked new possibilities, it also raises concerns about privacy and security, especially in regulated industries such as government, finance, and healthcare. One area where data privacy is crucial is patient records, which are used to train models to aid clinicians in diagnosis. Another example is in banking, where models that evaluate borrower creditworthiness are built from increasingly rich datasets, such as bank statements, tax returns, and even social media profiles. This data contains very personal information, and to ensure that its kept private, governments and regulatory bodies are implementing strong privacy laws and regulations to govern the use and sharing of data for AI, such as the General Data Protection Regulation (GDPR) and the proposed EU AI Act. You can learn more about some of the industries where its imperative to protect sensitive data in this Microsoft Azure Blog post.

Microsoft recognizes that trustworthy AI requires a trustworthy cloudone in which security, privacy, and transparency are built into its core. A key component of this vision is confidential computinga set of hardware and software capabilities that give data owners technical and verifiable control over how their data is shared and used. Confidential computing relies on a new hardware abstraction called trusted execution environments (TEEs). In TEEs, data remains encrypted not just at rest or during transit, but also during use. TEEs also support remote attestation, which enables data owners to remotely verify the configuration of the hardware and firmware supporting a TEE and grant specific algorithms access to their data.

At Microsoft, we are committed to providing a confidential cloud, where confidential computing is the default for all cloud services. Today, Azure offers a rich confidential computing platform comprising different kinds of confidential computing hardware (Intel SGX, AMD SEV-SNP), core confidential computing services like Azure Attestation and Azure Key Vault managed HSM, and application-level services such as Azure SQL Always Encrypted, Azure confidential ledger, and confidential containers on Azure. However, these offerings are limited to using CPUs. This poses a challenge for AI workloads, which rely heavily on AI accelerators like GPUs to provide the performance needed to process large amounts of data and train complex models.

The Confidential Computing group at Microsoft Research identified this problem and defined a vision for confidential AI powered by confidential GPUs, proposed in two papers, Oblivious Multi-Party Machine Learning on Trusted Processors and Graviton: Trusted Execution Environments on GPUs. In this post, we share this vision. We also take a deep dive into the NVIDIA GPU technology thats helping us realize this vision, and we discuss the collaboration among NVIDIA, Microsoft Research, and Azure that enabled NVIDIA GPUs to become a part of the Azure confidential computing ecosystem.

Today, CPUs from companies like Intel and AMD allow the creation of TEEs, which can isolate a process or an entire guest virtual machine (VM), effectively eliminating the host operating system and the hypervisor from the trust boundary. Our vision is to extend this trust boundary to GPUs, allowing code running in the CPU TEE to securely offload computation and data to GPUs.

Unfortunately, extending the trust boundary is not straightforward. On the one hand, we must protect against a variety of attacks, such as man-in-the-middle attacks where the attacker can observe or tamper with traffic on the PCIe bus or on a NVIDIA NVLink connecting multiple GPUs, as well as impersonation attacks, where the host assigns an incorrectly configured GPU, a GPU running older versions or malicious firmware, or one without confidential computing support for the guest VM. At the same time, we must ensure that the Azure host operating system has enough control over the GPU to perform administrative tasks. Furthermore, the added protection must not introduce large performance overheads, increase thermal design power, or require significant changes to the GPU microarchitecture.

Our research shows that this vision can be realized by extending the GPU with the following capabilities:

NVIDIA and Azure have taken a significant step toward realizing this vision with a new feature called Ampere Protected Memory (APM) in the NVIDIA A100 Tensor Core GPUs. In this section, we describe how APM supports confidential computing within the A100 GPU to achieve end-to-end data confidentiality.

APM introduces a new confidential mode of execution in the A100 GPU. When the GPU is initialized in this mode, the GPU designates a region in high-bandwidth memory (HBM) as protected and helps prevent leaks through memory-mapped I/O (MMIO) access into this region from the host and peer GPUs. Only authenticated and encrypted traffic is permitted to and from the region.

In confidential mode, the GPU can be paired with any external entity, such as a TEE on the host CPU. To enable this pairing, the GPU includes a hardware root-of-trust (HRoT). NVIDIA provisions the HRoT with a unique identity and a corresponding certificate created during manufacturing. The HRoT also implements authenticated and measured boot by measuring the firmware of the GPU as well as that of other microcontrollers on the GPU, including a security microcontroller called SEC2. SEC2, in turn, can generate attestation reports that include these measurements and that are signed by a fresh attestation key, which is endorsed by the unique device key. These reports can be used by any external entity to verify that the GPU is in confidential mode and running last known good firmware.

When the NVIDIA GPU driver in the CPU TEE loads, it checks whether the GPU is in confidential mode. If so, the driver requests an attestation report and checks that the GPU is a genuine NVIDIA GPU running known good firmware. Once confirmed, the driver establishes a secure channel with the SEC2 microcontroller on the GPU using the Security Protocol and Data Model (SPDM)-backed Diffie-Hellman-based key exchange protocol to establish a fresh session key. When that exchange completes, both the GPU driver and SEC2 hold the same symmetric session key.

The GPU driver uses the shared session key to encrypt all subsequent data transfers to and from the GPU. Because pages allocated to the CPU TEE are encrypted in memory and not readable by the GPU DMA engines, the GPU driver allocates pages outside the CPU TEE and writes encrypted data to those pages. On the GPU side, the SEC2 microcontroller is responsible for decrypting the encrypted data transferred from the CPU and copying it to the protected region. Once the data is in high bandwidth memory (HBM) in cleartext, the GPU kernels can freely use it for computation.

The implementation of APM is an important milestone toward achieving broader adoption of confidential AI in the cloud and beyond. APM is the foundational building block of Azure Confidential GPU VMs, now in private preview. These VMs, designed in collaboration with NVIDIA, Azure, and Microsoft Research, feature up to four A100 GPUs with 80 GB of HBM and APM technology and enable users to host AI workloads on Azure with a new level of security.

But this is just the beginning. We look forward to taking our collaboration with NVIDIA to the next level with NVIDIAs Hopper architecture, which will enable customers to protect both the confidentiality and integrity of data and AI models in use. We believe that confidential GPUs can enable a confidential AI platform where multiple organizations can collaborate to train and deploy AI models by pooling together sensitive datasets while remaining in full control of their data and models. Such a platform can unlock the value of large amounts of data while preserving data privacy, giving organizations the opportunity to drive innovation.

A real-world example involves Bosch Research, the research and advanced engineering division of Bosch, which is developing an AI pipeline to train models for autonomous driving. Much of the data it uses includes personal identifiable information (PII), such as license plate numbers and peoples faces. At the same time, it must comply with GDPR, which requires a legal basis for processing PII, namely, consent from data subjects or legitimate interest. The former is challenging because it is practically impossible to get consent from pedestrians and drivers recorded by test cars. Relying on legitimate interest is challenging too because, among other things, it requires showing that there is a no less privacy-intrusive way of achieving the same result. This is where confidential AI shines: Using confidential computing can help reduce risks for data subjects and data controllers by limiting exposure of data (for example, to specific algorithms), while enabling organizations to train more accurate models.

At Microsoft Research, we are committed to working with the confidential computing ecosystem, including collaborators like NVIDIA and Bosch Research, to further strengthen security, enable seamless training and deployment of confidential AI models, and help power the next generation of technology.

The Confidential Computing team at Microsoft Research Cambridge conducts pioneering research in system design that aims to guarantee strong security and privacy properties to cloud users. We tackle problems around secure hardware design, cryptographic and security protocols, side channel resilience, and memory safety. We are also interested in new technologies and applications that security and privacy can uncover, such as blockchains and multiparty machine learning. Please visit our careers page to learn about opportunities for both researchers and engineers. Were hiring.

Read more:
Powering the next generation of trustworthy AI in a confidential cloud using NVIDIA GPUs - Microsoft

U.S. Army Soldiers assigned to Americas First Corps maneuver a Stryker combat vehicle off United States Naval Ship City of Bismarck while conducting roll on-roll off training at Naval Base Guam, Feb. 9, 2022. (U.S. Army/ailene Bautista)

WASHINGTON: The US Armys I Corps is testing a new distributed mission command concept that could fundamentally change how the Army Corps functions across the vast distances of the Indo-Pacific.

Instead of the I Corps maintaining a single headquarters with several hundred staff shuffling around, the service is looking at breaking down the traditional headquarters infrastructure into functional nodes that would be distributed across the region but can remain in constant communication, Brig. Gen. Patrick Ellis, I Corps chief of staff, told Breaking Defense in a recent interview.

If were running the Corps and performing our command and control functions from six, three, five, six locations as opposed to just one, were much more resilient, and honestly much more survivable, in the event that were ever targeted, said Ellis. The existing Corps structure the kind of the doctrinal, the way the Corps are built didnt necessarily make the most sense for us.

The Corps and Army formations from large to small want to move away from static command posts or headquarters marked by tents, trucks and generators, and fight in a distributed manner, meaning that a Corps can coordinate the battlefield using assets that are hundreds, if not thousands, of miles apart.

In a recent experiment in Guam using four Strykers loaded with advanced communication capabilities, the I Corps worked to prove that it can pass important battlefield data, including fires and targeting information, between platforms, even while they are in transit in the air or on a boat. For example, Ellis was able to send mission command information from a Stryker, in mid-air transit aboard a C-17 headed to Guam, back to Joint Base Lewis-McChord using the airplanes antenna. Additionally, the Stryker element succeeded in sending information while plugged into the network of the Navys USNS City of Bismarck, a military sealift ship, though it remained portside.

U.S. Army Soldiers assigned to Americas First Corps and service members assigned to Joint Communications Support Element establish communications onboard the U.S. Naval Ship City of Bismarck, Feb. 16, 2022, as part of a joint training operation to experiment and exercise distributed mission command in the Indo-Pacific. (Army/Jailene Bautista,)

Instead of losing situational awareness, like we do now either in flight or in transit when your stuffs on a vessel, we figured out that if we could use the transport thats on these vessels and perform mission command functions and stay situationally aware as were transiting from one location to another, he said.

Related: At Project Convergence, Army struggling to see joint battlefield as it heeds hard lessons

Its worth reiterating what happened here: from the back of the C-17, the Army conducted a video teleconference. While that may not sound impressive given the last two years of remote work, its an important feat given the high bandwidth requirements to support live video particularly on a military aircraft in the air. Ellis said thats not a function the Corps would always use, but it proved that we could move large amounts of data.

Second, and perhaps a more germane function for the Corps, is that it was able to pass targeting data from the Armys Field Artillery Tactical Data System from the aircraft to the ground.

Its a useful capability for us because it allows us to provide updates in-flight from mission commander back down to launchers, Ellis said. Or in the event that we flipped it and we put some of our HIMARS launchers aboard the aircraft and we can actually update their targeting data from the ground while theyre in route.

The ability to update targeting data while flying for several hours would prevent targeting data from going stale, he added.

The Corps experiment will also help feed into Joint All-Domain Command and Control, the Pentagons future warfighting construct in which sensors and shooters across the battlespace are connected to provide up-to-date information.

The opportunity to operate with the Joint Forces is key in this organization, said Col. Elizabeth Casely, I Corps G6, or the network manager. The ability to sense from a different service component and fire from a different service component is predominant for JADC2.

Enabling technologies

Central to the whole concept is cloud computing at the edge. Using the cloud, soldiers at the different nodes can access the data they need to accomplish their mission and operate more efficiently. However, soldiers will need to bring forward some network hardware to have some immediate access to data in an environment where they were disconnected from the hyper-scale cloud hub thousands of miles away.

So instead of having to bring a separate computer to do fires, a separate computer to do common operational picture and a separate computer to do logistics, we could actually access all those systems through one standard laptop, Ellis said. You bring your little slice of the cloud forward with you when you come forward, so in the event that you are disconnected from the hyper-scale, you can continue to perform your mission command functions.

But the challenge that the Corps has to consider is how much data and what types of mission command data are vital for soldiers to bring forward with them, versus how much can remain in the main cloud hub. The I Corps is working through what information exchanges absolutely have to occur, Casely said.

For example, Ellis said its Corps-level fires personnel are being asked questions about how much data they need to do their job do they need all of their targeting data or just imagery, and how long before a mission? Thats a question similar to one from Project Convergence, where the Army grappled with what data has to be sent, and in what format, while not eating up the limited bandwidth in a conflict zone.

It really, really challenges not just the folks in the G6, but the entire staff and the entire whole staff process to think about what information needs to be exchanged, in order for them to perform their mission, Casely said.

Another challenge is deconflicting updates in the cloud system, Ellis said. If one group of soldiers come out of the disconnected environment and update the broader cloud, while someone else is updating the same type of information in the larger cloud, how do the soldiers sort out whose data is the most relevant?

Were in the very nascent stages of determining how some of the cloud computing works to support this manner of fighting, Casely said. But Im feeling really, really good about where were headed.

More:
Inside the Army's distributed mission command experiments in, and over, the Pacific - Breaking Defense