Cloud Hosting

Nearly one year ago, I wrote an Amazon.com (AMZN) article titled "Amazon's Graviton2 CPU: This Time, It's For Real". In that article I put forth the theory that Amazon.com's Graviton2 CPU, based on ARM's Neoverse N1 architecture, was finally and truly competitive with established x86 server-room alternatives.

That has come to be true. Right now, if you list AWS Instances (for instance for on-demand usage), you'll see Graviton2 instances prominently displayed in large numbers, among the equivalent x86 alternatives from Intel (INTC) - m5 - and Advanced Micro Devices (AMD) - m5a. A small observation here, Graviton2 instances are up to 20% cheaper than Intel instances, and AMD instances are up to 10% cheaper than Intel instances as well.

Now, at the time I said that the Graviton2 would be competitive with x86 solutions, we still lacked some information. For instance, the Neoverse N1 core design was brought to market to be run at 3GHz. Yet, Amazon.com runs the Graviton2 at 2.5GHz, thus leaving out 20% performance headroom.

Also, to establish competitiveness, I used single-threaded benchmarks. After all, cloud server instances are "sold by the core". I arrived at a calculation which showed the Graviton would exceed traditional x86 performance by around 20% on that performance basis. However, that calculation was arrived at by using a theoretical 3.1GHz frequency, and Amazon.com chose to run it at 2.5GHz.

Still, Amazon.com boasts that the Graviton2 instances are faster than their x86 equivalent, while at the same time being cheaper. Indeed, Amazon.com boasts of a 40% price/performance lead, which would roughly imply a 20% gain from a lower price and a 20% gain from higher performance.

How can that be if Amazon.com is running the chips at a frequency that should erase all its performance lead? The answer is simple and interesting.

This happens because when Amazon.com sells you the use of a Graviton2 instance, for instance m6g.large, it's selling you access to "2 vCPUs". And when it sells you the equivalent m5.large or m5a.large instance, it also sells you access to "2 vCPUs". However, there's a difference between these "vCPUs". In Graviton2's case, each vCPU is a physical CPU core. In the x86 case, each vCPU is a thread on a hyperthreaded core (1 physical core handles 2 physical threads).

Well, it's easy to see that 1 core handling 2 hyperthreaded cores is a lot slower than 2 cores each handling one thread alone. Roughly speaking, 2 x86 cores with hyperthreading disabled (thus equivalent to 1 thread per physical core) will be around 35% faster (or more) than running them through 1 physical core with hyperthreading enabled.

Hence, follow me here:

This is consistent with Amazon.com's claims. It's also consistent with numerous testimonials on AWS's pages, even including Netflix among other well-known customers. Indeed, by and large these customers report higher performance gains than those claimed by Amazon.com.

Here's a small sample of the many testimonials on the Graviton2. I highly recommend reading other testimonials as well:

Now, the above already explains Graviton2's growing popularity. But this article is about what will happen with Graviton3.

The Graviton3 will likely be based on ARM's Neoverse V1 architecture. As I wrote on my recent article titled "Intel And AMD: History Repeats - X86 Faces Obsolescence", the Neoverse V1 performance jump is so large that it's to be expected that the x86 world in general will face significant trouble. This trouble will be exacerbated in the server room.

Here, I should say that the server room of today is evolving to be like a smartphone/tablet. Yes, it sounds weird but that's what's happening. Let me explain

In the server room, higher core counts per CPU are highly sought after. This is obvious, as cloud providers "sell cores", and the more cores you have on each server, the fewer servers you need, thus keeping other costs lower.

So, more cores per CPU are a desirable thing and this is evolving fast (AMD is already up to 64 cores on a chip, so 128 vCPUs). The Graviton2 also has 64 cores in a chip. The Neoverse V1 enables 96 cores on a chip, so the Graviton3 might have 96 cores on a chip. But while the number of cores on a chip explodes quickly, the thermal constraint for each CPU -- which requires cooling -- remains roughly the same (for air cooling) at around 300W per chip, before more expensive water cooling is needed.

So, what happens when you have more cores but the same thermal constraint? You have less power available per core. It's basically the TDP divided by the number of cores, so on 64 cores you have 4.7W available per core (at most - less if you take other components into account). This starts going down towards the smartphone/tablet/low power laptop space. And who dominates there due to the higher efficiency needed to burn less power? ARM. At the other end, the cores being repurposed for server chips by Intel and AMD typically come from PC applications with fewer power constraints.

This very valid thesis was already explained by Nuvia, which then produced this compelling comparison showing the power and performance envelope of recent generation chips/cores:

You can see just how ugly the power/performance envelope looks for x86, even if improving.

Anyway, the Neoverse V1 should at least broadly keep the efficiency of a 2-generation old Neoverse N1 (based on the ARM A76 core, while the V1 will take its cues from the ARM X1), while delivering an extreme performance jump.

But what will happen to Graviton3's performance? Well, Amazon.com will have two choices:

Either performance increase will absolutely destroy existing x86 instances. Even new instances based on the EPYC Milan with Zen3 cores will have no chance. The performance uplift there is expected to be just 15-20%.

Now remember, in practice the Graviton2 already has a 20-40% performance advantage (because of 2 physical cores vs. 1 x86 hyperthreaded core). The Graviton3 will compound on this advantage while not even giving back the entire existing (Graviton2) performance advantage to the EPYC Milan

As a result, Amazon.com will likely boast at least a 50-80% performance advantage on the Graviton3 even over new generation x86 server instances.

To produce or run Graviton3's will remain cheaper for Amazon.com than to do the same with x86 chips. Thus, Amazon.com could likely keep the same 20% discount to entice even more migration towards the Graviton. However, it's also possible that Amazon.com could reap the benefits of an extreme performance advantage to just price the Graviton3 instances at the same price as the x86 instances.

When will the Graviton3 arrive? There aren't many clues to this. However, we can say the following:

Hence, it took about 9 months for the Graviton2 baby to show up.

So, if we draw a parallel for the Graviton3:

Right now, on AWS, the Graviton2 already constitutes the best performance/price alternative. This alternative won't be dethroned before the arrival of the Graviton3.

The Graviton3 will gain from the extreme jump in performance from ARM's Neoverse V1 architecture, before the Graviton2 is even caught on a performance or at least a price/performance basis.

Hence, as Amazon.com launches the Graviton3, the x86 instances in AWS will be fully outclassed. Then, it will be just a matter of whether customers can easily migrate their workloads to Graviton3 instances or not (which will depend on the availability of many other services). Amazon.com has been steadily increasing the range of instances and services the Graviton2 enables, of course.

Finally, other vendors will try to provide the same ARM architecture to other cloud providers. So, the pressure which first started at Amazon.com's AWS will then quickly spread to most cloud vendors.

In my view these dynamics are rather unstoppable.

Idea Generator is my subscription service. It's based on a unique philosophy (predicting the predictable) and seeks opportunities wherever they might be found, by taking into account both valuation (deeply undervalued situations) and a favorable thesis.

Idea Generator has beaten the S&P 500 by around 13% since inception (in May 2015). There is a no-risk, free, 14-day trial available for those wanting to check out the service.

Disclosure: I/we have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours. I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.

View original post here:
Amazon: The Coming Graviton3 - Seeking Alpha