Performance is paramount when developing web applications. A slow, unresponsive application results in poor user experience, losing users, and possibly business. For ASP.NET Core developers, there are many techniques and best practices to optimize application performance. Lets explore some of these approaches in this article.

When we talk about performance, the first thing to ask is Where is the issue? Without understanding where the bottlenecks are, we could end up optimizing parts of our application that wont have a significant impact on overall performance.

There are many tools and techniques to identify performance bottlenecks in an ASP.NET Core application:

One of the simplest approaches is to add logging and metrics to your application. You can measure how long operations take and log any issues that occur.

ASP.NET Core supports a logging API that works with a variety of built-in and third-party logging providers. You can configure the built-in logging providers to output logs to the console, debug, and event tracing.

Heres an example of how you can use the ILogger service to log the execution time of a method:

public IActionResult Index(){var watch = Stopwatch.StartNew();

// Code to measure goes here...

watch.Stop();var elapsedMs = watch.ElapsedMilliseconds;_logger.LogInformation("Index method took {ElapsedMilliseconds}ms", elapsedMs);

return View();}}

A more advanced way to identify performance bottlenecks is to use a profiler. A profiler is a tool that monitors the execution of an application, recording things like memory allocation, CPU usage, and other metrics.

There are many profilers available, including:

Application Performance Management (APM) tools go a step further, providing in-depth, real-time insights into an applications performance, availability, and user experience. APM tools can identify performance bottlenecks in real-world scenarios, not just in development and testing.

Asynchronous programming is a way to improve the overall throughput of your application on a single machine. It works by freeing up a thread while waiting for some IO-bound operation (such as a call to an external service or a database) to complete, rather than blocking the thread until the operation is done. When the operation is complete, the framework automatically assigns a thread to continue the execution.

The result is that your application can handle more requests with the same number of threads, as those threads can be used to serve other requests while waiting for IO-bound operations to complete.

ASP.NET Core is built from the ground up to support asynchronous programming. The framework and its underlying I/O libraries are asynchronous to provide maximum performance.

Heres how you might write an asynchronous action in an ASP.NET Core controller:

In this example, GetDataAsync might be making a call to a database or an external service. By awaiting this method, the thread executing this action can be freed up to handle another request.

Heres an example of how you might use async in a service that calls Entity Framework Core:

public MyService(MyDbContext context){_context = context;}

public async Task GetDataAsync(){return await _context.MyData.OrderBy(d => d.Created).FirstOrDefaultAsync();}}

Caching is an effective way to boost the performance of your ASP.NET Core applications. The basic idea is simple: instead of executing a time-consuming operation (like a complex database query) every time you need the result, execute it once, cache the result, and then just retrieve the cached result whenever you need it.

ASP.NET Core provides several built-in ways to cache data:

In-memory caching is the simplest form of caching. It stores data in the memory of the web server. This makes accessing the cached data extremely fast.

In-memory caching in ASP.NET Core stores cache data in the memory of the web server. The data is stored as key-value pairs and can be any object. The access to the in-memory cache is extremely fast, making it an efficient way to store data thats accessed frequently.

One thing to note about in-memory caching is that the cache data is not shared across multiple instances of the application. If you run your application on multiple servers, or if you use a process-per-request model, then the in-memory cache will be separate for each instance or process.

In-memory caching can be an effective way to improve the performance of your application in the following scenarios:

Heres an example of how you might use in-memory caching in an ASP.NET Core controller:

public MyController(IMemoryCache cache){_cache = cache;}

public IActionResult Index(){string cacheEntry;

if (!_cache.TryGetValue("_MyKey", out cacheEntry)) // Look for cache key.{// Key not in cache, so get data.cacheEntry = GetMyData();

// Set cache options.var cacheEntryOptions = new MemoryCacheEntryOptions()// Keep in cache for this time, reset time if accessed..SetSlidingExpiration(TimeSpan.FromMinutes(2));

// Save data in cache._cache.Set("_MyKey", cacheEntry, cacheEntryOptions);}

return View(cacheEntry);}

private string GetMyData(){// Simulating a time-consuming operationThread.Sleep(2000);return "Hello, world!";}}

In this example, the GetMyData method simulates a time-consuming operation. This could be a complex database query, a call to an external service, or any operation that takes time to execute. By caching the result, we avoid the need to execute this operation every time the Index action is called.

Distributed caching involves using a cache thats shared by multiple instances of an application. ASP.NET Core supports several distributed cache stores, including SQL Server, Redis, and NCache.

When using a distributed cache, an instance of your application can read and write data to the cache. Other instances can then read this data from the cache, even if theyre running on different servers.

You should consider using distributed caching in the following scenarios:

When we talk about improving the performance of web applications, one area often overlooked is the size of the HTTP responses. Large responses take longer to transmit over the network, and this latency can have a significant impact on performance, especially for clients with slow network connections.

Response compression is a simple and effective way to reduce the size of HTTP responses, thereby improving the performance of your application. It works by compressing the response data on the server before sending it to the client. The client then decompresses the data before processing it. This process is transparent to the end user.

The most common compression algorithms used for response compression are Gzip and Brotli. They can significantly reduce the size of responses, often by 70% or more.

ASP.NET Core includes middleware for response compression. To enable it, you need to add the middleware to your Startup.ConfigureServices and Startup.Configure methods, like this:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env){app.UseResponseCompression();

// Other middleware...}

By default, the response compression middleware compresses responses for compressible MIME types (like text, JSON, and SVG). You can add additional MIME types if necessary.

You should also configure your server (like IIS, Kestrel, or HTTP.sys) to use dynamic compression. This ensures that your responses are compressed even if youre not using the response compression middleware (for example, for static files).

While response compression can improve performance, there are a few things to keep in mind:

Entity Framework Core (EF Core) is a powerful Object-Relational Mapper (ORM) that simplifies data access in your .NET applications. However, if used without consideration for its performance behavior, you can end up with an inefficient application. Here are some techniques to improve the performance of your applications that use EF Core:

Lazy loading is a concept where the related data is only loaded from the database when its actually needed. On the other hand, Eager loading means that the related data is loaded from the database as part of the initial query.

While lazy loading can seem convenient, it can result in performance issues due to the N+1 problem, where the application executes an additional query for each entity retrieved. This can result in many round-trips to the database, which increases latency.

Eager loading, where you load all the data you need for a particular operation in one query using the Include method, can often result in more efficient database access. Here's an example:

In this example, each Order and its related Customer are loaded in a single query.

When you query data, EF Core automatically tracks changes to that data. This allows you to update the data and persist those changes back to the database. However, this change tracking requires additional memory and CPU time.

If youre retrieving data that you dont need to update, you can use the AsNoTracking method to tell EF Core not to track changes. This can result in significant performance improvements for read-only operations.

EF Core 5.0 and above support batch operations, meaning it can execute multiple Create, Update, and Delete operations in a single round-trip to the database. This can significantly improve performance when modifying multiple entities.

In this example, all the new orders are sent to the database in a single command, rather than one command per order.

Try to filter data at the database level rather than in-memory to reduce the amount of data transferred and memory used. Use LINQ to create a query that the database can execute, rather than filtering the data after its been retrieved.

In this example, only the orders from the last seven days are retrieved from the database.

The Select N+1 issue is a common performance problem where an application executes N additional SQL queries to fetch the same data that could have been retrieved in just 1 query. EF Cores Include and ThenInclude methods can be used to resolve these issues.

This query retrieves all orders, their related Customers, and the Addresses of the Customers in a single query.

When your application needs to interact with a database, it opens a connection to the database, performs the operation, and then closes the connection. Opening and closing database connections are resource-intensive operations and can take a significant amount of time.

Connection pooling is a technique that can help mitigate this overhead. It works by keeping a pool of active database connections. When your application needs to interact with the database, it borrows a connection from the pool, performs the operation, and then returns the connection to the pool. This way, the overhead of opening and closing connections is incurred less frequently.

Connection pooling is automatically handled by the .NET Core data providers. For example, if you are using SQL Server, the SqlConnection object automatically pools connections for you.

When you create a new SqlConnection and call Open, it checks whether there's an available connection in the pool. If there is, it uses that connection. If not, it opens a new connection. When you call Close on the SqlConnection, the connection is returned to the pool, ready to be used again.

You can control the behavior of the connection pool using the connection string. For example, you can set the Max Pool Size and Min Pool Size options to control the size of the pool.

Optimizing the performance of your ASP.NET Core applications can be a challenging task, especially when youre dealing with complex, data-rich applications. However, with the right strategies and tools at your disposal, its a task thats well within your reach.

In this article, weve explored several key strategies for performance optimization, including understanding performance bottlenecks, leveraging asynchronous programming, utilizing different types of caching, compressing responses, optimizing Entity Framework Core usage, and taking advantage of advanced features such as connection pooling and HTTP/2.

The key takeaway here is that performance optimization is not a one-time event, but a continuous process that involves monitoring, analysis, and iterative improvement. Always be on the lookout for potential bottlenecks, and remember that sometimes the smallest changes can have the biggest impact.

Moreover, while we focused on ASP.NET Core, many of these principles and techniques apply to web development in general. So, even if youre working in a different framework or language, dont hesitate to apply these strategies. The ultimate goal of performance optimization is to provide a smooth, seamless experience for your users.

Happy coding, and heres to fast, efficient applications!

Original post:
Turbocharging ASP.NET Core Applications: A Deep Dive into ... - Medium

Opinion All tech grunts know an update from a vendor can be good news, bad news, or both. Fortunately, theres a quick way to tell even before the first sentence of the community blog post that is today's royal proclamation of choice. If the person addressing the community is an engineer, its good news. If marketing, not so much.

Hence the sinking feeling when the renaming of Azure AD to Microsoft Entra ID was revealed to the community (thats you) by a General Marketing Manager. This isnt a dig at marketing or marketing people in general, nor exalting engineering as a higher, purer, more noble calling. Even though it is. Nor that a cartoon penguin as the entire Linux marketing department has done a lot less damage. Even though it has.

The trouble is that people who have to make things work have little interest in marketing, at least as something of immediate interest. If things are changing, people need to hear the hows and whys from those who live that work and if youre doing daily hands-on enterprise IT infrastructure, you want to hear from senior devs, PMs or higher. If you dont get that, the vendor doesnt know or doesnt care what you need, or has to bluff that the incoming change isnt just putting a pile of work on you for its own benefit.

In this case, the story is that the rebranding of Azure AD to Entra makes no difference to anyone, its just a name, man. Which is true if you dont have to update management documents, procedures, licenses, training materials, asset portfolios, and so on. Marketing people dont see this, they see a clean name change.

There are good reasons for new names, when there are new things to be named. There are good reasons for old names, when they refer to important, established ideas. Unfortunately for us, old names also have what marketing people call mindshare. Thats why celebrities get big bucks for smiling at consumer goods - the marketing people know some of that positive mind-share the celeb has will spill over. Its a shortcut to our psyches. Enterprise IT doesnt really do influencers, which saves Linus from hawking manscaping razors, but we do have big name technologies.

Thats why, when Microsoft needed to put identity management into the cloud, it decided to retain the Active Directory name in Azure AD, despite it being a bad fit and the two products doing quite different things. The cloud is not your on-prem collection of LED flashers. Active Directory hails from 2000; it knows about PCs and group policies and physical servers. Azure knows about the virtualised landscape of cloud resources.

The arrival of Azure ID management would have been a great time for a new name, too, that reflected the fundamental change in philosophy. Azure AD could not replace Active Directory, and Active Directory could not manage cloud services.

Yet the pull of the old was too strong to resist. Heck, this was in the days when Microsofts cloud was still called Windows Azure a legacy of it being born in 2010 under Steve Ballmers watch. It was only itself renamed to Microsoft Azure less than 10 years ago. Upwards compatibility had to be implied in name even if lacking in fact.

It was lacking. The two identity managers couldnt even share identity management at the simplest level. Users had to use both, which of course they did? Then they had to have two separate IDs, which would not be a problem. It just means that if you have a password change policy that users will have to do this twice (and they could of course choose the same password for both). Tell me youve never managed users without saying youve never managed users.

But now Azure AD has been around long enough to be bedded in, marketing no longer sees Windows as a selling point. Time to rename it and make it just part of the Entra brand of things that can be licensed in so many creative and lucrative ways. Let the rest of the industry pay for the bulk of the rebranding consequences. Marathon to Snickers, Windscale atomic disaster site to Sellafield. Does it matter?

Much more than many might think. In most of the world, the names we give to things are just that names. They dont form part of the thing itself. You can recap a vintage IBM PC, and it doesnt matter that capacitors were once called condensers, or that IBM called printed circuit boards planars. Both work just fine if you call them Freds and Cynthias. But in the world of code and data, or objects and procedures, or APIs and spaces, the names are an intrinsic feature. If youre building something new, you name a lot of its internals after the whole thing. Your APIs will reference the name,, your internal code structure will identify themselves thus, and other software, tools and automation will use that name. Its part of the hierarchy by which everything works.

So when, for marketing purposes, the package changes its name, there are just two choices. Rename all those references, which has the potential for a rolling fustercluck that scales faster than an Oracle invoice after an audit, or leave well alone and watch the code fossilize, like a forgotten kingdom known only for the placenames it leaves behind. Its not that this is impossible to manage, but youd think a world where keeping old systems alive is increasingly difficult, IT vendors would have some idea of the responsibility they have for future, that they understood something about the unique nature of the industry theyre part of.

Naw. Only engineers worry about those sorts of horizons. Theres a saying that marketing is the price a company pays for being boring; in IT, we all get to chip in too.

See more here:
Goodbye Azure AD, Entra the drag on your time and money - The Register

The growth and success of computing has been a story about creating ways to make cross-platform connections.

The rise of hypertext markup language, or HTML, as a common language for web page creation in the 1990s allowed a broad audience to fuel internet growth. Bluetooth emerged in the early 2000s as a short-range wireless standard that enabled cross-device communication.

This same story is now being written in the cloud world, specifically the supercloud, a hybrid and multicloud abstraction layer that sits above and across hyperscale infrastructure. As theCUBE, SiliconANGLE Medias livestreaming studio, prepares to hold its Supercloud 3 eventon July 18, the landscape is continuing to evolve as a growing number of major companies are taking steps to build cross-cloud services. This is being driven by customers who prefer having multiple clouds yet are frustrated by the challenge of having to manage them.

Its too complicated to really take advantage of multicloud to the degree theyd like without engaging outside talent, said Dave Vellante, industry analyst for theCUBE, SilconANGLE Medias livestreaming studio, in a recent Breaking Analysis post. Thats an opportunity, thats supercloud a mesh of multiple clouds that are interconnected and managed as a single entity. Cross-cloud simplification is needed and will deliver business benefits in terms of getting more done with less, faster and more securely.

The technology driving supercloud has advanced since theCUBE held its inauguralSupercloud event last August. Soon after the event, VMware Inc. unveiled new solutions for its Cross-Cloud Services portfolio, including Cloud Foundation+ for managing and operating full stack hyperconverged infrastructure in data centers and two projects designed to provide multicloud networking and advanced cross-platform security controls.

Cloudflare Inc. has built the equivalent of a distributed supercomputer that connects multiple clouds and can allocate resources at a large scale. In June, the company announced an agreement with Databricks Inc. to more simply share and collaborate on live data across clouds.

The technology that will make up a supercloud is getting better constantly: AIOps, cross-cloud security, cross-cloud data management, etc., said David Linthicum, chief cloud strategy officer of Deloitte Consulting, in an interview for this story. Suitable investments have been made in that tech over the past year.

Where does this leave the major cloud providers such as Amazon Web Services Inc., Microsoft Corp. and Google LLC? The hyperscalers have provided the foundation on which superclouds were built, such as in the case of AWS serving as the host platform for Snowflake Inc.

There have been signs of interest among hyperscalers in providing cross-cloud services, such as Microsofts continued partnership with Oracle Corp. to deliver streamlined access to Oracle databases for Azure customers via Oracle Cloud Infrastructure. In late May, Google LLC. announced Cross-Cloud Interconnect services through Google Cloud.

Yet there is a growing belief among some analysts that the hyperscalers will ultimately have to do more.

The hyperscalers should be more focused on this space, in my opinion, Linthicum said. I think it is an opportunity for them, as well as a moment to invest in existing supercloud players, which are smaller companies.

Some of those smaller companies are working on solutions for architecting at the edge. This emerging field of edge architecture for supercloud represents a growth opportunity for dealing with the volume and scale of connected devices and decisions around where data will be processed.

Its no longer a uniform set of compute and storage resources that are available at your disposal, said Priya Rajagopal, director of product management at the database company Couchbase Inc., in an interview with theCUBE Youve got a variety of IoT devices. Youve got mobile devices, different processing capabilities, different storage capabilities. When it comes to edge data centers, its not uniform in terms of what services are available.

Couchbases NoSQL database technology powers complex business applications and its Capella offering provides a cloud-hosted database as a service that is available on AWS, Microsoft Azure and Google Cloud.Couchbases technology offers an intriguing view of how supercloud can help solve connectivity issues at the edge. Data sync at the edge is a challenge when low network connectivity can be common in remote locations.

Couchbase solves this problem through the use of global load balancers that can redirect traffic in a connectivity failure. Applications continue to run and then are automatically synced to backend servers when connectivity resumes.

I think once you start going from the public cloud, the clouds there scale, said Muddu Sudhakar, co-founder and CEO of Aisera Inc., in an interview on theCUBE. The lack of computer function will kick in. I think everything should become asynchronous. I think as long as algorithms can take that into the edge, I think that superclouds can really bridge between the public cloud to the edge.

While supercloud offers promise for solving the nettlesome problem of processing data at the edge, its potential for the delivery of key cross-cloud services, such as AI and security, remains a central focus for the enterprise.

Vellante has termed the evolving enterprise model as an AI-powered hybrid-multi-supercloud, and the role of artificial intelligence cannot be underestimated. The rise of ChatGPT, an OpenAI LP tool for generating articulate, human-like responses to a wide range of queries, has opened the gates for AI adoption on a massive scale.

Where the supercloud converges with ChatGPT and other AI tools will be in the delivery of services in an abstraction layer that deconstructs complexity, according to Howie Xu, former vice president of machine learning and artificial intelligence at Zscaler Inc. and now with Palo Alto Networks Inc. In order for me as a developer to create applications, I have so many things to worry about, and thats complexity, said Xu, in an interview on theCUBE. But with ChatGPT, with the AI, I dont have to worry about it. Those kinds of details will be taken care of by the underlying layer.

As businesses increasingly put data foundations in place to take full advantage of hybrid multicloud environments, management of this infrastructure and the massive amount of information required by AI tools will create further demand for cross-cloud connectivity.

The growth of AI will drive more use cases that span clouds, Linthicum said. These AI systems need vast amounts of data to work well, and that data will be in any number of cloud platforms and needs to be accessed. Since the relocation of data is typically not cost viable, the data will be leveraged where it exists and thus the need to manage data integration at the supercloud level. This will drive tremendous growth.

Perhaps one of the most difficult services to deliver in a supercloud model is security. This is because the cloud is complex and there is a lack of visibility for workloads operating in software containers that drive modern applications.

In theCUBEs Supercloud 2 event in January, Piyush Sharma, founder of Accurics (acquired by Tenable Inc.), called for common standards that would help implement consistent security practices across cloud models.

I think we need a consortium, we need a framework that defines that if you really want to operate in supercloud, Sharma said. Otherwise, security is going everywhere. [SecOps] will have to fix everything, find everything its not going to be possible.

Not long after Sharma appeared on theCUBE, there was movement toward implementing the common framework he described. AWS, Splunk Inc. and over a dozen other firms announced the launch of the Open Cybersecurity Schema Framework.

The goal of OCSF is to streamline the processing of data after cyberattacks and reduce the amount of manual work required to share information between security tools. In November, Cribl Inc. announced its support for OCSF and the use of Cribl Stream by AWS customers for converting data from any source into the OCSF format.

Security is challenging because of the complexity of the security models that run within each native cloud provider, Linthicum said. You somehow need to figure out how to abstract and automate those services so they can function with some cross-cloud commonality. This needs to be a focus for the supercloud tech providers, even the hyperscalers.

Over the past year, supercloud has moved beyond being merely a platform built by others successfully on the hyperscalers into a new phase where enterprises are clamoring for unified cross-cloud services and companies are lining up to provide them. With a nod to the author Norman Mailer, supercloud has come to the supermarket.

TheCUBEis an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate thecontent you create as well Andy Jassy

THANK YOU

Originally posted here:
Supercloud comes to the supermarket: How growing interest in ... - SiliconANGLE News

Munich Terminal 1 Airline Club, a group of carriers operating from terminal 1 in Munich, Germanys second-largest airport, has adopted Amadeus cloud technology to simplify and improve the passenger service operations of its members.

The club is a group of airlines responsible for selecting and managing shared technology at key service points like check-in and boarding.

Providing IT infrastructure that multiple airlines share at the airport can be complex. Previously, multiple costly network links needed to be maintained between each airline and Munich airport, with computing happening on hard-to-maintain servers located at the terminal.

Passenger service agents connected to these local servers with energy-intensive traditional PCs in order to access multiple airline systems.

With the objective of improving the efficiency of this shared infrastructure, Munich T1 Airline Club has now migrated 330 workstations at check-in counters, boarding gates and lost & found desks to the Amadeus Airport Cloud Use Service (ACUS).

The move eliminates the need for local servers and costly legacy networks. Instead, agents can now access any Departure Control System they need using energy-efficient thin client machines, which connect to the cloud using a single cost-effective internet link.

Patrik Toepfner, Chairman of the Munich T1 Airline Club, Munich Airport said: Weve selected Amadeus as our partner for shared infrastructure because its cloud model greatly simplifies the technology we use at the airport.

We are confident this choice will streamline our operations and improve the overall travel experience for passengers at Munich terminal 1.

Yannick Beunardeau, SVP Airport & Airline Operations EMEA, Amadeus added: A growing number of airports and airlines are recognizing the simplicity of accessing passenger service technology from the cloud.

With this modern approach, agents can focus on delivering the best possible service to passengers using any airline system they need through an internet browser.

Were seeing specialist software at airports become more like the simple consumer applications we use in our personal lives and that can only be a good thing.

See the original post:
Airlines at Munich Airport streamline IT with Amadeus cloud ... - Airline Routes & Ground Services

Digital Realty's CEO and more on what generative AI means for the data center industry

A potential shift in the nature of workloads will filter down to the wider data center industry, impacting how they are built and where they are located.

Digital Realtys CEO Andy Power believes that generative AI will lead to a monumental wave of demand.

It's still new as to how it plays out in the data center industry, but it's definitely going to be large-scale demand. Just do the math on these quotes of spend and A100 chips and think about the gigawatts of power required for them.

When he joined the business nearly eight years ago we were moving from one to three megawatt IT suites, and we quickly went to six to eight, then tens, he recalled. I think the biggest building we built was 100MW over several years. And the biggest deals we'd sign were 50MW-type things. Now you're hearing some more deals in the hundreds of megawatts, and I've had preliminary conversations in the last handful of months where customers are saying talk to me about a gigawatt.

For training AI models, Power believes that well see a change from the traditional cloud approach which focuses on splitting up workloads across multiple regions while keeping it close to the end user.

Given the intensity of compute, you cant just break these up and patchwork them across many geographies or cities, he said. At the same time, you're not going to put this out in the middle of nowhere, because of the infrastructure and the data exchange.

These facilities will still need close proximity to other data centers with more traditional data and workloads, but the proximity and how close that AI workload needs to sit relative to cloud and data is still an unknown.

He believes that it will still be very major metro focused, which will prove a challenge because youre going to need large swaths of contiguous land and power, but its harder and harder to find a contiguous gigawatt of power, he said, pointing to transmission challenges in Virginia and elsewhere.

As for the data centers themselves, plain and simple, it's gonna be a hotter environment, you're just going to put a lot more power-dense servers in and you're gonna need to innovate your existing footprints, and your design for new footprints, he said.

We've been innovating for our enterprise customers in terms of looking at liquid cooling. It's been quite niche and trial, to be honest with you, he said. We've also been doing co-design with our hyperscale customers, but those have been exceptions, not the norms. I think you're gonna see a preponderance of more norms.

Moving forward, he believes that you'll have two buildings that will be right next to each other and one will be supporting hybrid cloud. And then you have another one next to it that is double or triple the size, with a different design, and a different cooling infrastructure, and a different power density.

Amazon agrees that large AI models will need specialized facilities. Training needs to be clustered, and you need to have really, really large and deep pools of a particular capacity, AWS Chetan Kapoor said.

The strategy that we have been executing over the last few years, and we're going to double down on, is that we're going to pick a few data centers that are tied to our main regions, like Northern Virginia (US-East-1) or Oregon (US-West-2) as an example, and build really large clusters with dedicated data centers. Not just with the raw compute, but also couple it with storage racks to actually support high-speed file systems.

On the training side, the company will have specialized cluster deployments. And you can imagine that we're going to rinse and repeat across GPUs and Trainium, Kapoor said. So there'll be dedicated data centers for H100 GPUs. And there'll be dedicated data centers for Trainium.

Things will be different on the inference side, where it will be closer to the traditional cloud model. The requests that we're seeing is that customers need multiple availability zones, they need support in multiple regions. That's where some of our core capability around scale and infrastructure for AWS really shines. A lot of these applications tend to be real-time in nature, so having the compute as close as possible to the user becomes super, super important.

However, the company does not plan to follow the same dense server rack approach of its cloud competitors.

Instead of packing in a lot of compute into a single rack, what we're trying to do is to build infrastructure that is scalable and deployable across multiple regions, and is as power-efficient as possible, Kapoor said. If you're trying to densely pack a lot of these servers, the cost is going to go up, because you'll have to come up with really expensive solutions to actually cool it.

Googles Vahdat agreed that we will see specific clusters for large-scale training, but noted that over the longer term it may not be as segmented. The interesting question here is, what happens in a world where you're going to want to incrementally refine your models? I think that the line between training and serving will become somewhat more blurred than the way we do things right now.

Comparing it to the early days of the Internet, where search indexing was handled by a few high-compute centers but is now spread across the world, he noted: We blurred the line between training and serving. You're gonna see some of that moving forward with this.

While this new wave of workload risks leaving some businesses in its wake, Digital Realtys CEO sees this moment as a rising tide to raise all ships, coming as a third wave when the second and first still haven't really reached the shore.

The first two waves were customers moving from on-prem to colocation, and then to cloud services delivered from hyperscale wholesale deployments.

Thats great news for the industry, but one that comes after years of the sector struggling to keep up. Demand keeps out-running supply, [the industry] is bending over coughing at its knees because it's out of gas, Power said. The third wave of demand is not coming at a time that is fortuitous for it to be easy streets for growth.

Our largest feature ever looks at the next wave of computing

17 Apr 2023

For all its hopes of solving or transcending the challenges of today, the growth of generative AI will be held back by the wider difficulties that have plagued the data center market - the problems of scale.

How can data center operators rapidly build out capacity at a faster and larger scale, consuming more power, land, and potentially water - ideally all while using renewable resources and not causing emissions to balloon?

Power constraints in Northern Virginia, environmental concerns, moratoriums, nimbyism, supply chain problems, worker talent shortages, and so on, Power listed the external problems.

And that ignores the stuff that goes into the data centers that the customer owns and operates. A lot of these things are long lead times, with GPUs currently hard for even hyperscalers to acquire, causing rationing.

The economy has been running hot for many years now, Power said, And it's gonna take a while to replenish a lot of this infrastructure, bringing transmission lines into different areas. And it is a massive interwoven, governmental, local community effort.

While AI researchers and chip designers face the scale challenges of parameter counts and memory allocation, data center builders and operators will have to overcome their own scaling bottlenecks to meet the demands of generative AI.

We'll continue to see bigger milestones that will require us to have compute not become the deterrent for AI progress and more of an accelerant for it, Microsofts Nidhi Chappell said. Even just looking at the roadmap that I am working on right now, it's amazing, the scale is unprecedented. And it's completely required.

As we plan for the future, and try to extrapolate what AI means for the data center industry and humanity more broadly, it is important to take a step back from the breathless coverage that potentially transformational technologies can engender.

After the silicon boom, the birth of the Internet, the smartphone and app revolution, and cloud proliferation, innovation has plateaued. Silicon has gotten more powerful, but at slower and slower rates. Internet businesses have matured, and solidified around a few giant corporations. Apps have winnowed to a few major destinations, rarely displaced by newcomers. Each new smartphone generation is barely distinguishable from the last.

But those who have benefitted from the previous booms remain paranoid about what could come next and displace them. Those who missed out are equally seeking the next opportunity. Both look to the past and the wealth generated by inflection points as proof that the next wave will follow the same path. This has led to a culture of multiple false starts and overpromises.

The metaverse was meant to be the next wave of the Internet. Instead, it just tanked Meta's share price. Cryptocurrency was meant to overhaul financial systems. Instead, it burned the planet, and solidified wealth in the hands of a few. NFTs were set to revolutionize art, but rapidly became a joke. After years of promotion, commercial quantum computers remain as intangible as Schrodingers cat.

Generative AI appears to be different. The pace of advancement and the end results are clearly evidence that there are more tangible use cases. But it is notable that crypto enthusiasts have rebranded as AI proponents, and metaverse businesses have pivoted to generative ones. Many of the people promoting the next big thing could be pushing the next big fad.

The speed at which a technology advances is a combination of four factors: The intellectual power we bring to bear, the tools we can use, luck, and the willingness to fund and support it.

We have spoken to some of the minds exploring and expanding this space, and discussed some of the technologies that will power what comes next - from chip-scale up to data centers and the cloud.

But we have not touched on the other two variables.

Luck, by its nature, cannot be captured until it has passed. Business models, on the other hand, are usually among the easier subjects to interrogate. Not so in this case, as the technology and hype outpace attempts to build sustainable businesses.

Again, we have seen this before with the dotcom bubble and every other tech boom. Much of it is baked into the Silicon Valley mindset, betting huge sums on each new tech without a clear monetization strategy, hoping that the scale of transformation will eventually lead

to unfathomable wealth.

Higher interest rates, a number of high-profile failures, and the collapse of Silicon Valley Bank has put such a mentality under strain.

At the moment, generative AI companies are raising huge sums on the back of wild promises of future wealth. The pace of evolution will depend on how many can escape the gravity well of scaling and operational costs, to build realistic and sustainable businesses before the purse strings inevitably tighten.

And those eventual winners will be the ones to define the eventual shape of AI.

We do not yet know how expensive it will be to train larger models, nor if we have enough data to support them. We do not know how much they will cost to run, and how many business models will be able to bring in enough revenue to cover that cost.

We do not know whether large language model hallucinations can be eliminated, or whether the uncanny valley of knowledge, where AIs produce convincing versions of realities that do not exist, will remain a limiting factor.

We do not know in what direction the models will grow. All we know is that the process of growth and exploration will be nourished by ever more data and more compute.

And that will require a new wave of data centers, ready to meet the challenge.

13 Jul 2023

13 Jul 2023

13 Jul 2023

13 Jul 2023

13 Jul 2023

13 Jul 2023

Go here to see the original:
Generative AI & the future of data centers: Part VII - The Data Centers - DatacenterDynamics