Cloud Hosting

Analysis and prediction are core to todays IT as organisations embark on digital transformation, with use cases that range from speech recognition and pattern analysis in science, to use in fraud detection and security, to AIOps in IT and storage.

As artificial intelligence and predictive methods machine learning, deep learning, neural processing, and so on become more prevalent, ways to streamline these operations have developed.

Key among these is the emergence of new ways of dealing with large numbers, and bfloat16 originally developed by Google is the most prominent among these.

In this article, we look at what bfloat16 is, the impact it will have on memory and back-end storage, and which hardware makers support it.

Bfloat16 short for Brain Float 16 is a way of representing floating point numbers in computing operations. Floating point numbers are a way that computers can handle very large numbers (think millions, billions, trillions, and so on) or very small ones (think lots of zeros after decimal points) while using the same schema.

In floating point schemes, there are a set number of binary bits. A bit indicates whether the number is positive or negative, some of the bits indicate the number itself, and the floating point element the exponent is a number of bits that say where the decimal point should be.

Bfloat16, as the name suggests, uses a 16-bit format to do all this. In doing so, it cuts in half the weight in bits of the most-prevalent existing standard, IEEE 754, which is 32-bit.

But bfloat16 uses an exponent that is the same size as that in IEEE 754, which allows it to represent the same range in size of numbers, but with less precision.

Bfloat16 was developed by Google the B represents the companys Brain project specifically for its tensor processing units (TPUs) used for machine learning. The key thing here is that for machine learning operations you dont need the levels of precision in terms of binary powers that other calculations might require. But you do want speed of operation, and thats what bfloat16 is aimed at.

The key benefits of bfloat16 are that it reduces storage requirements during processing and speeds up individual calculations during machine learning operations.

Bfloat16 takes half the memory of equivalent operations that use IEEE 754 32-bit numbers, meaning that more can be held in memory and take less time to swap in and out of memory. That means larger models and datasets can be used. Also, bfloat16 takes less time to load into memory from bulk storage.

Hardware support for bfloat16 is something that gets built into processors and processing units, so it will be tailored to the standard.

Back-end storage volumes are likely to be positively impacted. In other words, youll need less storage if you do a lot of machine learning operations with bfloat16. But its more likely, at least for a while, that IEEE 754 will predominate, and bfloat16 converts from that existing standard.

Bfloat16 was first deployed on Googles hardware TPUs, supported by Intel, which can be used via the providers cloud services (Cloud TPU) or bought as a product for customer on-premise use.

At the time of writing, it is also supported by Intels third-generation Xeon Scalable CPUs, IBMs Power10 and ARMs Neoverse processors.

Bfloat16 is also supported in a number of NPUs neural processing units, of which the TPU is one including in ARMs Trillium, Centaur, Flex, Habana, Intels Nervana and Wave.

See the original post:
Bfloat16: What it is and how it impacts storage - ComputerWeekly.com