Cloud Hosting

Artificial intelligence (AI) has quickened its progress in 2021.

A new administration is in place in the US and the talk is about a major push forGreen Technologyand the need to stimulate next generation infrastructure including AI and 5G to generate economic recovery withDavid Knight forecasting that 5G has the potential - thepotential- to drive GDP growth of 40% or more by 2030.TheBiden administration has statedthat it will boost spending in emerging technologies that includes AI and 5G to $300Bn over a four year period.

On the other side of the Atlantic Ocean, the EU have announced aGreen Dealand also need to consider theEuropean AI policyto develop next generation companies that will drive economic growth and employment. It may well be that theEU and US(alongside Canada and other allies) will seek ways to work together on issues such as 5G policy and infrastructure development. TheUK will be hosting COP 26and has also made noises about AI and 5G development.

The world needs to find a way to successfully end the Covid-19 pandemic and in the post pandemic world move into a phase of economic growth with job creation. An opportunity exists for a new era of highly skilled jobs with sustainable economic development built around next generation technologies.

AI and 5G: GDP and jobs growth potential plus scope to reduce GHG emissions (source for numbers PWC / Microsoft, Accenture)

The image above sets out the scope for large reductions in emissions of GHGs whilst allowing for economic growth.

GDP and jobs growth will be very high on the post pandemic agendas of governments around the world. At the same time those economies that truly proposer and grow rapidly in this decade will be those who adopt Industry 4.0 technology and in turn will lead to a shift away from the era of heavy fossil fuel consumption towards a digital world that may be powered by renewable energy and with transportation that is either heavily electric or over time, hydrogen based.

2021 will mark the continued acceleration of Digital Transformation across the economy.

Firms will be increasingly "analytics driven" (it needs to be stressed that analytics rather than data driven is the key term).Data is the fuelthat needs to be processed.Analytics provide the ability for organisations to make actionable insights.

Source for image above Lean BI

The examples of how Machine to Machine Communication at the Edge enabled by AI could work maybe demonstrated by the following image as an example:

In the image above the Machine to Machine communication allows for broadcast across the network that a person has been detected stepping onto the road so that even the car that does not have line of sight of the person is aware of their presence

It is important to note that AI alongside 5G networks will be at the heart of this transition to the world ofIndustry 4.0.

5G will play an important role as 5G networks are not only substantially faster than 4G networks, but they also enablesignificant reductions in latency in turn allowing for near real-time analytics and responses, and also enable far greater capacity for connection thereby facilitating massive machine to machine communication forIoT devices on the Edge of the network(closer to where the data is created on the device).

The image below sets out the speed advantage of 5G networks relative to 4G.

Source for image above Thales Group

However, as noted above 5G has many more advantages over 4G than speed alone as shown in the image below:

Source for image above Thales Group

The growth in Edge Computing will reduce the amount of data being sent backwards and forwards between a remote cloud server and thereby make the system more efficient.

Source for image above Thales Group

The economic benefits of 5G are set out below:

$13.2 Trillion dollars of global economic output

22.3 Million new jobs created

$2.1 Trillion dollars in GDP growth

Towards AI at the Edge (AIIoT)

To date AI has been most pervasive and effective in the areas of Social Media and Ecommerce giants whose large digital data sets give them an advantage and whereedge casesdon't matter so much in terms of their consequences. No fatalities, injuries or material damages arise from an incorrect recommendation for a video, a post, or an item of clothing, other than a bad user experience.

However, when we seek to scale AI into the real world, edge cases and interpretability matter. Issues such as causality and explainability become key in areas such as autonomous vehicles and robots and also in healthcare.

Equally data privacy and security also really matter. On the one hand as noted above, data is the fuel for Machine Learning models. However, on the other hand in areas such as healthcare much of that data is often siloed and decentralised plus also protected by strict privacy rules in the likes of the US (HIPAA) and Europe (GDPR). It is also an issue in areas such as Finance and Insurance where data privacy and regulation are of significant importance to the operations of financial services firms.

This is an area whereFederated LearningwithDifferential Privacycould play a big role in scaling Machine Learning across areas such as healthcare and financial services.

Source for image above NVIDIA What is Federated Learning?

It is also an area where the US and Europe could work together to enable collaborative learning and help scale Machine Learning that also provides for Data Security and Privacy for end users (patients). The Healthcare sector around the world is at breakpoint due to the strains of the Covid-19 pandemic and augmenting our healthcare workers with AI to reduce the strain upon them whilst ensuring that patient data security is maintained will be key to transforming our Healthcare systems to reduce the strain on them and deliver better outcomes for the patient.

Source for Image above TensorFlow Federated

For more on Federated Learning see:Federated Learning an Introduction.

In relation to AI, we will need to move away from the giant models and techniques that were predominant in the last decade towards neural compression (pruning)that in turn will enable models to operate more efficiently on the Edge and help preserve battery life of devices and also reduce carbon footprint with reduced energy consumption.

Furthermore, we won't only requireDeep Learning models that may inference on the Edge, but also models thatmay continue to learn on the Edge, on the fly, from smaller data sets and respond dynamically to their environments. This will be key to enabling effective autonomous systems such as autonomous vehicles (cars, drones) and also robots.

Solving for these challenges will be key to enabling AI to scale beyond Social Media and Ecommerce across the sectors of the economy.

It is no surprise that the most powerful AI companies today and last few years tend to be from the Ecommerce and social media sector.

Furthermore, the images below from Valuewalk show how ByteDance (owner of TikTok) is the world's most valuable Unicorn and an AI company.

Source for image above Valuewalk, Tipalti, The Most Valuable Unicorn in the World 2020

Venture Capitalist and Angel Investors should also try to understand that in order to scale AI startup ventures access to usable data and meeting the requirements of their customer in terms of usability (which may include some or all of transparency, causality, explainability, model size, ethics) are key for many sectors.

The number of connected devices and volume of data is forecast to grow dramatically as Digital Technology continues to expand its reach for example the image below shows a forecast from Statista for 75 Billion internet connected devices by 2025, an average of over 9 per person on the planet!

Data will grow but an increasing amount of data will be decentralised data dispersed around the Edge.

Source for image above IDC

In factIDC forecast that" Theglobal dataspherewill grow from 45 zettabytes in 2019 to 175 by2025. Nearly 30% of the world's data will need real-time processing. ... Many of these interactions are because of the billions of IoT devices connected across the globe, which are expected to create over 90 ZB of data in2025."

Illustration of the AI IoT across the Edge

Source for infographic images below:Iman Ghosh VisualCapitalist.com

In the past decade key Machine Learning tools such as XG Boost, Light Gradient Boosting Machines and Cat Boost emerged (approximately 2015 to 2017) and these tools will continue to be popular with Data Scientists for powerful insights with structured data using supervised learning. No doubt we will see continued enhancements in Machine Learning tools over the next few years.

In relation to areas such as Natural Language Processing (NLP), Computer Vision and Drug Discovery efforts Deep Learning will continue to be the effective tool. However, it is submitted that increasingly the techniques will move towards the following:

Transformers(including inComputer Vision);

Neuro Symbolic AI(hybrid AI that combines Deep Learning with symbolic Logic);

Neuroevoutionary(hybrid approaches that combinedDeep Learning with Evolutionary algorithm approaches);

Some or all of the above combined withDeep Reinforcement Learning.

This will lead to an era ofBroad AIas AI starts to move beyond narrow AI (performing just one task) and starts working with multitasking but not at the level where AI can match the human brain (AGI).

My own work is focused on the above hybrid approaches for Broad AI as we seek to find ways to scale AI across the economy beyond Social Media and Ecommerce the above will be key to enabling true Digital Transformation with AI across traditional sectors of the economy and enabling our moving into the era of Industry 4.0.

Source for Image above David Cox, IBM Watson

MIT IBM Watson Lab define Broad AIand the types of AI as follows:

"Narrow AIis the ability to perform specific tasks at a super-human rate within various categories, from chess, Jeopardy!, and Go, to voice assistance, debate, language translation, and image classification."

"Broad AIis next. Were just entering this frontier, but when its fully realized, it will feature AI systems that use and integrate multimodal data streams, learn more efficiently and flexibly, and traverse multiple tasks and domains. Broad AI will have powerful implications for business and society."

"Finally,General AIis essentially what science fiction has long imagined: AI systems capable of complex reasoning and full autonomy. Some scientists estimate that General AI could be possible sometime around 2050 which is really little more than guesswork. Others say it will never be possible. For now, were focused on leading the next generation of Broad AI technologies for the betterment of business and society."

I would addArtificial Super Intelligence(or Super AI) to the list above as this is a type of AI that often gains much attention in Hollywood movies and television series.

In Summary

Whether one views 2021 as the first year of a decade or not, 2021 will mark a year for reset across the economy and hopefully one whereby we start to move beyond the Covid pandemic to a post pandemic world.

California will remain as a leading area for AI development with the presence of Stanford, UC Berkley, Caltech, UCLA, and University of San Diego. However, other centres for AI will continue to grow around the US and the world, for example Boston, Austin, Toronto, London, Edinburgh, Oxford, Cambridge, Tel Aviv, Dubai, Abu Dhabi, Singapore, Berlin, Paris, Barcelona, Madrid, Lisbon, Sao Paulo, Tallinn, Bucharest, Kyiv / Kharkiv, Moscow and of course across China (many other examples of cities could be cited too). AI will become a pervasive technology that is increasingly in the devices (including within our mobile phones) that we interact with everyday and not just when we enter our social media accounts or go online to shop.

It will also mark a reset for AI to be increasingly on the Edge and across the "real-world" sectors of the economy with the emergence of Broad AI to take over from Narrow AI as we move across the decade.

Smaller models will be more desirable / more useful

GPT-3is an exciting development in AI and shows the potential for Transformer models, however, in the future small will be beautiful and crucial. The human brain does not require the amount of server capacity of GPT-3 and uses far less energy. For AI to scale across the edge we'll need powerful models that are energy efficient and optimised to work on small devices. For exampleMao et al. set out LadaBERT: lightweight adaptation of the BERT ( a large Transformer language model) through hybrid model compression.

The authors note "...a major blocking issue of applying BERT to online services is that it is memory-intensive and leads to unsatisfactory latency of user requests, raising the necessity of model compression. Existing solutions leverage the knowledge distillation framework to learn a smaller model that imitates the behaviours of BERT."

"However, the training procedure of knowledge distillation is expensive itself as it requires sufficient training data to imitate the teacher model."

"In this paper, we address this issue by proposing a hybrid solution named LadaBERT (Lightweight adaptation of BERT through hybrid model compression), which combines the advantages of different model compression methods, including weight pruning, matrix factorization and knowledge distillation. LadaBERT achieves state-of-the-art accuracy on various public datasets while the training overheads can be reduced by an order of magnitude."

Reducing training overheads and avoiding unsatisfactory latency of user requests will also be a key objective of Deep Learning development and evolution over the course of 2021 and beyond.

My Vision of Connectionism: Connecting one human to another (we're all human beings), connecting AI with AI, and AI with humans all at the level of the mind.

When I adopted the @DeepLearn007 handle on Twitter many years ago, I was inspired by the notion ofconnectionism, and the image that I selected for the account illustrates how 2 human beings could connect at the level of the brain and how the exchange of information, in effect ideas, drives innovation and the development of humanity. In the virtual world much of that occurs at the level of data and the analytical insights that we gain from that data through application of AI (Machine Learning and Deep Learning) to generate responses.

I remain a connectionist, albeit an open minded one. I believe that Deep Neural Networks will remain very important and the cornerstone of AI development but just like Deep Reinforcement Learning combined Reinforcement Learning with Deep Learning to very powerful effect with the likes ofAlphaGo,AlphaZero, andMuZeroresulting, so too developing hybrid AI that combines Deep Learning with Symbolic and Evolutionary approaches will lead to exciting new product developments and enable Deep Learning to scale beyond Social Media and Ecommerce sectors where the likes of medics and financial services staff wantcausal inferenceandexplainabilityfor trust in the AI decision making. For example,Microsoft Researchstate that "understanding causality is widely seen as a key deficiency of current AI methods, and a necessary precursor for building more human-like machine intelligence."

Furthermore, in order for autonomous vehicles to truly take off we'll need the model explainability for situations where things have gone wrong in order to understand what happened and how we may reduce the probability of the same outcome in the future.

The next generation of AI will be in the direction towards the era Broad AI and the adventure will be here in 2021 as we move towards the Edge, towards a better world as we move beyond the scars and challenges of 2020. The journey may require scaled up 5G networks around the world to really transform the broader economy and that may only really start to happen at the end of the year and beyond but the direction of the pathway is clear.

The exciting potential for healthcare, smart industry, smart cities, smart living, education, and every other sector of the economy will mean that a new of businesses will emerge that we cannot even imagine today.

Perhaps a good point to conclude is with the forecast from Ovum and Intel for the impact of 5G for the media sector (of courseAI will play a big role alongside 5Gin developing new hyper personalised services and products andhave a symbiotic relationship together).

Source for the image above:Intel Study Finds 5G will Drive $1.3 Trillion in New Revenues in Media and Entertainment Industry by 2028

See more here:
Towards Broad Artificial Intelligence (AI) & The Edge in 2021 - BBN Times