Cloud Hosting

It is a long-standing joke among the industry experts that while AI may crunch massive amounts of data, write codes that run huge machinery or even author a book, it would still fail tasks that a three-year-old human child can accomplish. This is also why AI systems still have a long path to trace to be truly called intelligent.

Hubert Dreyfus, a well-known philosopher, was one of the staunchest critics of overestimating computer/AIs capabilities. He wrote three books Alchemy and AI, What Computers Cant Do, and Mind over Machine, where he critically assessed the progress of AI. One of his arguments was that humans learn from implied knowledge, and such capability cannot be incorporated into a machine.

Having said that, there have been tremendous advancements in the human endeavour to move away from Narrow AI towards the Coveted General AI. Many new models like GPT 3, DALL.E, LaMDA, Switch Transformer, etc., are extremely powerful, swallowing billions and even trillions of parameters to multi-task. But, the fact is, we are still far from reaching the goal.

The powerful language models and the newer zero-shot text-to-image generation models are all marching really fast towards the intended goal of performing tasks for which they were not trained. Each one is outwitting the previous one for its applications and uses.

DeepMind, one of the most well known AI research institutes (owned by Alphabet), has centred its ultimate goal at achieving AGI. Interestingly, this year, the lab published a paper titled Reward Is Enough, where the authors suggested that techniques like reward maximisation can help machines develop behaviour that exhibits abilities associated with intelligence. They further concluded that reward maximisation and reinforcement learning, in extension, can help achieve artificial general intelligence.

Lets take a closer look at GPT-3 created by DeepMinds closest competitor, OpenAI, which created a major buzz in the scientific community. It was widely considered a massive breakthrough when achieving General AI.

GPT-3 leverages NLP to imitate human conversations. It has been trained on one of the biggest datasets with 175 billion parameters, making it so powerful that it can complete a paragraph based on a few input words. Moreover, unlike typical narrow AI models, GPT-3 can perform tasks beyond generating human-like text, like translating between languages, reading comprehension tasks without additional input, and even writing codes!

On the flipside, GPT-3 does not have common sense. It almost blindly learns from the material that it has been trained on, scrouged from several pages on the internet. Due to its lack of common sense, GPT-3 can pick biassed, racist and sexist ideas from the internet and rehash them [Note: Open AI is working to mitigate bias and toxicity in many ways, but it is not eliminated yet]. Additionally, the transformer lacks causal reasoning and is unable to generalise correctly beyond the training set, making it far from General AI.

But, as we know, this quest is going to go on. This journey is fraught with billions of computational inputs, thousands of tonnes of energy power consumption and billions of dollars of expense to build and train these AGI models. Most ignore or fail to comprehend its consequences on the environment.

Climate change has become a fundamental crisis of our time, and AI plays a dual role. It can help reduce the effects of the climate crisis and control it through solutions like smart grid designing or developing low emission infrastructure. But, on the other hand, it can lead to the undoing of all sustainability efforts, making the extent of its carbon emission hard to ignore.

Estimates suggest that training a single AI generates close to 300 tonnes of carbon dioxide, equivalent to five times the lifetime emissions of an average car. AI requires increased computational powers, and data centres that store large amounts of AI data consume high energy levels. The high-powered GPUs required to train advanced AI systems need to run for days at a time, utilising tonnes of energy and generating carbon emissions. This is increasing the ethical price of running an AI model.

GPT-3 is one of the largest language models. The Neural Architecture Search process of training general transformer models requires more than 270,000 hours of training and 3000 times the energy, so much so that the training has to be split over dozens of chips and broken down over months. If the input is so massive, the output is worse. A 2019 study found that training an AI language-processing system generates anywhere between 1,400 to 78,000 pounds of emission. This is equivalent to 125 round trip flights between New York and Beijing.

Sure, it is better in performance, but at what cost? Carbontracker suggested training GPT-3 just once requires the same amount of power used by 126 homes in Denmark every year. It is also the same as driving a car to the moon and back.

GPT-3 isnt the only large language model in the market today. Microsoft, Google, and Facebook are working on and have released papers on more complex models involving images and powerful searches that go far and beyond language, to create multi-tasking and multi-modal models.

OpenAI identified how, since 2012, the amount of computing power in training large models has been increasing exponentially with a 3.4 month doubling time. If this is true, one can only imagine the energy consumption and carbon emissions until we reach AGI.

AI could become one of the most significant contributors to climate change if this trend continues.

This entails employing efficient techniques for data processing or search and training models on specialised hardware, like AI accelerators, that are more efficient per watt than general-purpose chips. Google published the paper on Switch Transformers that use more efficient sparse neural nets, facilitating the creation of larger models without increasing computational costs. Researchers such as Lasse Wolff Anthony, who has worked on AI power usage, have suggested that large companies train their models in greener countries such as Estonia or Sweden. Given the availability of greener energy supplies, a models carbon footprint can be reduced by more than 60 times.

The solutions arent many as of now, but attempts are being made to devise them. It is important that we have conversations about it. While innovation is the basis on which a society moves forward, we must also be conscious of the cost such innovation brings. The need of the hour is to strike a balance between the two.

This article is written by a member of the AIM Leaders Council. AIM Leaders Council is an invitation-only forum of senior executives in the Data Science and Analytics industry. To check if you are eligible for a membership, please fill the formhere.

See the rest here:
Council Post: Reflecting On The Cost Of Our Dream To Build General AI - Analytics India Magazine

Nervous State

A series about coping with our most anxious year yet.

Photo-Illustration: by The Cut; Photos: Getty Images

While it might feel like everyone you know is in therapy, more than a third of Americans live in an area with a shortage of mental-health professionals; plus, the need for therapy has only increased during the pandemic. Aside from that, so much goes into actually finding a therapist that we often breeze over including the time needed to find one and the money that it takes to continually attend, especially if its an out-of-network provider its easy to see that not everyone who needs therapy actually has access to it. So what can you do when systemic barriers stand in the way of therapy, especially when it comes to managing anxiety?

Instead of fighting our anxiety, Dr. Wendy Suzuki, a psychology professor at the Center for Neural Science at New York University, believes we should better understand it. Anxiety, at its core, is protective. It is actually critical for our survival, Dr. Suzuki tells me. Her recent book, Good Anxiety: Harnessing the Power of the Most Misunderstood Emotion, is devoted to reconsidering the role of anxiety in our lives and what we can do to live with it. Here are the five tools Dr. Suzuki recommends to cope.

In moments of distress, when a panic attack may be setting in, slowing down your breathing is critical. The science behind deep breathing is that it is activating your natural de-stressing part of your nervous system, says Dr. Suzuki. Its called the parasympathetic nervous system, or the rest and digest system. This is the counterpart to the more well-known fight or flight part of our nervous system. The best way to activate it? Deep breathing. While there are many different breathing patterns, Dr. Suzuki suggests a box-breathing method because of how simple it is four counts in, four counts hold, four counts out, and four counts hold. You can do breathing exercises anywhere even if youre in the middle of an anxiety-inducing situation without anyone knowing.

Youre probably tired of hearing this, but exercise really does work. (A lot of other people apparently need to hear it, too: Dr. Suzukis 2018 TED Talk about how moving your body can boost your mood has over 31 million views and counting.) Its like giving yourself a wonderful neurochemical bubble bath, she says. Even if its just a walk around your home or outside, you release chemicals that help your mood. These could be dopamine, serotonin, noradrenaline, and endorphins, which help make you feel happier, less stressed, and less anxious. Its not a magical thing. This is neuroscience that you are taking advantage of here, says Dr. Suzuki. Especially when you feel anxious and you can remove yourself from the situation, its helpful to step out of your mind and focus on your body.

Something that therapy offers is the chance to talk and have someone really listen. Even without it, we can teach ourselves and others that same skill. Practicing mindful listening (and having someone mindfully listen to you) is the easiest recommendation that Dr. Suzuki can give sometimes just being able to vent to a good friend that will really listen to what is going on in your life can be so relieving for you. Starting this practice takes little more thanreaching out to a friend and asking to chat. Explain that you will listen to what theyre feeling, without interrupting or offering advice, and that you think they can do the same for you.

It is an act of caring, to listen deeply and without judgment. You are there as a sounding board and to be supportive. You dont have to solve their problem. That is not the point, says Dr. Suzuki. The point is to listen and to take in their situation so they feel heard.

One way to better manage your anxiety is to understand what provokes it. Most of the anxiety-inducing situations we deal with are not out-of-the-blue surprises, but situations that have been stressors in our lives for a long time. Thats great because we can prepare for them and identify them, says Dr. Suzuki. In doing that, you can better prepare yourself to encounter them and have tools already in place. If you tend to get anxious whenever you have a job interview, it could be helpful to look back and try to find the moment when your anxiety began. Was it before the interview, during, or after? By understanding when and why your anxiety started, you can go into the experience next time with a plan to mitigate the stress.

You can apply those kinds of approaches to every single anxiety-provoking situation in your life. You might think, Oh my God, that takes a lot of preparation. Yes, it does. But once you do that, you start to learn those approaches, and they become more automatic for you, says Dr. Suzuki. And they are powerful.

Dr. Suzukis book is called Good Anxiety for a reason she wants us to understand that anxiety isnt always bad. It can inform us of our values and offer protection, but it takes self-analysis and time to do so. A mind-set is just a belief. So how do you shift your mind-set? You have to believe in what youre shifting it to, says Dr. Suzuki.

Think of it this way: You might feel overwhelmed trying to make sense of the new Omicron variant, but your anxiety about it can also help keep you safe. Your anxiety doesnt have to be your enemy in this situation. Dr. Suzuki suggests using it to your benefit by researching the best way to protect yourself. Shifting your mind-set includes a wonderful opportunity to mitigate the fear of Omicron by educating yourself as much as you can, with sources that you trust, for what your best strategies to travel safely are, says Dr. Suzuki.

While tools for dealing with anxiety on your own can be helpful, there are times when you might need outside help. Hotlines like the Samaritan Helpline, the National Alliance on Mental Illness Helpline, or the 24/7 Crisis Texting Line can help you during a crisis. For less urgent situations, there are warmlines, service lines that offer emotional support and are staffed by volunteers who have experienced mental-health struggles on their own. Other free resources come in the form of apps, such as Mindshift for meditation and mindfulness exercises, iBreathe for breathing exercises, and Moods for tracking how you feel each day.

Get the Cut newsletter delivered daily

Read the original here:
5 Tools to Cope With Anxiety When Therapy Isnt an Option - The Cut