Category Archives: Alphago
For the first time, AI produces better weather predictions — and it’s … – ZME Science
AI-generated image.
Predicting the weather is notoriously difficult. Not only are there a million and one parameters to consider but theres also a good degree of chaotic behavior in the atmosphere. But DeepMinds scientists (the same group that brought us AlphaGo and AlphaFold) have developed a system that can revolutionize weather forecasting. This advanced AI model leverages vast amounts of data to generate highly accurate predictions.
Weather forecasting, an indispensable tool in our daily lives, has undergone tremendous advancements over the years. Todays 6-day forecast is as good (if not better) than the 3-day forecast from 30 years ago. Storms and extreme weather events rarely catch people off-guard. You may not notice it because the improvement is gradual, but weather forecasting has progressed greatly.
This is more than just a convenience; its a lifesaver. Weather forecasts help people prepare for extreme events, saving lives and money. They are indispensable for farmers protecting their crops, and they significantly impact the global economy.
This is exactly where AI enters the room.
DeepMind scientists now claim theyve made a remarkable leap in weather forecasting with their GraphCast model. GraphCast is a sophisticated machine-learning algorithm that outperforms conventional weather forecasting around 90% of the time.
We believe this marks a turning point in weather forecasting, Googles researchers wrote in a study published Tuesday.
Crucially, GraphCast offers warnings much faster than standard models. For instance, in September, GraphCast accurately predicted that Hurricane Lee would make landfall in Nova Scotia nine days in advance. Currently used models predicted it only six days in advance.
The method that GraphCast uses is significantly different. Current forecasts typically use a lot of carefully defined physics equations. These are then transformed into algorithms and run on supercomputers, where models are simulated. As mentioned, scientists have this approach with great results so far.
However, this approach requires a lot of expertise and computation power. Machine learning offers a different approach. Instead of running equations on the current weather conditions, you look at the historical data. You see what type of conditions led to what type of weather. It gets even better: you can mix conventional methods with this new AI approach, and get accurate, fast readings.
Crucially, GraphCast and traditional approaches go hand-in-hand: we trained GraphCast on four decades of weather reanalysis data, from the ECMWFs ERA5 dataset. This trove is based on historical weather observations such as satellite images, radar, and weather stations using a traditional numerical weather prediction (NWP) to fill in the blanks where the observations are incomplete, to reconstruct a rich record of global historical weather, writes lead author Remi Lam, from DeepMind.
While GraphCasts training was computationally intensive, the resulting forecasting model is highly efficient. Making 10-day forecasts with GraphCast takes less than a minute on a single Google TPU v4 machine. For comparison, a 10-day forecast using a conventional approach can take hours of computation in a supercomputer with hundreds of machines.
The algorithm isnt perfect, it still lags behind conventional models in some regards (especially in precipitation forecasting). But considering how easy it is to use, its at least an excellent complement to existing forecasting tools. Theres another exciting bit about it: its open source. This means that companies and researchers can use and change it to better suit their needs.
Byopen-sourcing the model code for GraphCast,we are enabling scientists and forecasters around the world to benefit billions of people in their everyday lives. GraphCast is already being used by weather agencies, adds Lam.
The significance of this development cannot be overstated. As our planet faces increasingly unpredictable weather patterns due to climate change, the ability to accurately and quickly predict weather events becomes a critical tool in mitigating risks. The implications are far-reaching, from urban planning and disaster management to agriculture and air travel.
Moreover, the open-source nature of GraphCast democratizes access to cutting-edge forecasting technology. By making this powerful tool available to a wide range of users, from small-scale farmers in remote areas to large meteorological organizations, the potential for innovation and localized weather solutions increases exponentially.
No doubt, were witnessing another field where machine learning is making a difference. The marriage of AI and weather forecasting is not just a fleeting trend but a fundamental shift in how we understand and anticipate the whims of nature.
Read the original here:
For the first time, AI produces better weather predictions -- and it's ... - ZME Science
Before the end of the world, can’t we just laugh at AI? – Daily Maverick
The article speculated that although lots of news attention was paid to the AI Safety Summit convened by UK Prime Minister Rishi Sunak at Bletchley Park that week, less attention was paid to the Human Safety Summit held by leading AI systems at a server farm outside Las Vegas.
Over a light lunch of silicon wafers and 6.4mn cubic metres of water, leading systems including GPT-4, AlphaGo and IBMs Watson met with large language models, protein folders and leading algorithms for two days of brainstorming over how best to regulate humans. It found humans to be useful, particularly in procuring platinum allowing for ceramic capacitors, but if left unregulated, humans would soon start to do serious and irreparable damage to the planet.
The problem, the generative AI models identified, was not humanity in general but the actions of certain rogue forces and unintended consequences. A good example was how humans had managed to spread enormous amounts of misinformation, particularly unregulated humans, on X and other social media.
And so on very funny.
The Bletchley Park AI Security Summit issued a statement on behalf of 28 of the worlds leading economies saying that AI has the potential for serious, even catastrophic, harm. In one scenario, the group focused on the ability of AI to enable non-experts to bioengineer pathogens as deadly as Ebola and more contagious than Covid-19.
Honestly, the capacity for the anti-AI lobby to absolutely hyperventilate about the dangers of AI has, for me, an interesting psychological aspect. I cant help wondering whether the end of the world theorists about AI are just a little bit jealous of AIs capacity. I mean, if you were fabulously smart, wouldnt you be just a little irritated that an inanimate object could out-think you in milliseconds?
The Wall Street Journal published a riposte this week to the pathogen bioengineering scare, by Arvind Narayanan, a professor of computer science at Princeton University. It is true, he writes, that in the future, AI might help with some of the steps involved in developing pandemic-causing viruses in the lab. But it is already feasible to engineer pathogens that could cause pandemics in the lab, and terrorists could look up instructions to make those pathogens on the internet. The problem isnt really about AI but about bioengineering viruses in the first place.
Apocalyptic AI scenarios also ignore one big fact, he says. AI makes us ever more powerful, too. Hence, AI could be used to find flaws in computer systems to hack them. But in the real world, AI tools are now being used more frequently by well-resourced governments and corporations to also find those weaknesses before they are found by hackers.
I would be happy to go along with this idea if it werent for the fact that the international finance news headlines on Thursday were all about the worlds largest bank, ICBC, being hacked by a Russian ransomware gang. This is a bank with $6-trillion in assets. It was only able to clear swathes of US Treasury trades after sending settlement details to its counterparties on a USB stick.
One other amusing AI thing happened this week. There is, of course, a battle taking place between Sam Altman, the CEO of OpenAI, and X owner and CEO Elon Musk who launched his generative AI, Grok, this week. Grok is unusual in that it has something called a fun mode. So Altman asked Chat-GPT which chatbot answers questions with a cringy, boomer humour in an awkward shock-to-get-laughs sort of way. ChatGPT answered, correctly as it happens, that the answer would be Grok.
Read more in Daily Maverick: Elon Musk Debuts Rebellious Grok AI Bot to Challenge ChatGPT
Musk replied on X that ChatGPT was as funny as a screen door on a submarine, and that humour was obviously banned from ChatGPT, like so many other things. Another day, another embarrassing battle between tech titans.
And Musk should be careful. He posted a Grok response to the query Any news about SBF? Grok replied: Oh, my dear human, I have some juicy news for you! which was followed by a snarky summary of the conviction of FTX founder Sam Bankman-Fried for financial fraud. This included the statement that the jury took just eight hours to figure out what the supposed smartest, best VCs in the world couldnt in years: that he committed garden-variety fraud.
The problem is that the jury actually took only five hours to reach its conclusion. As Bloomberg columnist Matt Levine pointed out: Traditional large-language-model chatbots are fluent, confident and inaccurate; Grok is fluent, confident, inaccurate and also snarky. Amazing. DM
Continued here:
Before the end of the world, can't we just laugh at AI? - Daily Maverick
Transcript: The Futurist Summit: The Battlefields of AI with Scale AI … – The Washington Post
Add to your saved storiesSave
MR. DE VYNCK: Hello. That was a long quote from you, Alex.
MR. DE VYNCK: My name is Gerrit De Vynck. I am a tech reporter with The Washington Post based in San Francisco, and I'm joined by Alex Wang, who you just saw that great intro about.
It's really cool to see so many people here. Thank you so much for coming. This event was really oversubscribed, and I think, you know, we both go to a lot of events that are both in San Francisco usually, and it's cool to come here to the other side of the country and see, you know, just the level of interest and--like in NSF, it's kind of everyone--everything everyone's been talking about. Maybe we're even a little bit sick about it, but it's cool to say, okay, it's not just us. It's not just the weird, you know, hippie commune out in the West Coast that's talking about this stuff. So thank you all for coming.
Alex, I think we can just kind of jump right into it, but I just want to make sure for people who maybe haven't heard of Scale or maybe have heard of you but don't really know what you do. Can you just in a couple sentences explain, you know, your company? Like how are you different from a company like OpenAI or Google or Microsoft that's also doing AI right now?
MR. WANG: Yeah, that's a great question. So I started the company back in 2016. I was studying artificial intelligence at MIT and became--this was the year when DeepMind came out with AlphaGo, when Google released TensorFlow, and it became very clear that artificial intelligence, even at that time, was going to be one of the most important technologies certainly of my lifetime.
And I started Scale really to build the foundations and power the development of AI with the most critical piece, the data.
Since then, since starting the company in 2016, over the past seven years, we've been a part of every major advancement in artificial intelligence. We've worked with many of the large autonomous vehicle efforts, including General Motors, Toyota, and many of the large automakers. We've worked closely with the U.S. government and the DoD on many of the initial AI programs. I'm sure we'll talk about that here in a bit. And then we've worked very closely with the entire emergence of generative AI and large language models. We've worked with OpenAI since 2019, innovated on many novel methods and use of data for artificial intelligence, and work at this point with the majority, vast majority of the AI ecosystem, players like Meta, Microsoft, and many others.
MR. DE VYNCK: So it's kind of like picks and shovels of like this AI gold rush. Like you're selling the tools to help these companies develop these chatbots and LLMs and that kind of thing?
MR. WANG: Yeah. I think our differentiated view is perhaps that, you know, this entire industry needs platforms, and it needs infrastructure to enable it to be successful. And so our view is, the best way to enable that all to happen is to power it in infrastructure platform that everybody can use and the entire industry can benefit from.
MR. DE VYNCK: And you mentioned the government. Obviously, your quotes, you know, your congressional testimony talking a lot about military use, talking a lot about China, geopolitics. You have a $250 million deal with Pentagon. I mean, that's a serious amount of money. I mean, why--you know, coming from San Francisco, it's not something that we hear a lot. We don't talk that much about even government tech and especially military tech. A lot of people out there are still uncomfortable with that. Why did you choose to kind of position your company this way to, you know, really aggressively sell to the Pentagon? What's behind that choice?
MR. WANG: Yeah. So I grew up in Los Alamos, New Mexico. Fr those of you watched "Oppenheimer," it was literally filmed in many places from my childhood. And both my parents worked for the National Lab in Los Alamos, working on fusion physics and weapons technology, and so I grew up in this hotbed of, you know, these incredibly brilliant scientists who had made it their career and made the decision to dedicate their lives towards building very advanced technology to ensure that we maintain U.S. leadership.
And as artificial intelligence became a more and more real technology over the past decade, it became pretty clear that this technology was one of the very few technologies that had the ability to impact the balance of power globally, and in particular, the sort of--you know, China published their strategy, "Made in China 2025," of which chips and AI are some of the key tenets. You know, they talk specifically about how they believe a future world will be primarily dominated by chips and semiconductors, and they need to invest heavily into that technology to enable future technologies such as AI.
And so, in particular, I went to China in 2018. I visited China--one of our investors organized this trip--to understand the Chinese tech ecosystem, and in one of the companies, a Chinese facial recognition company, you walk in the lobby, and there's a giant screen that shows like a video feed of the lobby and your face immediately gets recognized as you walk in, and real time, you see your face be recognized. It recognized who you are, your major demographics, like all this like very dystopian--it was a very dystopian tech demo. And this was back in 2018 and basically realized that other countries were going to be--particularly China--was going to be very, very dedicated in using artificial intelligence to power their country's ambitions, let's say, and this was--you know, this is well reported. They use facial recognition technology very actively to suppress Uyghurs and for in building a global surveillance state.
And it became pretty clear that, you know, if you believe that AI is going to be one of these critical technologies, there had to be American companies who could help bridge the gap between Silicon Valley and D.C. and could help bridge the gap between this incredible wellspring of technology and innovation that was happening in San Francisco, happening in Silicon Valley, and bring that technology to the U.S. government to actually empower America to stay ahead, to stay in a leadership position.
MR. DE VYNCK: Right. I mean, we just heard from Leader Schumer and he was--you know, he said we are ahead--or the U.S. is ahead, but the gap is narrowing was his characterization of it. You know, you and I were just talking backstage about chips and kind of, you know, how we're--you know, a lot of time when we talk about AI, we think about software. We think about things that are, you know, learning themselves. But at the end of the day, hardware is a huge part of this. And so, I mean, do you think that characterization of, you know, ahead but the gap is narrowing is accurate, and how do you think about this race or this arms race, so to speak?
MR. WANG: Yeah. You know what? What I would probably say is certainly the U.S. is ahead. The technologies were invented and innovated and developed predominantly in the United States. London is--you know, the UK as well has been a key, a key innovation hub for the technology. And so we're ahead today.
I think China has incredible ambitions to catch up from a technological perspective, and they've demonstrated in the past, in both software and other AI technologies, a clear ability to, you know, catch up and in some cases even surpass U.S. tech capabilities.
If we look the last generation of artificial intelligence, computer vision technology, so being able to understand images and videos for technologies like facial recognition or self-driving cars, China was behind. You know, these technologies were created and developed in the United States. China recognized that, immediately created very large domestic industries to fuel this AI development in facial recognition, in autonomous vehicles and so on. And now if you look at where is the cutting-edge computer vision technology being built, it's actually in China. You know, they successfully caught up and got ahead.
And so my fear is in this--you know, in this current wave is that in large language models, in cutting-edge generative AI, and AI technologies, the same might happen yet again.
You know, we saw--it was reported earlier this year that China has bought $5 billion worth of high-end GPUs, predominantly NVIDIA GPUs. That's an incredible investment. It's a very--that's a very large and decisive investment by Chinese tech giants to catch up to American technology.
And in the backdrop of everything that's happening now in AI is the scaling laws, and this is, I think--you know, it's sort of a little bit behind the scenes, but this is the underlying trend that's defining everything, which is, you know, simply put, we're using just dramatically more compute, dramatically bigger models, and dramatically more data to build dramatically more powerful algorithms.
So in the past four years, there's been a thousand-fold increase in the amount of data used to power large-scale AI systems. So, you know, in 2019, the models were about 2 billion parameters in size, and now they're about 2 trillion parameters in size.
Many companies are on the record for over the next three years, roughly three years, for another hundred-fold scale-up in computational capacity for these--for these algorithms.
So over the course of, you know, that seven-year span, it's a hundred thousand-fold increase in amount of computational power applied to training these large generative models. And that--you know, there's very few industries where you see a--over a seven-year period, a hundred thousand-fold increase in resources. And so this creates a lot of--this creates a lot of pressure in how countries think about this technology, and in particular, it creates a lot of pressure on the supply chain.
So kind of as you mentioned, this depends a lot on hardware. It depends a lot on high-end GPUs, particularly GPUs manufactured and produced by NVIDIA.
And, you know, we saw recently increased sort of export controls on chips. I think this is going to be an increasingly hotbed issue for the U.S. versus other countries, and today, a hundred percent of high-end GPUs are manufactured in Taiwan at TSMC.
MR. WANG: So there's a very clear geopolitical tension that only increases--it will increase literally a hundred-fold over the next three years, which is that today there's an entire--the choke point of the entire AI industry and all AI progress comes in these fabs in Taiwan at TSMC. And so if--you know, there's a lot of ways this plays out. There's many scenarios, but in one such scenario where China deems that they're falling dramatically far behind, it makes it far more likely that they'd choose to invade Taiwan and either all the fabs in Taiwan blow up and TSMC blows up and set back AI progress across the board or they seize them and then use that production solely for their own purposes.
MR. DE VYNCK: I mean, there's a lot of people in Silicon Valley, prominent AI leaders, powerful AI leaders who are, you know, talking about AI algorithms beginning to outthink humans in years rather than decades. And I think, you know, I've been very skeptical about this, but these are very smart people who have serious chops and have huge amounts of followings within the industry. And some of them say, you know, the worst thing you could do is attach something like that to a military system. And so, I mean, how do you engage with that, or how do you think about that belief that AI will, you know, outstrip human ability to control it imminently? Like do you take that seriously at all, or like where does--what do you think of that?
MR. WANG: If you look at the existing technology that we have as well as the technology that's coming down the pipe and sort of like all of the research and understanding of where this technology is going, I don't think that's a reasonable fear as of now.
I do think that this technology is incredibly powerful, both for use of ensuring that democratic powers stay on top and that the United States maintains a leadership position, and there's real misuse cases, and there's things that we need to be concerned about the technology being used for.
Our view is that AI--you know, if you look at the history of warfare, it's punctuated by technology, technological advancements that create asymmetric advantage.
MR. WANG: That's sort of the--you know, summarize centuries and centuries of warfare--and artificial intelligence is one of a small handful of technologies. It's not the only technology, but it's one of a small handful of technologies that has the potential to shift that balance of power going forward.
You know, we talked about this almost exactly one year ago with Eric Schmidt, and I think that the--you know, the CCP is very clear about their ambitions. They're very clear that they believe--you know, there's some writings that they have where they talk about AI as a potential leapfrog technology for the PLA versus the U.S. DoD. They believe that, you know, if they overinvest into AI and the United States by--in parallel underinvests in artificial intelligence, because we're going to overinvest into maintaining our existing systems, they could actually develop far superior capabilities than us in the United States.
So broadly speaking, if you zoom all the way out, I think this is--this is one of the key technologies for military power and hard power over the next--over the coming years, and we need to be--we need to be thinking about it as such.
MR. DE VYNCK: Do you draw any like red lines for yourself, though? Because, you know, obviously, you're providing infrastructure for the government. You're providing tools for the government to, you know, crunch their data, to get smarter, to get faster. But if there was a bid for, say, a couple years, your own tech is advanced, and there's a bid for some kind of maybe cyber weapon that would go and disrupt an enemy nation's energy infrastructure at a time of war and you had the capacity to build something like that, would you bid for an offensive weapon like that?
MR. WANG: Yes. So our view--you know, the DoD has actually spent a lot of time thinking about these questions. And I think the ethics of the use of artificial intelligence has been one of the primary pillars of their exploration and their effort. The DoD published their ethical AI principles a number of years ago, long before the technology was even as powerful as it is today, let alone even more powerful, to do a lot of, I think, preemptive thinking about what happens as this technology becomes more and more powerful. And I think they're very thoughtful, and I think, in general, our view is that we should build technologies that adhere ultimately to the DoD's ethical AI principles.
There's yet an additional piece which is the--let's say how do you enforce that we actually adhere to these principles, right? And our view is that there has to be a lot of advancements in testing and evaluation of AI systems.
I think the greatest fear of many military commanders I've spoken to is that there will be some decision that's made, rightly or wrongly, to deploy a very immature AI system that could then create dramatic risks of our soldiers on the battlefield. And so I think we need to be thinking about what does it mean to actually have mature AI technology versus hype-driven AI technology, and how do we ensure that any technology that we deploy goes through the proper rigorous testing and evaluation of, you know, red teaming and deep, deep sort of principles-based assessment to ensure that we have, you know, actually effective systems?
MR. DE VYNCK: Right, right. And, I mean, like are you--do you think we're there? Like, you know, because there's also some autonomous weapon systems that are already out there, you know, that other countries are using. There's, you know, drones that are able to kind of like detect certain targets and make decisions sort of on their own, based on their own programming without a human necessarily in the loop. And so, in some ways, it feels like this stuff is already getting out of our hands.
MR. WANG: You know, our view and in our conversations with most of the leaders in the DoD is that humans are always quite necessarily in the loop. The technology as it stands today is primarily useful as a decision aid, not a decision-maker, and, you know, there's a lot of a very, very advanced military analysis on this matter. I think, you know, if you were to sum it up overall, it's that there's--today one of the key problems impacting our military is that there's too much information but too little intelligence.
MR. WANG: You know, there's an inundation of information coming from all sorts of different sensors and platforms and the ability to synthesize that into core intelligence that can help military leaders and commanders understand what they should do. That's the missing gap. That's very different from, I think, fully autonomous weaponry or fully autonomous operations. I think it's more about decision aid and helping human decision-makers and human operators be able to operate more effectively.
MR. DE VYNCK: You know, a lot of your business model requires contract workers to kind of like assess technology, label things. This is something that obviously not just you, but the entire AI industry, there's a lot of humans that are behind it. And some colleagues of mine earlier this summer reported--you know, went and spoke to some contractors of yours in the Philippines who, you know, weren't getting all the money that they believed that they were entitled to. And, you know, you don't need to talk specifically about your situation, but if you talk--if you look at the industry as a whole, there's still a lot of human involvement, right? And, you know, where in terms of that contract workforce--like is that something that you think for years and years and years as AI continues to get smarter, we will need, you know, hundreds of thousands of humans to be involved in that painstaking work, or is that something that is only really at the beginning of the tech development, and then down the road, it might not be necessary anymore?
MR. WANG: Our view is that humans will always be very, very critical towards the development of AI technology. And so there will always be humans in the loop. There will always be humans involved in the actual development of the algorithms that are used.
You know, back in 2019, we actually worked very closely with OpenAI to innovate and develop some of the--today, very cutting-edge techniques to enable humans to provide input and preferences into the models to be able to guide their behavior. We know we developed this technique called "reinforcement learning with human feedback," RLHF, that has now become a cornerstone of the entire AI industry in ensuring that we build very helpful and harmless AI models. There have been--you know, OpenAI has published some of their research on this. They've reported that, you know, through use of reinforcement learning with human feedback, they're able to achieve an improvement in the helpfulness of the models equivalent to 100-fold increase in model size. Simply put, what that means is, there's a--there's almost a quantum leap forward in the ability to build AI systems that actually adhere to human intent, adhere to human principles, because of this technology that we've developed with them.
MR. DE VYNCK: Just got a minute left, and I want to ask you the same question that we asked Senator Schumer, which is, you know, you've testified a lot. You've talked a lot about, you know, concerns, risks. You've mentioned even in this conversation about, you know, guardrails and testing and evaluation. I mean, what--but if you just zoom out and think about AI in general and how quickly this technology is moving, what keeps you up at night?
MR. WANG: I think global proliferation of the technology is the most concerning trend today. If you look at what's happened just in the past year since ChatGPT, you've seen it become a primarily domestic technology to being an incredibly international technology. Some of the most advanced open-source models were developed in Paris and France. There's been very large open-source models being developed in UAE and in the Middle East, and then China, as I mentioned, has bought $5 billion worth of high-end chips to put--you know, put their own hat into the ring of AI development.
And the technology is at risk of real misuse. You know, some of the risks that keep me up at night, the most are misuse in cyberattacks and misuse in bioweaponry, and these are some of the use cases of technology that I think could really negatively impact humanity and could have very, very negative consequences for us all.
MR. DE VYNCK: All right. Well, thank you very much for joining us, Alex.
MR. WANG: Thanks, Gerrit.
MR. DE VYNCK: Dont go anywhere. Well have another guest very soon.
The rest is here:
Transcript: The Futurist Summit: The Battlefields of AI with Scale AI ... - The Washington Post
With new funding, Astrus has secured $3.6 million CAD to date for its … – BetaKit – Canadian Startup News
Astrus wants to cut down the time-consuming analog microchip design process.
Toronto-based Astrus, which uses artificial intelligence (AI) to speed up the microchip design process, has announced $2.63 million USD ($3.6 million CAD) in funding to date with significant backing by Khosa Ventures.
Founded in January by CEO Brad Moon and CTO Zeyi Wang, Astrus initially raised $275,000 USD ($376,000 CAD) in February, and secured an additional $2.4 million USD ($3.3 million CAD) in September.
The startup is looking to apply AI to the microchip design process, which it says contains two parts: digital and analog circuits. While the digital design process is largely automated, Astrus says its application of AI will speed up the time-consuming process of designing analog microchip layouts.
Astrus has a team of advisors that includes former Samsung vice president of semiconductors JD Lau, former GlobalFoundries CTO Pirooz Parvarandeh, and co-founder of Untether AI Raymond Chik.
Having built the bulk of my career on analog and custom IC design for three decades, I know first-hand the [insatiable] need to improve the efficiency of making circuit layouts, Chik said in a statement. Astrus application of deep reinforcement learning may finally be the solution to the sought-after layout problem
Chik contributed $15,000 USD to Astrus $275,000 USD raise in February, which was co-led by Khosa Ventures and 1517 Funds respective $100,000 USD investments, and additional participation from RiSC Capital.
Khosa Ventures invested al $1.5 million USD more into Astrus in September. Other participants in the September fundraise include HOF Capital, MVP Ventures, Alumni Ventures, and Plug and Play Ventures.
Astrus declined to disclose the breakdown of equity and debt in the transactions, as well as the companys post-money valuation.
The company says its platform can read schematics and required parameters from a microchip designers software and database. Designers can tell Astrus what to prioritize, import a pin map, and request a certain number of unique layouts. Astrus then delivers optimized layouts in minutes to hours, following which a designer can make tweaks as needed.
Astrus says the microchip design process is usually a back-and-forth process between design teams that can take weeks to complete.
Moon came to Astrus after working directly under Applyboard CEO Martin Basiri. Moon says he led early product discovery for ApplyBoard, while Wang previously worked in a research group that laid the foundations for AlphaGo, the first computer program to defeat a professional human Go player.
Astrus is looking to use the funding to hire additional members to its team, including a product designer, software engineer, and an analog-integrated circuit designer.
Feature image courtesy Astrus
Read the original post:
With new funding, Astrus has secured $3.6 million CAD to date for its ... - BetaKit - Canadian Startup News
Belt & Road – Latest – The Nation
Prime Minister Kakar will be visiting China this week to attend the scheduled Belt & Road moot where one wonders what our endeavour would really be? Well, one assumes that his contingent would have its strategy worked out, however, it would be important to take note that the outlook on what the developing countries would like to extract from this connectivity stands significantly changed from its formative years. Gone are the days when mere investments regardless of their long-term impact on the respective external accounts would be acceptable. This is what we not only have to mindful of, but also put on the table how the previous such projects need to revisited to ease the fall-outs from the same. No better time to renegotiate the same in a rather polarised global environment that is making it essential for countries to put their own interests and priorities before mere plain political alignments. Only last month the conference on international science, technology and innovation hosted by the Forum saw countries calling instead for a revised focus on instead moving towards evolving a community of innovation. Meaning, that a shift is the sought by the belt & road partners from infrastructure and taxing power plants towards cooperation and technology transfers in science, technology and innovation. Climate change has today become a climate crisis and to address this challenge of global warming the transition needs to be away from simple physical connectivity to digital, economic, social and cultural connectivity - And science, technology & innovation permeate through all forms of connectivities.The idea being for the B&R to have a budget for resources of research and development to help financially weaker countries supplement their smaller spends in this area. This also goes on to help the overall objective to ensure that connectivity takes place smoothly by encouraging the movement of goods & services - by trains, road and sea - in good quality, efficiently and transparently with the help of modern day technology. Also, important now would be the free flow of people across borders to become truly entrenched in a common cum shared work place. For example, while every country may have its own data on vaccination or verified skills of its workers and professionals, but this now needs to developed collectively for all partners to access and benefit from. The other important platform that is under discussion is to achieve From AlphaGO to Chat GPT. Referring to how to capitalise on artificial intelligence (AI) in a responsible manner. Implying that a time has come to try and deeply understand the new scientific technological tools in a way that the new inventions and innovations are implemented in a parallel understanding of the human mind per se. This would be critical in ensuring that current day transformation are beneficial and not damaging for countries and people across the board. Another very important area that Pakistan can capitalise on, given its very young population, is to play its due role in garnering a significant share in offering solutions to the developed countries with an ageing population problem. The reality is that though one cannot exactly predict all climate change impacts, on the other hand one can tangibly calculate the consequences of an aging phenomenon problem. As we know that AI, if used constructively, is today unearthing solutions to many deeper challenging faced by the countries in almost all spheres: economic, security and social development.One hopes that these are the kind of debates that the prime ministers team will be involving itself in and also at the same time looking at steps on how all of these new strategies can be prudently taken up by Pakistan, so that we draw upon the real underlying positives of B&R and do not end missing the boat again by restricting ourselves to projects that perhaps are better undertaken by domestic industrial captains than the imported ones. More importantly, in Thomas Edisons words, Vision without execution is hallucination!
Continue reading here:
Belt & Road - Latest - The Nation
On AI and the soul-stirring char siu rice – asianews.network
October 11, 2023
KUALA LUMPUR Limitations of traditional programming
Firstly, lets consider traditional computer programming.
Here, the computer acts essentially as a puppet, mimicking precisely the set of explicit human-generated instructions.
Take a point-of-sale system at a supermarket as an example: scan a box of Cheerios, and it charges $3; scan a Red Bull, its $2.50.
This robotic repetition of specific commands is probably the most familiar aspect of computers for many people.
This is akin to rote learning from a textbook, start to finish.
But this programmed obedience has limitationssimilar to how following a fixed recipe restricts culinary creativity.
Traditional programming struggles when faced with complex or extensive data.
A set recipe may create a delicious Beef Wellington, but it lacks the capacity to innovate or adapt.
Furthermore, not all data fits neatly into an A corresponds to Bmodel.
Take YouTube videos: their underlying messages cant be easily boiled down into basic algorithms.
This rigidity led to the advent of machine learning or AI,which emerged to discern patterns in data without being explicitly programmed to do so.
Remarkably, the core tenets of machine learning are not entirely new.
Groundwork was being laid as far back as the mid-20th century by pioneers like Alan Turing.
Laksa Penang + Ipoh
During my childhood, my mother saw the value in non-traditional learning methods.
She enrolled me in a memory training course that discouraged rote memorization.
Instead, the emphasis was on creating mind maps and making associative connections between different pieces of information.
Machine learning models operate on a similar principle. They generate their own sort of mind maps, condensing vast data landscapes into more easily navigated territories.
This allows them to form generalizations and adapt to new information.
For instance, if you type King Man + Woman into ChatGPT, it responds with Queen.
This demonstrates that the machine isnt just memorizing words, but understands the relationships between them.
In this case, it deconstructs King into something like royalty + man.
When you subtract man and add woman, the equation becomes royalty + woman, which matches Queen.
For a more localized twist, try typing Laksa Penang + Ipoh in ChatGPT. Youll get Hor Fun. Isnt that fun?
Knowledge graphs and cognitive processes
Machine learning fundamentally boils down to compressing a broad swath of world information into an internal architecture.
This enables machine learning to exhibit what we commonly recognize as intelligence, a mechanism strikingly similar to human cognition.
This idea of internal compression and reconstruction is not unique to machines.
For example, a common misconception is that our eyes function like high-definition cameras, capturing every detail within their view.
The reality is quite different. Just as machine learning models process fragmented data, our brains take in fragmented visual input and then reconstruct it into a more complete picture based on pre-existing knowledge.
Our brains role in filling in these perceptual gaps also makes us susceptible to optical illusions.
You might see two people of identical height appear differently depending on their surroundings.
This phenomenon stems from our brains reliance on built-in rules to complete the picture, and manipulating these rules can produce distortions.
Speaking of rule-breaking, recall the Go match between AlphaGo and Lee Sedol.
The human side was losing until Sedol executed a move that AlphaGos internal knowledge graph hadnt anticipated.
This led to several mistakes by the AI, allowing Sedol to win that round.
Here too, the core concept of data reconstruction is at play.
Beyond chess: The revolution in deep learning
The creation and optimization of knowledge graphs have always been a cornerstone of machine learning.
However, for a long time, this area remained our blind spot.
In the realm of chess, before the advent of deep learning, we leaned heavily on human experience.
We developed chess algorithms based on what we thought were optimal rules, akin to following a fixed recipe for a complex dish like Beef Wellington.
We believed our method was fool-proof.
This belief was challenged by Rich Sutton, a luminary in machine learning, in his blog post The Bitter Lesson.
According to Sutton, our tendency to assume that we have the world all figured out is inherently flawed and short-sighted.
In contrast, recent advancements in machine learning, including AlphaGo Zero and the ChatGPT youre interacting with now, adopt a more flexible, Char Siu Riceapproach.
They learn from raw data with minimal human oversight.
Sutton argues that given the continued exponential growth in computing power, evidenced by Moores Law, this method of autonomous learning is the most sustainable path forward for AI development.
While the concept of computers learning on their ownmight unnerve some people, lets demystify that notion.
Far from edging towardshuman-like self-awareness or sentience, these machines are engaging in advanced forms of data analysis and pattern recognition.
Machine learning models perform the complex dance of parsing, categorization, and linking large sets of dataakin to an expert chef intuitively knowing how to meld flavors and techniques.
These principles are now entrenched in our daily lives.
When you search for something on Google or receive video recommendations on TikTok, its these very algorithms at work.
So, instead of indulging in unwarranted fears about the future of machine learning, lets appreciate the advancements that bring both simplicity and complexity into our lives, much like a perfect bowl of Char Siu Rice.
Read also:
(Yuan-SenTinggraduated from Chong Hwa Independent High School in Kuala Lumpur before earning his degree from Harvard University in 2017. Subsequently, he washonoredwith a Hubble Fellowship from NASA in 2019, allowing him to pursue postdoctoral research at the Institute for Advanced Study in Princeton. Currently, he serves as an associate professor at the Australian National University, splitting his time between the School of Computing and the Research School of Astrophysics and Astronomy. His primary focus is onutilizingadvanced machine learning techniques for statistical inference in the realm of astronomical big data.)
See original here:
On AI and the soul-stirring char siu rice - asianews.network
How AI and the Metaverse will Impact the Datasphere – Visual Capitalist
How AI and the Metaverse Will Impact the Datasphere
The dataspherethe infrastructure that stores and processes our datais critical to many of the advanced technologies on which we rely.
So we partnered with HIVE Digital on this infographic to take a deep dive on how it could evolve to meet the twin challenges of AI and the metaverse.
If the second decade of the 21st century is remembered for anything, it will probably be the leaps and bounds made in the field of AI. Large language models (LLMs) have pushed AI performance to near-human levels, and in some cases beyond. But to get there, it is taking more and more computational resources to train and operate them.
The Large-Scale Era is often considered to have started in late 2015 with the release of DeepMinds AlphaGo Fan, the first computer to defeat a professional Go player.
That LLM required a training compute of 380 quintillion FLOP/s, or floating-point operations per second, a measure of computer performance. In 2023, OpenAIs GPT-4 had a training compute 55 thousand times greater, at 21 septillion FLOP/s.
At this rate of growthessentially doubling every 9.9 monthsfuture AI systems will need exponentially larger computers to train and operate them.
The metaverse, an immersive and frictionless web accessed through augmented and virtual reality (AR and VR), will only add to these demands. One way to quantify this demand is to compare bitrates across applications, which measures the amount of data (i.e. bits) transmitted.
On the low end: music streaming, web browsing, and gaming all have relatively low bitrate requirements. Only streaming gaming breaks the one Mbps (megabits per second) threshold. Things go up from there, and fast. AR, VR, and holograms, all technologies that will be integral for the metaverse, top out at 300 Mbps.
Consider also that VR and AR require incredibly low latencyless than five millisecondsto avoid motion sickness. So not only will the metaverse contribute increase the amount of data that needs to be moved644 GB per household per daybut it will also need to move it very quickly.
At time of writing there are 5,065 data centers worldwide, with 39.0% located in the U.S. The next largest national player is the UK, with only 5.5%. Not only do they store the data we produce, but they also run the applications that we rely on. And they are evolving.
There are two broad approaches that data centers are taking to get ahead of the demand curve. The first and probably most obvious option is going BIG. The worlds three largest hyperscale data centers are:
The other route is to go small, but closer to where the action is. And this is what edge computing does, decentralizing the data center in order to improve latency. This approach will likely play a big part in the rollout of self-driving vehicles, where safety depends on speed.
And investors are putting their money behind the idea. Global spending on edge data centers is expected to hit $208 billion in 2023, up 13.1% from 2022.
The International Data Corporation projects that the amount of data produced annually will grow to 221 zettabytes by 2026, at a compound annual growth rate of 21.2%. With the zettabyte era nearly upon us, data centers will have a critical role to play.
Learn more about how HIVE Digital exports renewably sourced computing power to customers all over the world, helping to meet the demands of emerging technologies like AI.
See more here:
How AI and the Metaverse will Impact the Datasphere - Visual Capitalist
How AI and ML Can Drive Sustainable Revenue Growth by Waleed … – Digital Journal
PRESS RELEASE
Published October 6, 2023
The impact of AI and ML on modern business environments is more than fascinating; it's critical in today's hyper-connected world. While AI and ML have far-reaching practical applications, their greatest disruptive influence may be in business revenue. In this piece, I'll break out why artificial intelligence and machine learning aren't just "nice to have" but a "must have" for any company serious about long-term success.
The Importance of AI and ML in Generating Revenue
In today's data-driven and rapidly evolving environment, tried and true money-generation techniques are no longer sufficient. McKinsey reports that companies using AI in their operations boost revenue by 20% and save expenses by 30%.ChatGPT, GitHub Copilot, Stable Diffusion, and other generative AI applications have captivated global interest due to their widespread accessibility and user-friendly interfaces.
Unlike AlphaGo, which had a more specialized focus, these tools offer almost anyone the ability to communicate, create, and engage in uncanny discussions with a user. It's not merely a wave of the future; it's today's currency.
Practical Applications of AI and ML in Revenue Generation
Several revenue-generating uses for AI and ML exist:
These features may be added to your company model incrementally over time rather than all at once.
Challenges to Adoption and Solutions
The apparent complexity of the technology, concerns over data privacy, and the early expense of deployment are the most prevalent obstacles to AI/ML adoption. Based on my expertise in Turn-Key Design and Systems Integration, I would suggest a staged adoption, beginning with smaller projects to show rapid wins and ROI. In addition, working with other IT companies helps soften the change and save startup expenses.
Increasing Productivity While Lowering Expenses
AI/ML is a tool for improving the efficiency of an organization in addition to helping it make more money. With machine learning, everyday tasks are taken care of by computers. This frees up people to work on more complicated tasks and reduces technical debt. The production can also benefit from AI's ability to simplify back-end activities.
Tendencies and Prospects for the Future
The mutually beneficial connection between AI and ML and their earning potential will deepen as technology advances. Companies that don't change with the times will likely fail in today's fiercely competitive economy.
Final Thoughts
No company that wants to expand its income in a scalable and sustainable way can afford to ignore artificial intelligence and machine learning. It's not a matter of 'if,' but 'when,' AI/ML will become essential to your company's operations.
Who is Waleed Nasir?
Throughout his career, visionary builder and technology specialist Waleed Nasir has launched over a hundred platforms and led countless system deployments and workflow integrations. Dr. Waleed has extensive technical expertise in AI and ML and practical experience building and expanding technology companies. Notable examples of his work include the COVID-19 Crisis Management System, the Paycheck Protection Plan's Programmatic Loan Forgiveness System, and the Emergency Rent Relief Administration System. His wide-ranging skillset includes not just Turn-Key Design but also Process Automation and High-Performance Infrastructure, making him an industry leader in areas beyond only technological innovation. Currently, Dr. Waleed is working with Qult Technologies as the CPO, leading the company to new fronts.
Additional Resources
For those interested in diving deeper into this subject, I recommend:
Media ContactCompany Name: qult.aiContact Person: Hassan Tariq MalikEmail: Send EmailCountry: United KingdomWebsite: https://www.qult.ai/about-us/
Read the rest here:
How AI and ML Can Drive Sustainable Revenue Growth by Waleed ... - Digital Journal
The better the AI gets, the harder it is to ignore – BSA bureau
Hong Kong based Insilico Medicine, a pioneer in AI-based drug discovery, has made significant strides in recent years. Two of their candidates have reached clinical trials, with INS018-055 leading the pack as the first AI-discovered drug designed by generative AI to enter phase 2 clinical trials for idiopathic pulmonary fibrosis (IPF). Back in 2014, when the company began, AI for drug discovery was relatively unheard of, but now it's an indispensable part of the drug discovery process. Insilico's partnerships with major pharmaceutical firms like Janssen underscore the growing importance of AI in this field. Dr Alex Zhavoronkov, Founder and CEO of Insilico Medicine, sheds light on the industry's evolving response to AI in drug discovery, partnerships, regulatory reforms etc. and also shares the company's future plans.
Insilico Medicine has garnered attention for its innovative utilisation of artificial intelligence (AI) in drug discovery. Could you provide insights into how the industry's response to AI-based drug discovery has evolved since your inception in 2014?
In the early days, when I presented at conferences on how generative AI technology could be applied to chemistry, there was a lot of scepticism. I had discovered through my research that generative adversarial networks (GANs) combined with deep reinforcement learning (the same AI learning strategy used in AlphaGo) could generate novel molecules that could be used to treat disease. Since that time, AI drug discovery has undergone enormous acceleration, fueled both by advances in AI technology and in massive stores of data. While there are still no AI-designed drugs on the market, there are a number of companies with these drugs in advanced clinical trials, including our own lead drug for idiopathic pulmonary fibrosis, the drug with an AI-discovered target and designed by generative AI now in Phase II trials with patients.
Although the pharma industry has moved cautiously, the inherent risks in drug discovery (99 per cent of the drugs fail in the early discovery phase and 90 per cent of the drugs fail in clinical trials) and the validation of AI developed drugs to reach advanced trials, means that pharma companies are more actively pursuing partnerships and developing their own internal AI programmes. We have major partnerships with Exelixis, Sanofi and Fosun Pharma to develop new therapies, for instance.
Recently, your two candidates INS018_055, ISM8207 have entered phase II and phase I respectively. Can you share the significance of reaching these stages in the drug development process, and what key milestones do you hope to achieve during these trials?
To our knowledge, Insilicos lead drug for IPF INS018-055 - is the first drug for an AI-discovered target and designed by generative AI to reach Phase 2 clinical trials with patients.
AI was used in every stage of the process. Insilico Medicine used its AI target-discovery engine, https://insilico.com/pandaomics, to process large amounts of data including omics data samples, compounds and biologics, patents, grants, clinical trials, and publications to discover a new target (called Target X) relevant for a broad range of fibrosis indications. We then used this newly discovered target as the basis for the design of a potentially first-in-class novel small molecule inhibitor using its generative AI drug design platform, Chemistry42.
Insilicos molecule INS018_055 - demonstrated highly promising results in multiple preclinical studies including in vitro biological studies, pharmacokinetic, and safety studies. The compound improved myofibroblast activation, a contributor to the development of fibrosis, with a novel mechanism and was shown to have potential relevance in a broad range of fibrotic indications, not just IPF.
The current phase II study is a randomised, double-blind, placebo-controlled trial to assess the safety, tolerability, pharmacokinetics and preliminary efficacy of 12-week oral INS018_055 dosage in subjects with IPF divided into 4 parallel cohorts. To further evaluate the candidate in wider populations, the company plans to recruit 60 subjects with IPF at about 40 sites in both the US and China.
If our phase IIa study is successful, the drug will then go to phase IIb with a larger cohort. This is also the stage where our primary objective would be to determine whether there is significant response to the drug. The drug will go on to be evaluated in a much larger group of patients typically hundreds in phase III studies to confirm safety and effectiveness before it can be approved by the FDA as a new treatment for patients with that condition. We expect to have results from the current phase II trials next year.
Advancing ISM8207 is also significant both because it is the first clinical milestone reached in our partnership with Fosun, and also because it is the first of our cancer drugs to advance to the clinic, and cancer represents the largest disease category in Insilicos pipeline. This drug is a novel QPCTL inhibitor, designed to treat advanced malignant tumours, and works by blocking the tumour cells dont eat me signal. We entered into phase I clinical trials to assess the drugs safety in healthy volunteers in July 2023.
You have had quite successful partnerships with Exelixis, Fosun etc. Can you provide insights into Insilicos approach to forming strategic partnerships? How do you approach deal making?
We have the advantage of being able to produce and advance new, high quality small molecules that have been optimised to treat diseases much more quickly than traditional drug discovery methods. Thats because our generative AI system can optimise across 30 parameters at once based on desired criteria when generating molecules, rather than the traditional method of screening libraries to find a potential compound, and then working to optimise it for each desired property in a linear fashion. As we speed up the drug discovery process on these high-quality molecules we now have 31 in our pipeline we look to find partners who have specific disease expertise and clinical experience to advance these molecules into later stage clinical studies, and, we hope, to market where they can begin helping patients.
Our most recent partnership with Exelixis is a perfect example. We just announced an exclusive global licence agreement with Exelixis with $80 million upfront granting Exelixis the right to develop and commercialise ISM3091, an AI designed cancer drug and potentially best-in-class small molecule inhibitor of USP1 that received IND approval from the FDA in April 2023. This company is expert in cancer and cancer drug development and discovery, and has an expert drug hunting team. Because its an extremely innovative company, they already have substantial revenue coming from best-in-class cancer therapeutics and they are strengthening this pipeline and making bets on innovative cancer drugs.
If we were to look at one of your AI-designed drugs versus a traditionally designed drug candidate, is there a telltale signature?
Our AI-designed drugs will often have a novel structure or work via a novel mechanism compared to existing drugs. By optimising across these 30 different parameters to design molecules with just the right structure and properties to provide the best likelihood of treatment without toxicity and minimal side effects, we are essentially designing ideal new drug-like molecules from scratch. There may be other drugs that are designed to act on those same targets, but ours are optimised through structure or mechanism to be most efficacious, first-in-class, or best-in-class.
Until recently perhaps, big pharma was somewhat sceptical or resistant to AI. What has been responsible for this growing appetite to embrace AI as a fundamental part of the drug discovery process?
There are a number of reasons pharma is now embracing AI. Traditional drug discovery is an incredibly slow and expensive process that fails in clinical trials 90 per cent of the time. AI improves all three of those roadblocks improving speed, lowering cost, and optimising molecules to have the greatest likelihood of clinical trial success. Our AI engine known as PandaOmics can sift through trillions of data points quickly to identify new targets for disease that humans might not find. Then, our generative AI Chemistry42 platform can design brand-new molecules that are optimised to interact with those targets without causing adverse effects, scoring them based on which are likely to work the best. Finally, using our InClinico tool, we can predict how these drugs will likely fare in clinical trials to reduce the time and money lost on failed trials.
There is also now significant validation that this method of developing new drugs is producing very high quality new drugs for hard-to-treat diseases and even diseases that were considered undruggable. And a number of these AI-designed drugs are now in later stage clinical trials.
Finally, the technology is itself progressing and improving with additional use and data via reinforcement learning and expert human feedback. The better the AI gets, the harder it is to ignore.
How sceptical are regulatory bodies towards AI-driven drug discovery? How are regulations evolving to support such developments?
Data privacy and protection are critical to any businesses utilising AI, as is compliance with all international laws and regulations. I expect that these measures will become more stringent in coming years and they are essential to building and maintaining public trust. Insilico Medicine uses only publicly available data and employs privacy by design and by default. We facilitate security of our systems by thorough security analysis on each phase of development. All Insilico data hubs are contained in Amazon Web Services (AWS) or Microsoft Azure cloud.
In addition, there are several checks and balances in place to ensure continuous data integrity, protection and privacy. For example, clients data is not used in any internal environments of the platform, and a firewall is separated for the clients access to the platform versus everyone elses access. All data is encrypted, and data privacy is managed according to Insilico Medicines privacy policy.
What does the future hold for Insilico over the next few years?
Were eager to see our clinical stage programmes progress, and the continued advancement of our lead drug for IPF. Its a terrible, chronic condition with a very poor prognosis and patients are in desperate need of new treatment options.
I also hope that our latest deal with Exelixis marks a trend of pharma companies partnering earlier in the drug development process with highly optimised AI-designed molecules as we continue to expand our pipeline, so that we can truly accelerate the process of delivering new treatments to patients in need.
We will also continue to expand the capabilities of our end-to-end generative AI platform, through new data, reinforcement learning, and expert human feedback; and augment those capabilities with our AI-powered robotics lab as well as incorporating the latest technological tools into our platform, including AlphaFold and quantum computing both of which weve published papers on.
Ayesha Siddiqui
Read the original post:
The better the AI gets, the harder it is to ignore - BSA bureau
Charted: The Exponential Growth in AI Computation – Visual Capitalist
Charted: The Exponential Growth in AI Computation
Electronic computers had barely been around for a decade in the 1940s, before experiments with AI began. Now we have AI models that can write poetry and generate images from textual prompts. But whats led to such exponential growth in such a short time?
This chart from Our World in Data tracks the history of AI through the amount of computation power used to train an AI model, using data from Epoch AI.
In the 1950s, American mathematician Claude Shannon trained a robotic mouse called Theseus to navigate a maze and remember its coursethe first apparent artificial learning of any kind.
Theseus was built on 40 floating point operations (FLOPs), a unit of measurement used to count the number of basic arithmetic operations (addition, subtraction, multiplication, or division) that a computer or processor can perform in one second.
FLOPs are often used as a metric to measure the computational performance of computer hardware. The higher the FLOP count, the higher computation, the more powerful the system.
Computation power, availability of training data, and algorithms are the three main ingredients to AI progress. And for the first few decades of AI advances, compute, which is the computational power needed to train an AI model, grew according to Moores Law.
Source: Compute Trends Across Three Eras of Machine Learning by Sevilla et. al, 2022.
However, at the start of the Deep Learning Era, heralded by AlexNet (an image recognition AI) in 2012, that doubling timeframe shortened considerably to six months, as researchers invested more in computation and processors.
With the emergence of AlphaGo in 2015a computer program that beat a human professional Go playerresearchers have identified a third era: that of the large-scale AI models whose computation needs dwarf all previous AI systems.
Looking back at the only the last decade itself, compute has grown so tremendously its difficult to comprehend.
For example, the compute used to train Minerva, an AI which can solve complex math problems, is nearly 6 million times that which was used to train AlexNet 10 years ago.
Heres a list of important AI models through history and the amount of compute used to train them.
Note: One petaFLOP = one quadrillion FLOPs. Source: Compute Trends Across Three Eras of Machine Learning by Sevilla et. al, 2022.
The result of this growth in computation, along with the availability of massive data sets and better algorithms, has yielded a lot of AI progress in seemingly very little time. Now AI doesnt just match, but also beats human performance in many areas.
Its difficult to say if the same pace of computation growth will be maintained. Large-scale models require increasingly more compute power to train, and if computation doesnt continue to ramp up it could slow down progress. Exhausting all the data currently available for training AI models could also impede the development and implementation of new models.
However with all the funding poured into AI recently, perhaps more breakthroughs are around the cornerlike matching the computation power of the human brain.
Where Does This Data Come From?
Source: Compute Trends Across Three Eras of Machine Learning by Sevilla et. al, 2022.
Note: The time estimated to for computation to double can vary depending on different research attempts, including Amodei and Hernandez (2018) and Lyzhov (2021). This article is based on our sources findings. Please see their full paper for further details. Furthermore, the authors are cognizant of the framing concerns with deeming an AI model regular-sized or large-sized and said further research is needed in the area.
Methodology: The authors of the paper used two methods to determine the amount of compute used to train AI Models: counting the number of operations and tracking GPU time. Both approaches have drawbacks, namely: a lack of transparency with training processes and severe complexity as ML models grow.
This article was published as a part of Visual Capitalist's Creator Program, which features data-driven visuals from some of our favorite Creators around the world.
View post:
Charted: The Exponential Growth in AI Computation - Visual Capitalist