On November 18th, Dialogues with Global Young Scholars was held at South China University of Technology, Guangzhou, China. The theme of the conference was Innovation, Youth and Future. Four young scholars who had made outstanding contributions to the field of quantum physics and the other 35 under-35-year-old scholars among the laureates of MIT Technology ReviewsInnovators were invited.

Dai Lei, a researchprofessorat the Shenzhen Institutes of Advanced Technology of the Chinese Academy of Sciences, introduced the key role of gut microbiota in health. He set forth the research breakthroughs made in areas including design principles of synthetic organisms, synthetic yeast chromosomes, and genetically modified bacteria as a treatment for tumors, where the research is brimming with both opportunities and challenges. He also emphasized the significance ofbiotechnology in the boom of technological innovation.

He Ke, a professor from the Department of Physics at Tsinghua University, elaborated on the findings regarding the molecular beam epitaxial growth of low-dimensional topological quantum material systems, as well as its electron configuration and quantum effect. He pointed out that as the microscopic world is dominated by laws of quantum mechanics, it is possible for topological quantum computation to solve the issue of error correction in quantum computing.

Chen Yunji, a research professor at the Institute of Computing Technology of the Chinese Academy of Sciences, introduced the principles, design strategies and significance of deep learning processors. He believes that in the future, every computer may need a deep learning processor of its own. His vision for the future is to carry around AlphaGo in pockets.

Radha Boya, a professor of nanosciencefrom the Department of Physics & Astronomy at the University of Manchesterdelivered a presentation on the latest findings and developments in the field of graphene. Due to the COVID-19 pandemic, the lecture was given via a video recording. She believes that by processing graphene materials with filtration technologies to achieve the same effect as atomic-scale capillaries do, we can help solve the key issue of seawater desalination across the globe.

The event was held during the finals of 6th China International College Students Internet+ Innovation and Entrepreneurship Competition. Nearly 500 students, teachers, as well as representatives of the contestants and the South China University of Technologyattended the event. Live-streaming views of the event has hit more than 100,000 online.

Media ContactCompany Name: South China University of TechnologyContact Person: Tao ZhouEmail: Send EmailCountry: ChinaWebsite: http://www2.scut.edu.cn/internetplus2020/main.htm

Read more here:
Dialogues with Global Young Scholars Held in Guangzhou - Invest Courier

Earlier this year, the independent research organisation of which I am the Director, London-based Ada Lovelace Institute, hosted a panel at the worlds largest AI conference, CogX, called The Ethics Panel to End All Ethics Panels. The title referenced both a tongue-in-cheek effort at self-promotion, and a very real need to put to bed the seemingly endless offering of panels, think-pieces, and government reports preoccupied with ruminating on the abstract ethical questions posed by AI and new data-driven technologies. We had grown impatient with conceptual debates and high-level principles.

And we were not alone. 2020 has seen the emergence of a new wave of ethical AI one focused on the tough questions of power, equity, and justice that underpin emerging technologies, and directed at bringing about actionable change. It supersedes the two waves that came before it: the first wave, defined by principles and dominated by philosophers, and the second wave, led by computer scientists and geared towards technical fixes. Third-wave ethical AI has seen a Dutch Court shut down an algorithmic fraud detection system, students in the UK take to the streets to protest against algorithmically-decided exam results, and US companies voluntarily restrict their sales of facial recognition technology. It is taking us beyond the principled and the technical, to practical mechanisms for rectifying power imbalances and achieving individual and societal justice.

Between 2016 and 2019, 74 sets of ethical principles or guidelines for AI were published. This was the first wave of ethical AI, in which we had just begun to understand the potential risks and threats of rapidly advancing machine learning and AI capabilities and were casting around for ways to contain them. In 2016, AlphaGo had just beaten Lee Sedol, promoting serious consideration of the likelihood that general AI was within reach. And algorithmically-curated chaos on the worlds duopolistic platforms, Google and Facebook, had surrounded the two major political earthquakes of the year Brexit, and Trumps election.

In a panic for how to understand and prevent the harm that was so clearly to follow, policymakers and tech developers turned to philosophers and ethicists to develop codes and standards. These often recycled a subset of the same concepts and rarely moved beyond high-level guidance or contained the specificity of the kind needed to speak to individual use cases and applications.

This first wave of the movement focused on ethics over law, neglected questions related to systemic injustice and control of infrastructures, and was unwilling to deal with what Michael Veale, Lecturer in Digital Rights and Regulation at University College London, calls the question of problem framing early ethical AI debates usually took as a given that AI will be helpful in solving problems. These shortcomings left the movement open to critique that it had been co-opted by the big tech companies as a means of evading greater regulatory intervention. And those who believed big tech companies were controlling the discourse around ethical AI saw the movement as ethics washing. The flow of money from big tech into codification initiatives, civil society, and academia advocating for an ethics-based approach only underscored the legitimacy of these critiques.

At the same time, a second wave of ethical AI was emerging. It sought to promote the use of technical interventions to address ethical harms, particularly those related to fairness, bias and non-discrimination.The domain of fair-ML was born out of an admirable objective on thepart of computer scientists to bake fairness metrics or hard constraints into AI models to moderate their outputs.

This focus on technical mechanisms for addressing questions of fairness, bias, and discrimination addressed the clear concerns about how AI and algorithmic systems were inaccurately and unfairly treating people of color or ethnic minorities. Two specific cases contributed important evidence to this argument. The first was the Gender Shades study, which established that facial recognition software deployed by Microsoft and IBM returned higher rates of false positives and false negatives for the faces of women and people ofcolor. The second was the 2016 ProPublica investigation into the COMPAS sentencing algorithmic tool, whichfound that Black defendants were far more likely than White defendants to be incorrectly judged to be at a higher risk of recidivism, while White defendants were more likely than Black defendants to be incorrectly flagged as low risk.

Second-wave ethical AI narrowed in on these questions of bias and fairness, and explored technical interventions to solve them. In doing so, however, it may have skewed and narrowed the discourse, moving it away from the root causes of bias and even exacerbating the position of people of color and ethnic minorities. As Julia Powles, Director of the Minderoo Tech and Policy Lab at the University of Western Australia, argued, alleviating the problems with dataset representativeness merely co-opts designers in perfecting vast instruments of surveillance and classification. When underlying systemic issues remain fundamentally untouched, the bias fighters simply render humans more machine readable, exposing minorities in particular to additional harms.

Some also saw the fair-ML discourse as a form of co-option of socially conscious computer scientists by big tech companies. By framing ethical problems as narrow issues of fairness and accuracy, companies could equate expanded data collection with investing in ethical AI.

The efforts of tech companies tochampion fairness-related codes illustrate this point: In January 2018, Microsoft published its ethical principles for AI, starting with fairness; in May2018, Facebook announced a tool to search for bias called Fairness Flow; and in September2018, IBM announced a tool called AI Fairness 360, designed to check for unwanted bias in datasets and machine learning models.

What was missing from second-wave ethical AI was an acknowledgement that technical systems are, in fact, sociotechnical systems they cannot be understood outside of the social context in which they are deployed, and they cannot be optimised for societally beneficial and acceptable outcomes through technical tweaks alone. As Ruha Benjamin, Associate Professor of African American Studies at Princeton University, argued in her seminal text, Race After Technology: Abolitionist Tools for the New Jim Code, the road to inequity is paved with technical fixes. The narrow focus on technical fairness is insufficient to help us grapple with all of the complex tradeoffs, opportunities, and risks of an AI-driven future; it confines us to thinking only about whether something works, but doesnt permit us to ask whether it should work. That is, it supports an approach that asks, What can we do? rather than What should we do?

On the eve of the new decade, MIT Technology Reviews Karen Hao published an article entitled In 2020, lets stop AI ethics-washing and actually do something. Weeks later, the AI ethics community ushered in 2020 clustered in conference rooms at Barcelona, for the annual ACM Fairness, Accountability and Transparency conference. Among the many papers that had tongues wagging was written by Elettra Bietti, Kennedy Sinclair Scholar Affiliate at the Berkman Klein Center for Internet and Society. It called for a move beyond the ethics-washing and ethics-bashing that had come to dominate the discipline. Those two pieces heralded a cascade of interventions that saw the community reorienting around a new way of talking about ethical AI, one defined by justice social justice, racial justice, economic justice, and environmental justice. It has seen some eschew the term ethical AI in favor of just AI.

As the wild and unpredicted events of 2020 have unfurled, alongside them third-wave ethical AI has begun to take hold, strengthened by the immense reckoning that the Black Lives Matter movement has catalysed. Third-wave ethical AI is less conceptual than first-wave ethical AI, and is interested in understanding applications and use cases. It is much more concerned with power, alive to vested interests, and preoccupied with structural issues, including the importance of decolonising AI.An article published by Pratyusha Kalluri, founder of the Radical AI Network, in Nature in July 2020, has epitomized the approach, arguing that When the field of AI believes it is neutral, it both fails to notice biased data and builds systems that sanctify the status quo and advance the interests of the powerful. What is needed is a field that exposes and critiques systems that concentrate power, while co-creating new systems with impacted communities: AI by and for the people.

What has this meant in practice? We have seen courts begin to grapple with, and political and private sector players admit to, the real power and potential of algorithmic systems. In the UK alone, the Court of Appeal found the use by police of facial recognition systems unlawful and called for a new legal framework; a government department ceased its use of AI for visa application sorting; the West Midlands police ethics advisory committee argued for the discontinuation of a violence-prediction tool; and high school students across the country protested after tens of thousands of school leavers had their marks downgraded by an algorithmic system used by the education regulator, Ofqual. New Zealand published an Algorithm Charter and Frances Etalab a government task force for open data, data policy, and open government has been working to map the algorithmic systems in use across public sector entities and to provide guidance.

The shift in gaze of ethical AI studies away from the technical towards the socio-technical has brought more issues into view, such as the anti-competitive practices of big tech companies, platform labor practices, parity in negotiating power in public sector procurement of predictive analytics, and the climate impact of training AI models. It has seen the Overton window contract in terms of what is reputationally acceptable from tech companies; after years of campaigning by researchers like Joy Buolamwini and Timnit Gebru, companies such as Amazon and IBM have finally adopted voluntary moratoria on their sales of facial recognition technology.

The COVID crisis has been instrumental, surfacing technical advancements that have helped to fix the power imbalances that exacerbate the risks of AI and algorithmic systems. The availability of the Google/Apple decentralised protocol for enabling exposure notification prevented dozens of governments from launching invasive digital contact tracing apps. At the same time, governments response to the pandemic has inevitably catalysed new risks, as public health surveillance has segued into population surveillance, facial recognition systems have been enhanced to work around masks, and the threat of future pandemics is leveraged to justify social media analysis. The UKs attempt to operationalize a weak Ethics Advisory Board to oversee its failed attempt at launching a centralized contact-tracing app was the death knell for toothless ethical figureheads.

Research institutes, activists, and campaigners united by the third-wave approach to ethical AI continue to work to address these risks, with a focus on practical tools for accountability (we at the Ada Lovelace Institute, and others such as AI Now, are working on developing audit and assessment tools for AI; and the Omidyar Network has published itsEthical Explorer toolkit for developers and product managers), litigation, protest and campaigning for moratoria, and bans.

Researchers are interrogating what justice means in data-driven societies, and institutes such as Data & Society, the Data Justice Lab at Cardiff University, JUST DATA Lab at Princeton, and the Global Data Justice project at the Tilberg Institute for Law, Technology and Society in the Netherlands are churning out some of the most novel thinking. The Mindaroo Foundation has just launched its new future says initiative with a $3.5 million grant, with aims to tackle lawlessness, empower workers, and reimagine the tech sector. The initiative will build on the critical contribution of tech workers themselves to the third wave of ethical AI, from AI Now co-founder Meredith Whittakers organizing work at Google before her departure last year, to walk outs and strikes performed by Amazon logistic workersand Uber and Lyft drivers.

But the approach of third-wave ethical AI is by no means accepted across the tech sector yet, as evidenced by the recent acrimonious exchange between AI researchers Yann LeCun and Timnit Gebru about whether the harms of AI should be reduced to a focus on bias. Gebru not only reasserted well established arguments against a narrow focus on dataset bias but also made the case for a more inclusive community of AI scholarship.

Mobilized by social pressure, the boundaries of acceptability are shifting fast, and not a moment too soon. But even those of us within the ethical AI community have a long way to go. A case in point: Although wed programmed diverse speakers across the event, the Ethics Panel to End All Ethics Panels we hosted earlier this year failed to include a person of color, an omission for which we were rightly criticized and hugely regretful. It was a reminder that as long as the domain of AI ethics continues to platform certain types of research approaches, practitioners, and ethical perspectives to the exclusion of others, real change will elude us. Ethical AI can not only be defined from the position of European and North American actors; we need to work concertedly to surface other perspectives, other ways of thinking about these issues, if we truly want to find a way to make data and AI work for people and societies across the world.

Carly Kind is a human rights lawyer, a privacy and data protection expert, and Director of the Ada Lovelace Institute.

Visit link:
The term 'ethical AI' is finally starting to mean something - Report Door

A sesquipedalian is a person who overuses uncommon words like lameen (a bishops letter expressing a fault or reprimand) or salvestate (to transport car seats to the dining room) just for the sake of it. The first of those italicized words is real. The second two arent. But they totally should be. Theyre the invention of a new website called This Word Does Not Exist. Powered by machine learning, it conjures up entirely new words never before seen or used, and even generates a halfway convincing definition for them. Its all kinds of brilliant.

In February, I quit my job as an engineering director at Instagram after spending seven intense years building their ranking algorithms like non-chronological feed, Thomas Dimson, creator of This Word Does Not Exist, told Digital Trends. A friend and I were trying to brainstorm names for a company we could start together in the A.I. space. After [coming up with] some lame ones, I decided it was more appropriate to let A.I. name a company about A.I.

Then, as Dimson tells it, a global pandemic happened, and he found himself at home with lots of time on his hands to play around with his name-making algorithm. Eventually I stumbled upon the Mac dictionary as a potential training set and [started] generating arbitrary words instead of just company names, he said.

If youve ever joked that someone who uses complex words in their daily lives must have swallowed a dictionary, thats pretty much exactly what This Word Does Not Exist has done. The algorithm was trained from a dictionary file Dimson structured according to different parts of speech, definition, and example usage. The model refines OpenAIs controversial GPT-2 text generator, the much-hyped algorithm once called too dangerous to release to the public. Dimsons twist on it assigns probabilities to potential words based on which letters are likely to follow one another until the word looks like a reasonably convincing dictionary entry. As a final step, it checks that the generated word isnt a real one by looking it up in the original training set.

This Word Does Not Exist is just the latest in a series of [Insert object] Does Not Exist creations. Others range from non-existent Airbnb listings to fake people to computer-generated memes which nonetheless capture the oddball humor of real ones.

People have a nervous curiosity toward what makes us human, Dimson said. By looking at these machine-produced demos, we are better able to understand ourselves. Im reminded of the fascination with Deep Blue beating Kasparov in 1996 or AlphaGo beating Lee Sedol in 2016.

Read more from the original source:
This A.I. makes up gibberish words and definitions that sound astonishingly real - Yahoo Tech

Inez Fung

Professor of atmospheric science, University of California, Berkeley

Leah Fasten

Prediction for 2030: Well light up the world safely

Ive spoken to people who want climate model information, but theyre not really sure what theyre asking me for. So I say to them, Suppose I tell you that some event will happen with a probability of 60% in 2030. Will that be good enough for you, or will you need 70%? Or would you need 90%? What level of information do you want out of climate model projections in order to be useful?

I joined Jim Hansens group in 1979, and I was there for all the early climate projections. And the way we thought about it then, those things are all still totally there. What weve done since then is add richness and higher resolution, but the projections are really grounded in the same kind of data, physics, and observations.

Still, there are things were missing. We still dont have a real theory of precipitation, for example. But there are two exciting things happening there. One is the availability of satellite observations: looking at the cloud is still not totally utilized. The other is that there used to be no way to get regional precipitation patterns through historyand now there is. Scientists found these caves in China and elsewhere, and they go in, look for a nice little chamber with stalagmites, and then they chop them up and send them back to the lab, where they do fantastic uranium--thorium dating and measure oxygen isotopes in calcium carbonate. From there they can interpret a record of historic rainfall. The data are incredible: we have got over half a million years of precipitation records all over Asia.

I dont see us reducing fossil fuels by 2030. I dont see us reducing CO2 or atmospheric methane. Some 1.2 billion people in the world right now have no access to electricity, so Im looking forward to the growth in alternative energy going to parts of the world that have no electricity. Thats important because its education, health, everything associated with a Western standard of living. Thats where Im putting my hopes.

Dvora Photography

Anne Lise Kjaer

Futurist, Kjaer Global, London

Prediction for 2030: Adults will learn to grasp new ideas

As a kid I wanted to become an archaeologist, and I did in a way. Archaeologists find artifacts from the past and try to connect the dots and tell a story about how the past might have been. We do the same thing as futurists; we use artifacts from the present and try to connect the dots into interesting narratives in the future.

When it comes to the future, you have two choices. You can sit back and think Its not happening to me and build a great big wall to keep out all the bad news. Or you can build windmills and harness the winds of change.

A lot of companies come to us and think they want to hear about the future, but really its just an exercise for themlets just tick that box, do a report, and put it on our bookshelf.

Sign up for The Download your daily dose of what's up in emerging technology

So we have a little test for them. We do interviews, we ask them questions; then we use a model called a Trend Atlas that considers both the scientific dimensions of society and the social ones. We look at the trends in politics, economics, societal drivers, technology, environment, legislationhow does that fit with what we know currently? We look back maybe 10, 20 years: can we see a little bit of a trend and try to put that into the future?

Whats next? Obviously with technology we can educate much better than we could in the past. But its a huge opportunity to educate the parents of the next generation, not just the children. Kids are learning about sustainability goals, but what about the people who actually rule our world?

Courtesy Photo

Philip Tetlock

Coauthor of Superforecasting and professor, University of Pennsylvania

Prediction for 2030: Well get better at being uncertain

At the Good Judgment Project, we try to track the accuracy of commentators and experts in domains in which its usually thought impossible to track accuracy. You take a big debate and break it down into a series of testable short-term indicators. So you could take a debate over whether strong forms of artificial intelligence are going to cause major dislocations in white-collar labor markets by 2035, 2040, 2050. A lot of discussion already occurs at that level of abstractionbut from our point of view, its more useful to break it down and to say: If we were on a long-term trajectory toward an outcome like that, what sorts of things would we expect to observe in the short term? So we started this off in 2015, and in 2016 AlphaGo defeated people in Go. But then other things didnt happen: driverless Ubers werent picking people up for fares in any major American city at the end of 2017. Watson didnt defeat the worlds best oncologists in a medical diagnosis tournament. So I dont think were on a fast track toward the singularity, put it that way.

Forecasts have the potential to be either self-fulfilling or self-negatingY2K was arguably a self-negating forecast. But its possible to build that into a forecasting tournament by asking conditional forecasting questions: i.e., How likely is X conditional on our doing this or doing that?

What Ive seen over the last 10 years, and its a trend that I expect will continue, is an increasing openness to the quantification of uncertainty. I think theres a grudging, halting, but cumulative movement toward thinking about uncertainty, and more granular and nuanced ways that permit keeping score.

Ryan Young

Keith Chen

Associate professor of economics, UCLA

Prediction for 2030: Well be moreand lessprivate

When I worked on Ubers surge pricing algorithm, the problem it was built to solve was very coarse: we were trying to convince drivers to put in extra time when they were most needed. There were predictable timeslike New Yearswhen we knew we were going to need a lot of people. The deeper problem was that this was a system with basically no control. Its like trying to predict the weather. Yes, the amount of weather data that we collect todaytemperature, wind speed, barometric pressure, humidity datais 10,000 times greater than what we were collecting 20 years ago. But we still cant predict the weather 10,000 times further out than we could back then. And social movementseven in a very specific setting, such as where riders want to go at any given point in timeare, if anything, even more chaotic than weather systems.

These days what Im doing is a little bit more like forensic economics. We look to see what we can find and predict from peoples movement patterns. Were just using simple cell-phone data like geolocation, but even just from movement patterns, we can infer salient information and build a psychological dimension of you. What terrifies me is I feel like I have much worse data than Facebook does. So what are they able to understand with their much better information?

I think the next big social tipping point is people actually starting to really care about their privacy. Itll be like smoking in a restaurant: it will quickly go from causing outrage when people want to stop it to suddenly causing outrage if somebody does it. But at the same time, by 2030 almost every Chinese citizen will be completely genotyped. I dont quite know how to reconcile the two.

Sarah Deragon

Annalee Newitz

Science fiction and nonfiction author, San Francisco

Prediction for 2030: Were going to see a lot more humble technology

Every era has its own ideas about the future. Go back to the 1950s and youll see that people fantasized about flying cars. Now we imagine bicycles and green cities where cars are limited, or where cars are autonomous. We have really different priorities now, so that works its way into our understanding of the future.

Science fiction writers cant actually make predictions. I think of science fiction as engaging with questions being raised in the present. But what we can do, even if we cant say whats definitely going to happen, is offer a range of scenarios informed by history.

There are a lot of myths about the future that people believe are going to come true right now. I think a lot of peoplenot just science fiction writers but people who are working on machine learningbelieve that relatively soon were going to have a human-equivalent brain running on some kind of computing substrate. This is as much a reflection of our time as it is what might actually happen.

It seems unlikely that a human--equivalent brain in a computer is right around the corner. But we live in an era where a lot of us feel like we live inside computers already, for work and everything else. So of course we have fantasies about digitizing our brains and putting our consciousness inside a machine or a robot.

Im not saying that those things could never happen. But they seem much more closely allied to our fantasies in the present than they do to a real technical breakthrough on the horizon.

Were going to have to develop much better technologies around disaster relief and emergency response, because well be seeing a lot more floods, fires, storms. So I think there is going to be a lot more work on really humble technologies that allow you to take your community off the grid, or purify your own water. And I dont mean in a creepy survivalist way; I mean just in a this-is-how-we-are-living-now kind of way.

Noah Willman

Finale Doshi-Velez

Associate professor of computer science, Harvard

Prediction for 2030: Humans and machines will make decisions together

In my lab, were trying to answer questions like How might this patient respond to this antidepressant? or How might this patient respond to this vasopressor? So we get as much data as we can from the hospital. For a psychiatric patient, we might have everything about their heart disease, kidney disease, cancer; for a blood pressure management recommendation for the ICU, we have all their oxygen information, their lactate, and more.

Some of it might be relevant to making predictions about their illnesses, some not, and we dont know which is which. Thats why we ask for the large data set with everything.

Theres been about a decade of work trying to get unsupervised machine-learning models to do a better job at making these predictions, and none worked really well. The breakthrough for us was when we found that all the previous approaches for doing this were wrong in the exact same way. Once we untangled all of this, we came up with a different method.

We also realized that even if our ability to predict what drug is going to work is not always that great, we can more reliably predict what drugs are not going to work, which is almost as valuable.

Im excited about combining humans and AI to make predictions. Lets say your AI has an error rate of 70% and your human is also only right 70% of the time. Combining the two is difficult, but if you can fuse their successes, then you should be able to do better than either system alone. How to do that is a really tough, exciting question.

All these predictive models were built and deployed and people didnt think enough about potential biases. Im hopeful that were going to have a future where these human-machine teams are making decisions that are better than either alone.

Guillaume Simoneau

Abdoulaye Banire Diallo

Professor, director of the bioinformatics lab, University of Quebec at Montreal

Prediction for 2030: Machine-based forecasting will be regulated

When a farmer in Quebec decides whether to inseminate a cow or not, it might depend on the expectation of milk that will be produced every day for one year, two years, maybe three years after that. Farms have management systems that capture the data and the environment of the farm. Im involved in projects that add a layer of genetic and genomic data to help forecastingto help decision makers like the farmer to have a full picture when theyre thinking about replacing cows, improving management, resilience, and animal welfare.

With the emergence of machine learning and AI, what were showing is that we can help tackle problems in a way that hasnt been done before. We are adapting it to the dairy sector, where weve shown that some decisions can be anticipated 18 months in advance just by forecasting based on the integration of this genomic data. I think in some areas such as plant health we have only achieved 10% or 20% of our capacity to improve certain models.

Until now AI and machine learning have been associated with domain expertise. Its not a public-wide thing. But less than 10 years from now they will need to be regulated. I think there are a lot of challenges for scientists like me to try to make those techniques more explainable, more transparent, and more auditable.

Go here to read the rest:
The professionals who predict the future for a living - MIT Technology Review

Geoffrey Hinton, center. talks about what future deep learning neural nets may look like, flanked by Yann LeCun of Facebook, right, and Yoshua Bengio of Montreal's MILA institute for AI, during a press conference at the 34th annual AAAI conference on artificial intelligence.

The rise of dedicated chips and systems for artificial intelligence will "make possible a lot of stuff that's not possible now," said Geoffrey Hinton, the University of Toronto professor who is one of the godfathers of the "deep learning" school of artificial intelligence, during a press conference on Monday.

Hinton joined his compatriots, Yann LeCun of Facebook and Yoshua Bengio of Canada's MILA institute, fellow deep learning pioneers, in an upstairs meeting room of the Hilton Hotel on the sidelines of the 34th annual conference on AI by the Association for the Advancement of Artificial Intelligence. They spoke for 45 minutes to a small group of reporters on a variety of topics, including AI ethics and what "common sense" might mean in AI. The night before, all three had presented their latest research directions.

Regarding hardware, Hinton went into an extended explanation of the technical aspects that constrain today's neural networks. The weights of a neural network, for example, have to be used hundreds of times, he pointed out, making frequent, temporary updates to the weights. He said the fact graphics processing units (GPUs) have limited memory for weights and have to constantly store and retrieve them in external DRAM is a limiting factor.

Much larger on-chip memory capacity "will help with things like Transformer, for soft attention," said Hinton, referring to the wildly popular autoregressive neural network developed at Google in 2017. Transformers, which use "key/value" pairs to store and retrieve from memory, could be much larger with a chip that has substantial embedded memory, he said.

Also: Deep learning godfathers Bengio, Hinton, and LeCun say the field can fix its flaws

LeCun and Bengio agreed, with LeCun noting that GPUs "force us to do batching," where data samples are combined in groups as they pass through a neural network, "which isn't efficient." Another problem is that GPUs assume neural networks are built out of matrix products, which forces constraints on the kind of transformations scientists can build into such networks.

"Also sparse computation, which isn't convenient to run on GPUs ...," said Bengio, referring to instances where most of the data, such as pixel values, may be empty, with only a few significant bits to work on.

LeCun predicted they new hardware would lead to "much bigger neural nets with sparse activations," and he and Bengio both emphasized there is an interest in doing the same amount of work with less energy. LeCun defended AI against claims it is an energy hog, however. "This idea that AI is eating the atmosphere, it's just wrong," he said. "I mean, just compare it to something like raising cows," he continued. "The energy consumed by Facebook annually for each Facebook user is 1,500-watt hours," he said. Not a lot, in his view, compared to other energy-hogging technologies.

The biggest problem with hardware, mused LeCun, is that on the training side of things, it is a duopoly between Nvidia, for GPUs, and Google's Tensor Processing Unit (TPU), repeating a point he had made last year at the International Solid-State Circuits Conference.

Even more interesting than hardware for training, LeCun said, is hardware design for inference. "You now want to run on an augmented reality device, say, and you need a chip that consumes milliwatts of power and runs for an entire day on a battery." LeCun reiterated a statement made a year ago that Facebook is working on various internal hardware projects for AI, including for inference, but he declined to go into details.

Also: Facebook's Yann LeCun says 'internal activity' proceeds on AI chips

Today's neural networks are tiny, Hinton noted, with really big ones having perhaps just ten billion parameters. Progress on hardware might advance AI just by making much bigger nets with an order of magnitude more weights. "There are one trillion synapses in a cubic centimeter of the brain," he noted. "If there is such a thing as General AI, it would probably require one trillion synapses."

As for what "common sense" might look like in a machine, nobody really knows, Bengio maintained. Hinton complained people keep moving the goalposts, such as with natural language models. "We finally did it, and then they said it's not really understanding, and can you figure out the pronoun references in the Winograd Schema Challenge," a question-answering task used a computer language benchmark. "Now we are doing pretty well at that, and they want to find something else" to judge machine learning he said. "It's like trying to argue with a religious person, there's no way you can win."

But, one reporter asked, what's concerning to the public is not so much the lack of evidence of human understanding, but evidence that machines are operating in alien ways, such as the "adversarial examples." Hinton replied that adversarial examples show the behavior of classifiers is not quite right yet. "Although we are able to classify things correctly, the networks are doing it absolutely for the wrong reasons," he said. "Adversarial examples show us that machines are doing things in ways that are different from us."

LeCun pointed out animals can also be fooled just like machines. "You can design a test so it would be right for a human, but it wouldn't work for this other creature," he mused. Hinton concurred, observing "house cats have this same limitation."

Also: LeCun, Hinton, Bengio: AI conspirators awarded prestigious Turing prize

"You have a cat lying on a staircase, and if you bounce a soccer ball down the stairs toward a care, the cat will just sort of watch the ball bounce until it hits the cat in the face."

Another thing that could prove a giant advance for AI, all three agreed, is robotics. "We are at the beginning of a revolution," said Hinton. "It's going to be a big deal" to many applications such as vision. Rather than analyzing the entire contents of a static image or video frame, a robot creates a new "model of perception," he said.

"You're going to look somewhere, and then look somewhere else, so it now becomes a sequential process that involves acts of attention," he explained.

Hinton predicted last year's work by OpenAI in manipulating a Rubik's cube was a watershed moment for robotics, or, rather, an "AlphaGo moment," as he put it, referring to DeepMind's Go computer.

LeCun concurred, saying that Facebook is running AI projects not because Facebook has an extreme interest in robotics, per se, but because it is seen as an "important substrate for advances in AI research."

It wasn't all gee-whiz, the three scientists offered skepticism on some points. While most research in deep learning that matters is done out in the open, some companies boast of AI while keeping the details a secret.

"It's hidden because it's making it seem important," said Bengio, when in fact, a lot of work in the depths of companies may not be groundbreaking. "Sometimes companies make it look a lot more sophisticated than it is."

Bengio continued his role among the three of being much more outspoken on societal issues of AI, such as building ethical systems.

When LeCun was asked about the use of factual recognition algorithms, he noted technology can be used for good and bad purposes, and that a lot depends on the democratic institutions of society. But Bengio pushed back slightly, saying, "What Yann is saying is clearly true, but prominent scientists have a responsibility to speak out." LeCun mused that it's not the job of science to "decide for society," prompting Bengio to respond, "I'm not saying decide, I'm saying we should weigh in because governments in some countries are open to that involvement."

Hinton, who frequently punctuates things with a humorous aside, noted toward the end of the gathering his biggest mistake with respect to Nvidia. "I made a big mistake back in with Nvidia," he said. "In 2009, I told an audience of 1,000 grad students they should go and buy Nvidia GPUs to speed up their neural nets. I called Nvidia and said I just recommended your GPUs to 1,000 researchers, can you give me a free one, and they said no.

"What I should have done, if I was really smart, was take all my savings and put it into Nvidia stock. The stock was at $20 then, now it's, like, $250."

Read more here:
AI on steroids: Much bigger neural nets to come with new hardware, say Bengio, Hinton, and LeCun - ZDNet