Cloud Hosting

Counterfactual scenarios will be simulated to predict the outcomes of different courses of action.

Computer Science and Coordinated Science Lab professor Shenlong Wang at The Grainger College of Engineering at the University of Illinois received a National Science Foundation (NSF) CAREER award to support the creation of AI systems that can make digital twinsthat is, digital replicasof the physical world that are capable of simulating counterfactual what-if scenarios, enabling users to assess the potential outcomes of actions if they are carried out in the real world.

To do so, he will need to create digital replicas with a greater understanding of the world than current systems, making it possible for them to imagine unseen scenarios rather than just represent things theyve already observed.

Wang explained that the developed tools will be able to produce high-quality imagery with applications in the entertainment industry and in virtual and augmented realitybut, more importantly, that they could also have a very helpful and very profound impact in the real world.

He said the twin worlds will act like the real world, providing realistic observations. The agent can then use these realistic observations to take multiple different actions and then return it to the twin world, and then the twin world can generate multiple expected outcomes. Then, we can decide which outcome we want. And then we can transfer this insight to help the agent make the right decision in the real world.

There are multiple reasons why such a capability would be highly desirable. It would allow users to try out dangerous actionssay, new surgical techniquesin a risk-free way. It would make it possible to see what happens in scenarios for which little data are available, such as situations that are rare in real lifefor example, what happens to a citys water distribution infrastructure during a 100-year floodwithout having to wait for such scenarios to happen in reality. Further, because virtual time can run much faster than real-time, models can be run forward to predict the long-term consequences of actions, even decades into the future.

If computers can help us to make more informed predictions, we essentially have a time machine, said Wang. We might be able to see whats going on in the next century!

This diagram shows how hypothetical scenarios can be run through the worlds digital twin (right side), allowing users to rehearse multiple proposed solutions in the virtual world before choosing one to apply in the real world (left side).

Photo Credit: Shenlong Wang

Wang will consider two use cases in the project. One of them is autonomous driving, which is familiar territory for him. Before joining Illinois'faculty, he worked as a research scientist at Uber, building simulators to test self-driving vehicles safety. His second use case will be climate risk assessment for agriculture. For that, the shorter-term goal is to determine things like carbon emission levels based on ordinary mobile phone photographs of a soybean field; in the longer-term, the more ambitious goal is to gain such insights from satellite imagery.

Wang said that hes particularly happy about the planned education and outreach components of his project. He anticipates that the strong visual appeal of the work and the excitement of being able to create ones own virtual world will be attractive to young people. While studying computer vision and machine learning normally starts with a lot of intimidating math, he plans to offer an immersive initial experience to make this area more approachable to K through 12 schoolchildren.

Wang'sFaculty Early Career Development (CAREER) Award to Digitize and Simulate the Large Physical World via Knowledge-Grounded Scene Representation is the NSF's most prestigious award in support of early-career faculty who have the potential to serve as academic role models in research and education and to lead advances in the mission of their department or organization.

See more here:

Shenlong Wang wins NSF CAREER award to create AI systems that can imagine hypothetical scenarios in the physical ... - Illinois Computer Science News