They’ll use Riemannian geometry, which originally was invented in the early 20th century mathematics community to study the curvature of surfaces. With careful reworking and fusion with deep networks, we will be able to assess how the shape and geometry of an object can make things easier to identify.

The Geometric Media Lab at ASU is well positioned to be the bridge between the neural networks world and the physics world, said Turaga, whose PhD thesis was about advancing a similar technique of using geometric methods to encode models of physics.

A neural network can be trained, through repetition, to recognize images. The networks do that by focusing on regions of the image. 

“The attention of networks is often on color, and shading – it’s very diffuse and not precise,” Turaga said. “It’s looking at a lot and coming up with a decision that may or may not be optimal.

“We’re proposing, let’s look at more detailed geometry. What is the shape? And encode that definition in a way that neural networks can train under.”

The network will process the pixels the same way but with better reliability.

“When we say, ‘Tell me what you’re paying attention to while you’re reaching this decision,’ it’s more clearly focusing on the features of the objects that make it what it is,” they said.

“We’re able to get the neural network to pay attention to the relevant part of the object, such as shape and texture. If you don’t do that, you get a neural network that pays attention to a lot of distracting things.”

Turaga’s lab is able to show how their technique improves the network’s processing through “attention map” images that look like heat maps.

“Our approach shows that we can train networks to pay attention by highlighting shapes more precisely.

“We’re trying to make the neural networks more trustworthy by creating these visualizations that help us trust what they’re doing.”

 

 

Above This image shows how the Geometric Media Lab’s technology is improving computers’ ability to reliably identify objects in images by teaching the computer to pay attention to the objects’ shapes. The top row is the object in the image. The second row shows what the computer neural networks pay attention to while discerning the objects. The attention is diffuse. The bottom row shows how the computer, using the lab’s method, pays closer attention to specific areas of the objects, increasing the reliability of the identification. Image courtesy the Geometric Media Lab at ASU.

Trustworthiness of the systems is key because DARPA is interested in defense applications, such as drone images.

Using standard benchmark data tests, Turaga’s method shows an increase of four percentage points in accuracy, from 68% to 72%

“The interesting thing is not that we pushed (the accuracy), but to say it went up, because we’re paying attention to the underlying property of the object.

“We’re actually encoding knowledge of the object shape and geometry.”

Turaga said they couldn’t have done this kind of work without a joint appointment between the schools.

“That role allows me time and space to think about these kinds of speculative questions, which would not be easy if I were only in mainstream engineering.

“It lets me step back and say, ‘Let’s think deeper about how media technologies are encoding knowledge.’”

The Geometric Media Lab is working with the Global Security Initiative at ASU on the grant project, part of which is subcontracted out to Johns Hopkins and Florida State universities. The project will last 18 months and include teams of students and faculty members. The co-principal investigators are Visar Berisha, an associate professor, and Gautam Dasarathy, assistant professor, both in the School of Electrical, Computer and Energy Engineering at ASU, and also includes student Rajhans Singh and postdoc Ankitha Shukla as key contributors.

 

A version of this story originally appeared in ASU News.