March 31, 2011
Speaking the Same Language
With over quarter-of-a-million words in the Oxford dictionary, we sometimes take for granted the complex nature of the English language.
Cognitive science student Connor Smith has the spent the past three years attempting to take parts of our multifaceted language and simplify it into a code understood by computers.
He is deciphering how the human mind uses language to describe where objects are in relation to each other, so that he can transfer that skill into artificial intelligence.
“We’re hoping to build a foundation of artificial intelligence that can hopefully model higher level human imagination,” Smith says.
Smith first became involved in this project during his second semester of the cognitive science program when his professor, Jim Davies, requested volunteers for the Science of Imagination Lab (SOIL).
Smith’s research began with a year of teaching himself Python, a computer programming language.
From there, he worked with other students, looking at English spatial prepositions or detectors used in everyday language. While other students in the lab focused on prepositions like “right”, “left”, “under” and “over”, Smith looked into the “above/below” detector.
When we describe that an exit sign in “above” the door, it is understood that the sign is located just over the doorframe, Smith explains. But the way we communicate isn’t always that simple.
It is also important to think about the degree to which an object is above or below another object, says Smith. This takes into account the “Fuzzy Logic”, which places a person’s spatial relation decision on a gradient scale.
You get a lot more motivation to do something knowing that your work sort of matters
For example, if the sign is placed slightly left to the door, it would likely still be considered “above” the doorframe.
This implied understanding is what Smith and the SOIL team are trying to integrate into a computer algorithm.
By taking this spatial factor into account, the computer would be able to think and imagine more like a human.
Smith has also integrated this volunteer work into his final research project. After compiling 157,000 images from Google, Smith added these detectors—like above and below—to describe objects found in each picture. This should allow the computer to describe the spatial relationship between objects in the images.
“Every spatial relationship that is detected in these images will be considered fact in this artificial agent’s experience or memory,” he explains.
The lab’s project is still in its early stages, but has already been published as a poster in a Diagrams 2010 conference—an accomplishment Smith found to be very surprising and rewarding.
“You get a lot more motivation to do something like that knowing that your work sort of matters,” he says.
Smith plans on a career in cognitive science, but he says he “shopped around” before deciding on the degree that borders on science and the social sciences.
In his first year and a half at Carleton, Smith took a huge array of courses, including geography, physics, math, and political science. He eventually became intrigued with programmed language in a linguistics course.
“I realized with an undergraduate cognitive science degree, I could be doing philosophy, linguistics, psychology, neuroscience and artificial intelligence all in one, so I jumped on that,” he says.
And although Smith admits it has sometimes been a challenge to stay motivated, he says the knowledge he’s acquired in the program has been worth the challenge.
He plans to continue on with a Master’s in cognitive science, or at the very least, continue working in the science of imagination laboratory.
Share: Twitter, Facebook