A research team led by Professor Songhwai Oh, developed a robot, which can act as human, with just one sentence

- The first artificial neural network ‘Text2Action’ that is based on daily life data
 - A foundation for the more familiar robot platform


▲ Professor Songhwai Oh of Department of Electrical and Computer Engineering

 
An artificial intelligence technique that acquires the relationship between human language and behavior has been developed by domestic researchers.
 
Seoul National University College of Engineering (Dean Kookheon Char) announced on 2th that the research team led by Songhwai Oh (Hyemin Ahn, Timothy Ha, Yunho Choi, Hwiyeon Yoo) developed ‘Text2Action’ that can understand the relationship between human language and behavior.
 
When sentence describing human behavior is given to the Text2Action Network, the artificial neural network learns relation between language and behavior. After the learning is completed, the neural network structure generates a behavior corresponding to the sentence that is given by the user.
 
Artificial neural networks developed by the research team are based on the Generative Adversarial Network (GAN). The GAN consists of a pair of 'discriminators' that distinguish between real and fake data, and 'generators' that can generate data that is genuinely enough to fool the discriminator. When the artificial neural network learns in an environment where the discriminator and generator are both used, more accurate discriminator can be developed.
 
In order to teach the artificial neural network, the research team extracted 29,770 pairs of language and behavior data from YouTube videos. As a result, 'Text2Action Network' showed that it is possible to synthesize behavior data corresponding to human language more realistically based on characteristics of GAN. In addition, it was shown that the neural network can synthesize appropriate behaviors for the input of the language based on the data even when the language input is not included in the learning data.
 
Professor Oh said, "The methodology for learning the relationship between human language and behavior has been studied previously, but it is the first time to develop GAN-based artificial neural networks that learn through big-data from Youtube videos, which contain daily life data. The technology can be applied to build a more familiar robot platform since this technology can be applied to the artificial intelligence-based virtual characters and robots.
 
Meanwhile, Professor Oh, who led the research, received his Ph.D. in 2006 from the Department of Electrical and Computer Engineering at the University of California, Berkeley. From 2007 to 2009, he served as an assistant professor at the Department of Electrical and Computer Engineering at the University of California, Merced. Since 2009, he is working as an associate professor at the Department of Electrical and Computer Engineering, Seoul National University.