SNU Professor Zhang Byoung Tak’s Research Team Wins First Place at the “Visual Storytelling Challenge”



▲ Professor Zhang’s Team (From Left to Right: PhD Student Kim Taehyeong, MS Student Son Seonil, and PhD Candidate Heo Min-Oh)

 
On 12^th, SNU College of Engineering (Dean Cha Kook-Heon) revealed that Team SnuBiVtt led by Professor Zhang Byoung Tak of the Department of Computer Science and Engineering and consisting of Kim Taehyeong (PhD Course), Park Kyung-Wha (PhD Course), and Son Seonil (Master’s Course) has won first place at the “Visual Storytelling Challenge” of the “2018 North American Chapter of the Association for Computational Linguistics (NAACL)” held at New Orleans on 5^th.
 
Team SnuBiVtt presented their Video Tutoring Test (VTT) project, also an AI national strategic task of the Ministry of Science and ICT, at this competition. Professor Zhang’s team scored full marks on all 6 criteria.
 
The “Visual Storytelling Challenge” is a competition that assesses the storytelling (sentence structuring) ability of AI by reading and interpreting 5 different images in natural language. While previous contest involved mechanical evaluation of these storytelling devices, in order to provide a stricter judgment, this year’s competition switched to human evaluation that assessed on 6 different criteria.
 
Professor Zhang who has advised the team explains, “The emphasis of this competition, which evaluates the visual perception and natural language processing of devices, is to demonstrate the need of AI that can autonomously analyze video clips and human language. This can be applied to services that automatically record travelogue with just groups of photos or simply any technology that interprets our daily lives using AI.”
 
[Reference]
 
1. The 6 Criteria for Human Evaluation
1) Story Concentration
2) Coherence
3) Sharing of the Results
4) Closeness to Actual Human Storytelling
5) Application of Objects Inside the Photos
6) Adequate Amount of Details


2. Sample of Storytelling
 

▲ Actual Storytelling Samples: Enter 5 Photos and the Device Outputs 5 Related Sentences