Roger that! Learning How Laypersons Teach New Functions to Intelligent Systems
Authors: Weigelt, Sebastian, Steurer, Vanessa, Hey, Tobias and Tichy, Walter F.
Conference: IEEE 14th International Conference on Semantic Computing (ICSC), 2020
Paper
Pre-Print (OA)
Poster
Slides
Abstract: Intelligent systems are rather smart today but still limited to built-in functionality. To break through this barrier, future systems must allow users to easily adapt the system by themselves. For humans the most natural way to communicate is talking. But what if users want to extend the systems’ functionality with nothing but natural language? Then intelligent systems must understand how laypersons teach new skills. To grasp the semantics of such teaching sequences, we have defined a hierarchical classification task. On the first level, we consider the existence of a teaching intent in an utterance; on the second, we classify the distinct semantic parts of teaching sequences: declaration of a new function, specification of intermediate steps, and superfluous information. We evaluate twelve machine learning techniques with multiple configurations tailored to this task ranging from classical approaches such as naı̈ve-bayes to modern techniques such as bidirectional LSTMs and task-oriented adaptations. On the first level convolutional neural networks achieve the best accuracy (96.6%). For the second task, bidirectional LSTMs are the most accurate (98.8%). With the additional adaptations we are able to improve both classifications distinctly (up to 1.8%).
@inproceedings{weigeltRoger2020,
title = {Roger That! {{Learning How Laypersons Teach New Functions}} to {{Intelligent Systems}}},
booktitle = {2020 {{IEEE}} 14th {{International Conference}} on {{Semantic Computing}} ({{ICSC}})},
author = {Weigelt, Sebastian and Steurer, Vanessa and Hey, Tobias and Tichy, Walter F.},
year = {2020},
month = feb,
pages = {93--100},
issn = {2325-6516},
doi = {10.1109/ICSC.2020.00020},
abstract = {Intelligent systems are rather smart today but still limited to built-in functionality. To break through this barrier, future systems must allow users to easily adapt the system by themselves. For humans the most natural way to communicate is talking. But what if users want to extend the systems' functionality with nothing but natural language? Then intelligent systems must understand how laypersons teach new skills. To grasp the semantics of such teaching sequences, we have defined a hierarchical classification task. On the first level, we consider the existence of a teaching intent in an utterance; on the second, we classify the distinct semantic parts of teaching sequences: declaration of a new function, specification of intermediate steps, and superfluous information. We evaluate twelve machine learning techniques with multiple configurations tailored to this task ranging from classical approaches such as na??ve-bayes to modern techniques such as bidirectional LSTMs and task-oriented adaptations. On the first level convolutional neural networks achieve the best accuracy (96.6\%). For the second task, bidirectional LSTMs are the most accurate (98.8\%). With the additional adaptations we are able to improve both classifications distinctly (up to 1.8\%).},
keywords = {Intelligent systems,Machine learning,Neural networks,Semantics,Task analysis,Training}
}