Programming in Natural Language with fuSE: Synthesizing Methods from Spoken Utterances Using Deep Natural Language Understanding
Authors: Weigelt, Sebastian, Steurer, Vanessa, Hey, Tobias and Tichy, Walter F.
Conference: 58th Annual Meeting of the Association for Computational Linguistics
Paper
Presentation (Video)
Slides
Abstract: The key to effortless end-user programming is natural language. We examine how to teach intelligent systems new functions, expressed in natural language. As a first step, we collected 3168 samples of teaching efforts in plain English. Then we built fuSE, a novel system that translates English function descriptions into code. Our approach is three-tiered and each task is evaluated separately. We first classify whether an intent to teach new functionality is present in the utterance (accuracy: 97.7% using BERT). Then we analyze the linguistic structure and construct a semantic model (accuracy: 97.6% using a BiLSTM). Finally, we synthesize the signature of the method, map the intermediate steps (instructions in the method body) to API calls and inject control structures (F1: 67.0% with information retrieval and knowledge-based methods). In an end-to-end evaluation on an unseen dataset fuSE synthesized 84.6% of the method signatures and 79.2% of the API calls correctly.
@inproceedings{weigeltProgramming2020,
title = {Programming in {{Natural Language}} with {{fuSE}}: {{Synthesizing Methods}} from {{Spoken Utterances Using Deep Natural Language Understanding}}},
shorttitle = {Programming in {{Natural Language}} with {{fuSE}}},
booktitle = {Proceedings of the 58th {{Annual Meeting}} of the {{Association}} for {{Computational Linguistics}}},
author = {Weigelt, Sebastian and Steurer, Vanessa and Hey, Tobias and Tichy, Walter F.},
year = {2020},
month = jul,
pages = {4280--4295},
publisher = {{Association for Computational Linguistics}},
address = {{Online}},
abstract = {The key to effortless end-user programming is natural language. We examine how to teach intelligent systems new functions, expressed in natural language. As a first step, we collected 3168 samples of teaching efforts in plain English. Then we built fuSE, a novel system that translates English function descriptions into code. Our approach is three-tiered and each task is evaluated separately. We first classify whether an intent to teach new functionality is present in the utterance (accuracy: 97.7\% using BERT). Then we analyze the linguistic structure and construct a semantic model (accuracy: 97.6\% using a BiLSTM). Finally, we synthesize the signature of the method, map the intermediate steps (instructions in the method body) to API calls and inject control structures (F1: 67.0\% with information retrieval and knowledge-based methods). In an end-to-end evaluation on an unseen dataset fuSE synthesized 84.6\% of the method signatures and 79.2\% of the API calls correctly.}
}