Learning of Complex-Structured Tasks from Verbal Instruction

This paper presents a novel approach to robottask learning from language-based instructions, which focuseson increasing the complexity of task representations that canbe taught through verbal instruction. The major proposedcontribution is the development of a framework fordirectlymapping a complex verbal instruction to an executable taskrepresentation, from a single training experience. The methodcan handle the following types of complexities: 1) instructionsthat use conjunctions toconvey complex execution constraints(such as alternative paths of execution, sequential or non-ordering constraints, as well as hierarchical representations)and 2) instructions that use prepositions and multiple adjectivestospecify action/object parameters relevant for the task. Specificalgorithms have been developed for handlingconjunctions,adjectivesandprepositionsas well as for translating the parsedinstructions into parameterized executable task representations.The paper describes validation experiments with a PR2 hu-manoid robot learning new tasks from verbal instruction, aswell as an additional range of utterances that can be parsedinto executable controllers by the proposed system.