International Winter School on Humanoid Robot Programming

Humanoids capable of merging: a fundamental cognitive mechanism for meaning creation in the motoric, perceptual and language space

Katerina Pastra, Cognitive Systems Research Institute, Athens, Greece


Syntax has been studied and analysed mostly as a fundamental mechanism of human language. However, increasing evidence in Neuroscience points to the fact that a fundamental syntactic mechanism is shared between language and action, both of which have a hieararchical and compositional organisation; Broca's area has been suggested as the neural locus of this mechanism. In this talk, we will present theory, experiments and computational models of syntax, as a basic merging mechanism that crosses over language to the visuomotor space. We will focus on syntax in the action space and present the first formal specification of action with biological bases, the Minimalist Grammar of Action. We will demonstrate what humanoids endowed with such grammar can do. Furthemore, we will go a step further, arguing that 'syntactic' activities manifest themselves in any space that comprises 'merging' of elements into more and more complex units, such as the Semantic Association space (Semantic Memory). We will present the PRAXICON, the first ever recursive and referential semantic network, in which merging of embodied concepts into complex conceptual structures is captured as a core element of the network. We will demonstrate the importance of such network for humanoids employed in human-robot interaction, and in particular in generalisation and reasoning tasks.


Katerina Pastra is the Director of the Cognitive Systems Research Institute, and a Senior Researcher at the Institute for Language and Speech Processing, ATHENA Research Center. She has coordinated a number of European and national research and development projects, including POETICON++ (FP7-ICT), its predecessor POETICON (FP7-ICT) and a distinguishing Latsis Foundation award for research on “Image-Language Dialectics”. Her research is focused on the computational integration of language, perception and action based on experimental findings from neuroscience. Among others, she has developed the first ‘Minimalist Grammar of Action’, the PRAXICON semantic memory model, the PLT affordances network and the COSMOROE image-language semantic interplay framework.

Katerina holds a BA in Linguistics (University of Athens, Greece), an MSc in Machine Translation (UMIST, UK) and a PhD in Artificial Intelligence (University of Sheffield, U.K.) in which she explored the integration of vision and language within artificial agents engaged in everyday interaction. She has worked on information extraction and automatic text-based image/video indexing and retrieval in a number of projects. She has lectured on Human-Computer/Human-Robot Interaction and the use of cognitive and psychological methods in software engineering and has organised international workshops on image-language association with sponsorships from GOOGLE, the EU-Cognition Network and other international foundations. She is the author of a number of publications on the above topics, one of which has won a distinction by the British Computer Society. She is a Senior IEEE Member, the vice-chair of the European Network on Vision and Language Integration, and co-editor of the Language and Cognition Research Topic in Frontiersin Robotics and Artificial Intelligence.