öffnet PDF -->

Modul "Natural Media of Human Communication"

Interdisciplinary insights into human cognition, language, and technology

SoSe 2014


Prof. Irene Mittelberg, Ph.D. (HumTec)
Prof. Dr. rer. nat. Klaus Willmes-von Hinckeldey (Neuropsychology, University Hospital Aachen)

 

time and venue: tuesdays, 18:00-19:30; 1050|U104 (AS)

 

Course description
This module explores connections among the human mind, language, and other media we naturally use to express ourselves and to understand the people around us. We use spoken language(s) to talk to others and to solve problems, we read our friends’ facial expressions to find out what they might think or feel, we point with our fingers when giving directions, and we are quite good at telling whether somebody is relaxed or agitated from looking at her or his body posture. Users of sign languages recruit their hands and the space around them to converse, tell elaborate stories, or create poetry.
What is common to all these uses of ‘natural media’ is that no artificial or technical medium is needed to connect and communicate. Human face-to-face interaction probably is the most fundamental way to communicate, and yet it is a complex mix of finely coordinated bodily movements, speech sounds, mimics, looks, posture, and so forth. It concerns all of us, and understanding the principles – and the beauty – of it, remains one of the big challenges of personal and scientific inquiry. Now, in the era of electronic media, another crucial task is to preserve some of the naturalness in technological and virtual environments (e.g., mobile communication, virtual reality games, etc.).
Class presentations and discussions will concern the multimodal nature of language. One of the guiding questions is how the different natural media are used in everyday communication, and what the advantages and limits of each medium seem to be. What is the evolutionary advantage of spoken language? What can gestures do that oral language cannot do as easily? And what does this tell us about human cognition and behavior? Another central interest is to see how scientists from various disciplines merge their expertise to investigate the forms and functions of natural media in different contexts. For example, what happens in people’s heads when language breaks down due to brain damage? What does it take to translate from one language into another? We will also discuss issues in designing gestural interfaces with technology for intuitive use. Along the way, students will become familiar with the methods researchers use to examine human communicative behavior, such as audio/video analysis, experimental design, and brain imaging.  

 

Language of instruction:
To give all students the opportunity to benefit from an international setting, all lectures and discussions will be held in English. During the discussion sessions, questions and answers may be translated. The English readings fort he course will be at a manageable level.


Requirements:

Participants are expected to do the assigned readings as class preparations and to actively engage in class discussions. To earn credits for a ‘Leistungsnachweis’, each student is further asked to participate in a science slam presentation on one of the topics treated during the semester. More information will be provided in class

 

 

 

 

Schedule:

 

April 15                                  Introduction; what is a science slam?          

 

April 22                                  Survival Semiotics
Understanding Natural and Cultural Signs
Prof. Irene Mittelberg, Ph.D.
(Linguistics & Cognitive Semiotics; Natural Media Lab/HumTec)

 

April 29                                  Talking heads and hands  
On the Role of Gestures in Multimodal Interaction
Dr. phil. Vito Evola
(Bonn International School of Information Technology; Natural Media Lab/HumTec)
                                              
May 06                                   Hands, Numbers, and Counting
Behavioral and Brain Activation Studies
Prof. Dr. rer. nat. Klaus Willmes-von Hinckeldey
(Neuropsychology, Uniclinic Aachen)

 

May 20                                   Visual Analysis of Human Motion for Robot Navigation
-- Individuals, Social Groups, Person-Object Interactions               
Prof. Dr. sc. techn. Bastian Leibe
(RWTH Informatics 8, Computer Vision/UMIC Research Centre)

 

June 03                                 Signed Languages: Insights into the Multimodality of Language
Influence of Language Modality on Categorization                                                     

Dr. phil. Klaudia Grote & Horst Sieprath
(RWTH Aachen Center for Sign Language and Gesture (SignGes))

 

June 17                                 Cognitive Control in Language Processing

Mathieu Declerck, M.Sc. & Simone Schäffner, M.Sc. 
(RWTH Cognitive & Experimental Psychology)

 

July 01                                   Science slam competition
           

 

This Leonardo module is a joint initiative of the
HumTec research group Natural Media/NeuroPeirce and the
RWTH Aachen Center for Sign Language and Gesture (SignGes).