Modul:

Natural Media of Human Communication – Interdisciplinary insights into multimodal language, cognition, and technology

 

This Leonardo module is a joint initiative of the
Natural Media Lab (IfAAR/HumTec) and the Center for Sign Language and Gesture (SignGes).

 

 

Management:

 

Prof. Dr. phil. Irene Mittelberg (SignGes; IfAAR)
Prof. Dr. rer. nat. Klaus Willmes-von Hinckeldey (SignGes; Dept. of Neurology, University Hospital Aachen)

 

Contact Person: Sven Pollmann (sven.pollmann[@]leonardo.rwth-aachen.de)

 

Start: 10.04.2019

 

Time: Mittwoch, 18:30-20:00 Uhr

 

Place: III, Hauptgebäude, Templergraben 55, 52062 Aachen

 

Course Descpription

This module explores connections among the human mind, language, and other media we naturally use to express ourselves and to understand the people around us. We use spoken language(s) to talk to others and to solve problems, we read our friends’ facial expressions to find out what they might think or feel, we point with our fingers when giving directions, and we are quite good at telling whether somebody is relaxed or agitated from looking at her or his bodily movements. Users of sign languages recruit their hands, facial articulators and the space around them to converse, tell elaborate stories, or create poetry.


What is common to all these uses of ‘natural media’ is that no artificial or technical medium is needed to connect and communicate. Human face-to-face interaction is the most natural and fundamental way to communicate, and yet it is a complex mix of finely coordinated bodily movements, speech sounds, mimics, gaze, posture, and so forth. It concerns all of us, and understanding the principles – and the beauty – of it, remains one of the big challenges of personal and scientific inquiry. In the era of electronic media, another crucial task is to preserve some of the intuitiveness and naturalness in technological and virtual environments (e.g., mobile communication, gesture-based interfaces, serious games, etc.).


Class presentations and discussions will concern the multimodal nature of language and interaction in various settings. One of the guiding questions is how the different natural media are used in everyday communication, and what the advantages and limits of each medium seem to be. What can gestures do that spoken language cannot achieve as easily? And what does this tell us about human cognition and behavior? We will gain insights into body-based aspects of numerical cognition and in how hearing-impaired and deaf people communicate with sign language. Another central interest is to see how RWTH scholars from various disciplines draw on their expertise to investigate the forms and functions of natural media in different contexts. For example, what happens in people’s heads when language or motor skills break down due to brain damage? We will also discuss concepts and methods used to develop interactive, communicating robots. Along the way, students will become familiar with the methods and tools researchers use to examine human cognitive and communicative behavior, such as audio/video analysis, experimental design, and brain imaging.           

 

Schedule

 

April 10    Introduction; what is a science slam?   

April 17    Survival Semiotics: Understanding natural and cultural signs - Prof. Dr. Irene Mittelberg (Linguistics & Cognitive Semiotics; IfAAR & SignGes)                                  
April 24    Speech in Rooms - Prof. Dr. Michael Vorländer (Institute for Technical Acoustics)

May 8        American Sign Language: Insights from Psycholinguistics - Dr. Chris Brozdowski (Human Technology Center, SignGes, Informatics 5)

May 15      Two sides of Human-Agent-Interaction: Creating engaging artificial agents and using them as a testbed to study human communication - Prof. Dr. Astrid Rosenthal-von der Pütten

(Chair Individual and Technology; Human Technology Centre)

May 29      Hands, numbers, and counting: Behavioral and brain activation studies - Prof. Dr. Klaus Willmes-von Hinckeldey (Dept. of Neurology, University Hospital RWTH Aachen, & SignGes)

June 5       Introduction to German Sign Language and Culture in German Sign Language (DGS) - Ege Karar (accompanied by simultaneous interpreting into spoken English)
(RWTH Aachen Center for Sign Language and Gesture (SignGes))

 

June 26     Science slam competition

 

Language of instruction:
To give all students the opportunity to benefit from an international group of students (and lecturers), all lectures and discussions will be held in English. During the discussion sessions, questions and answers may be translated. The English readings to prepare for the lectures will be at a manageable level.


Requirements:
Participants are expected to do the assigned readings as class preparation and to actively engage in discussions.

- For a ‘Teilnahmenachweis’ (0 CPs), one missed class is allowed.

- To earn credits for a ‘Leistungsnachweis’ (3 CPs), each student is further asked to participate in a science slam competition and prepare a performance on one of the topics treated during the semester (in groups of 2 or 3 students).

 

More information will be provided in class.