We designed a 3-5 minute play protocol, between a trained examiner and a child aged 15-30 months. This play protocol is an interactive assessment consist of five stages, in which the examiner elicits social attention, back-and-forth interaction, and nonverbal communication from the child. These behaviors reflect key socio-communicative milestones in the first two years of life, and their diminished occurrence and qualitative difference in expression have been found to represent early markers of autism spectrum disorders.
Data
We have collected 160 sessions of 5-minute interaction from 121 children. All multimodal signals are synchronized, including:
- 2 frontal view Basler cameras (1920x1080 at 60 FPS)
- An overhead view Kinect (RGB-D) camera
- 8 side view & 3 overhead view AXIS cameras (640x480 at 30 FPS)
- An omnidirectional and a cardioid microphone, ceiling mounted
- 2 wireless lapel microphones, worn by both the child and the adult
- 4 Affectiva Q-sensors for electrodermal activity and accelerometry, worn by both the adult and the child.
Annotations
The MMDB dataset contains fine-grained annotations of behaviors, including
- Ratings of engagement and responsiveness at substage level
- Frame-level, continuous annotation of relevant child behaviors
- Include 5 activities with summary rating and 18 nonverbal communication behaviors
