The smart environment will augment the sensory capabilities of the robot, providing it with information that would otherwise have required very advanced perceptual and cognitive capabilities. Such purpose will be achieved through off-body sensors, since children with ASD do not like having to wear sensors, especially cumbersome ones that may agitate their sensitive skin. In order to ease the complexity of the sensing and data-fusion task, the traditional therapy room has been transformed into a smart space environment with a set of additional sensors that provide information. We will only consider small inconspicuous wearable sensors if all else fails.
A novel multi-camera system capable of capturing child movements and interactions with the robot and therapist will be designed. This system will consist on several Microsoft Kinect® cameras and high resolution cameras distributed over the room to cover expected (physical) locations of the child’s motions and facial expressions; some cameras positioned around the table between the child and the social robot to capture the interaction; and the cameras in the robot itself. This multi-camera recording system will be hidden as far as technically possible.