Calendar- No Events
Publications
Long, Cao Hoang; Pablo, Esteban; Madeleine, Bartlett; Edward, Baxter Paul; Tony, Belpaeme; Erik, Billing; Haibin, Cai; Mark, Coeckelbergh; Cristina, Costescu; Daniel, David; Albert, De Beir; Daniel, Hernandez Garcia; James, Kennedy; Honghai, Liu; Silviu, Matu; Alexandre, Mazel; Kumar, Pandey Amit; Kathleen, Richardson; Emmanuel, Senft; Serge, Thill; de Greet, Van Perre; Bram, Vanderborght; David, Vernon; Kutoma, Wakanuma; Hui, Yu; Xiaolong, Zhou; Tom., Ziemke Robot-Enhanced Therapy: Development and Validation of a Supervised Autonomous Robotic System for Autism Spectrum Disorders Therapy Journal Article In: IEEE Robotics & Automation Magazine, 2019. @article{Cao2019c, title = {Robot-Enhanced Therapy: Development and Validation of a Supervised Autonomous Robotic System for Autism Spectrum Disorders Therapy}, author = {Cao Hoang Long and Esteban Pablo and Bartlett Madeleine and Baxter Paul Edward and Belpaeme Tony and Billing Erik and Cai Haibin and Coeckelbergh Mark and Costescu Cristina and David Daniel and De Beir Albert and Hernandez Garcia Daniel and Kennedy James and Liu Honghai and Matu Silviu and Mazel Alexandre and Pandey Amit Kumar and Richardson Kathleen and Senft Emmanuel and Thill Serge and Van de Perre Greet and Vanderborght Bram and Vernon David and Wakanuma Kutoma and Yu Hui and Zhou Xiaolong and Ziemke Tom.}, url = {http://www.dream2020.eu/wp-content/uploads/2019/05/08683968.pdf http://www.dream2020.eu/wp-content/uploads/2019/05/Robot_Enhanced_Therapy-2.pdf https://ieeexplore.ieee.org/document/8683968}, doi = {10.1109/MRA.2019.2904121 }, year = {2019}, date = {2019-04-09}, journal = {IEEE Robotics & Automation Magazine}, abstract = {Robot-assisted therapy (RAT) offers potential advantages for improving the social skills of children with autism spectrum disorders (ASDs). This article provides an overview of the developed technology and clinical results of the EC-FP7 funded Development of Robot-Enhanced therapy for children with AutisM spectrum disorders (DREAM) project, which aims to develop the next level of RAT in both clinical and technological perspectives, commonly referred to as robot-enhanced therapy (RET). Within this project, a supervised autonomous robotic system is collaboratively developed by an interdisciplinary consortium including psychotherapists, cognitive scientists, roboticists, computer scientists, and ethicists, which allows robot control to exceed classical remote control methods, e.g., Wizard of Oz (WoZ), while ensuring safe and ethical robot behaviour. Rigorous clinical studies are conducted to validate the efficacy of RET. Current results indicate that RET can obtain an equivalent performance compared to that of human standard therapy for children with ASDs. We also discuss the next steps of developing RET robotic systems.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Robot-assisted therapy (RAT) offers potential advantages for improving the social skills of children with autism spectrum disorders (ASDs). This article provides an overview of the developed technology and clinical results of the EC-FP7 funded Development of Robot-Enhanced therapy for children with AutisM spectrum disorders (DREAM) project, which aims to develop the next level of RAT in both clinical and technological perspectives, commonly referred to as robot-enhanced therapy (RET). Within this project, a supervised autonomous robotic system is collaboratively developed by an interdisciplinary consortium including psychotherapists, cognitive scientists, roboticists, computer scientists, and ethicists, which allows robot control to exceed classical remote control methods, e.g., Wizard of Oz (WoZ), while ensuring safe and ethical robot behaviour. Rigorous clinical studies are conducted to validate the efficacy of RET. Current results indicate that RET can obtain an equivalent performance compared to that of human standard therapy for children with ASDs. We also discuss the next steps of developing RET robotic systems. |
Mazel, A; Matu, S DREAM Lite: Simplifying Robot Assisted Therapy for ASD Workshop The second workshop of social robots in therapy and care. In conjunction with the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2019). Daegu, Korea. March 11-14. , 2019. @workshop{Mazel2019, title = {DREAM Lite: Simplifying Robot Assisted Therapy for ASD}, author = {A. Mazel and S. Matu}, url = {http://www.dream2020.eu/wp-content/uploads/2019/03/Mazel-and-Matu-2019-DREAM-Lite-Simplifying-Robot-Assisted-Therapy-for-ASD.pdf}, year = {2019}, date = {2019-03-14}, booktitle = {The second workshop of social robots in therapy and care. In conjunction with the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2019). Daegu, Korea. March 11-14. }, abstract = {Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD). The DREAM project explores how to deliver effective RAT interventions to ASD children within a supervised-autonomy framework for controlling the robotic agent, which could decrease the burden on the clinicians delivering such interventions. In this paper we describe how to use in real life settings the experimental protocols that were developed and extensively tested in the DREAM Project, as well as their deployment and validation in a new ecological study conducted by clinicians.}, keywords = {}, pubstate = {published}, tppubtype = {workshop} } Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD). The DREAM project explores how to deliver effective RAT interventions to ASD children within a supervised-autonomy framework for controlling the robotic agent, which could decrease the burden on the clinicians delivering such interventions. In this paper we describe how to use in real life settings the experimental protocols that were developed and extensively tested in the DREAM Project, as well as their deployment and validation in a new ecological study conducted by clinicians. |
Mazel, A; Matu, S DREAM Lite: Simplifying Robot Assisted Therapy for ASD Workshop The second workshop of social robots in therapy and care. In conjunction with the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2019). Daegu, Korea. March 11-14. , 2019. @workshop{Mazel2019b, title = {DREAM Lite: Simplifying Robot Assisted Therapy for ASD}, author = {A. Mazel and S. Matu}, url = {http://www.dream2020.eu/wp-content/uploads/2019/03/Mazel-and-Matu-2019-DREAM-Lite-Simplifying-Robot-Assisted-Therapy-for-ASD.pdf}, year = {2019}, date = {2019-03-14}, booktitle = {The second workshop of social robots in therapy and care. In conjunction with the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2019). Daegu, Korea. March 11-14. }, abstract = {Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD). The DREAM project explores how to deliver effective RAT interventions to ASD children within a supervised-autonomy framework for controlling the robotic agent, which could decrease the burden on the clinicians delivering such interventions. In this paper we describe how to use in real life settings the experimental protocols that were developed and extensively tested in the DREAM Project, as well as their deployment and validation in a new ecological study conducted by clinicians.}, keywords = {}, pubstate = {published}, tppubtype = {workshop} } Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD). The DREAM project explores how to deliver effective RAT interventions to ASD children within a supervised-autonomy framework for controlling the robotic agent, which could decrease the burden on the clinicians delivering such interventions. In this paper we describe how to use in real life settings the experimental protocols that were developed and extensively tested in the DREAM Project, as well as their deployment and validation in a new ecological study conducted by clinicians. |
García, D H; Esteban, P G; Lee, H R; Romeo, M; Senft, E; Billing, E Social Robots in Therapy and Care Inproceedings In: 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 669-670, 2019, ISSN: 2167-2148. @inproceedings{García2019, title = {Social Robots in Therapy and Care}, author = {D H García and P G Esteban and H R Lee and M Romeo and E Senft and E Billing}, url = {http://his.diva-portal.org/smash/record.jsf?pid=diva2%3A1290716&dswid=-6830}, doi = {10.1109/HRI.2019.8673243}, issn = {2167-2148}, year = {2019}, date = {2019-03-01}, booktitle = {2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI)}, pages = {669-670}, abstract = {The Social Robots in Therapy workshop series aims at advancing research topics related to the use of robots in the contexts of Social Care and Robot-Assisted Therapy (RAT). Robots in social care and therapy have been a long time promise in HRI as they have the opportunity to improve patients life significantly. Multiple challenges have to be addressed for this, such as building platforms that work in proximity with patients, therapists and health-care professionals; understanding user needs; developing adaptive and autonomous robot interactions; and addressing ethical questions regarding the use of robots with a vulnerable population. The full-day workshop follows last year's edition which centered on how social robots can improve health-care interventions, how increasing the degree of autonomy of the robots might affect therapies, and how to overcome the ethical challenges inherent to the use of robot assisted technologies. This 2nd edition of the workshop will be focused on the importance of equipping social robots with socio-emotional intelligence and the ability to perform meaningful and personalized interactions. This workshop aims to bring together researchers and industry experts in the fields of Human-Robot Interaction, Machine Learning and Robots in Health and Social Care. It will be an opportunity for all to share and discuss ideas, strategies and findings to guide the design and development of robot-assisted systems for therapy and social care implementations that can provide personalize, natural, engaging and autonomous interactions with patients (and health-care providers).}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The Social Robots in Therapy workshop series aims at advancing research topics related to the use of robots in the contexts of Social Care and Robot-Assisted Therapy (RAT). Robots in social care and therapy have been a long time promise in HRI as they have the opportunity to improve patients life significantly. Multiple challenges have to be addressed for this, such as building platforms that work in proximity with patients, therapists and health-care professionals; understanding user needs; developing adaptive and autonomous robot interactions; and addressing ethical questions regarding the use of robots with a vulnerable population. The full-day workshop follows last year's edition which centered on how social robots can improve health-care interventions, how increasing the degree of autonomy of the robots might affect therapies, and how to overcome the ethical challenges inherent to the use of robot assisted technologies. This 2nd edition of the workshop will be focused on the importance of equipping social robots with socio-emotional intelligence and the ability to perform meaningful and personalized interactions. This workshop aims to bring together researchers and industry experts in the fields of Human-Robot Interaction, Machine Learning and Robots in Health and Social Care. It will be an opportunity for all to share and discuss ideas, strategies and findings to guide the design and development of robot-assisted systems for therapy and social care implementations that can provide personalize, natural, engaging and autonomous interactions with patients (and health-care providers). |
Van de Perre, Greet; De Beir, Albert; Cao, Hoang-Long; Esteban, Pablo Gómez; Lefeber, Dirk; Vanderborght, Bram Studying Design Aspects for Social Robots Using a Generic Gesture Method Journal Article In: International Journal of Social Robotics, 2019, ISSN: 1875-4805. @article{Van de Perre2019, title = {Studying Design Aspects for Social Robots Using a Generic Gesture Method}, author = {Greet Van de Perre and Albert De Beir and Hoang-Long Cao and Pablo Gómez Esteban and Dirk Lefeber and Bram Vanderborght}, url = {https://doi.org/10.1007/s12369-019-00518-x http://www.dream2020.eu/wp-content/uploads/2019/05/Design-Aspects.pdf}, doi = {10.1007/s12369-019-00518-x}, issn = {1875-4805}, year = {2019}, date = {2019-02-06}, journal = {International Journal of Social Robotics}, abstract = {Since social robots are aimed to interact and communicate with humans in a natural way and operate in our daily environment, their design should be adapted to this. Although many social robots are for that reason more or less based on the human model, the exact morphology of the robot depends on their specific application. In this paper, we propose a novel methodology to study the influence of different design aspects, based on a generic gesture method. The gesture method was developed to overcome the difficulties in transferring gestures to different robots, providing a solution for the correspondence problem. A small set of morphological information, inputted by the user, is used to evaluate the generic framework of the software at runtime. Therefore, gestures can be calculated fast and easy for a desired robot configuration. By generating a set of gestures for different morphologies, the importance of specific joints and their influence on a series of postures and gestures can be studied. The gesture method proves its usefulness in the design process of social robots by providing an impression of the necessary amount of complexity needed for a specified task, and can give interesting insights in the required joint angle range. In this paper, this design methodology is illustrated by using the virtual model of the robot Probo.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Since social robots are aimed to interact and communicate with humans in a natural way and operate in our daily environment, their design should be adapted to this. Although many social robots are for that reason more or less based on the human model, the exact morphology of the robot depends on their specific application. In this paper, we propose a novel methodology to study the influence of different design aspects, based on a generic gesture method. The gesture method was developed to overcome the difficulties in transferring gestures to different robots, providing a solution for the correspondence problem. A small set of morphological information, inputted by the user, is used to evaluate the generic framework of the software at runtime. Therefore, gestures can be calculated fast and easy for a desired robot configuration. By generating a set of gestures for different morphologies, the importance of specific joints and their influence on a series of postures and gestures can be studied. The gesture method proves its usefulness in the design process of social robots by providing an impression of the necessary amount of complexity needed for a specified task, and can give interesting insights in the required joint angle range. In this paper, this design methodology is illustrated by using the virtual model of the robot Probo. |
Cai, H; Fang, Y; Ju, Z; Costescu, C; David, D; Billing, E; Ziemke, T; Thill, S; Belpaeme, T; Vanderborght, B; Vernon, D; Richardson, K; Liu, H Sensing-Enhanced Therapy System for Assessing Children With Autism Spectrum Disorders: A Feasibility Study Journal Article In: IEEE Sensors Journal, 19 (4), pp. 1508-1518, 2019, ISSN: 1530-437X. @article{Cai2019, title = {Sensing-Enhanced Therapy System for Assessing Children With Autism Spectrum Disorders: A Feasibility Study}, author = {H Cai and Y Fang and Z Ju and C Costescu and D David and E Billing and T Ziemke and S Thill and T Belpaeme and B Vanderborght and D Vernon and K Richardson and H Liu}, url = {https://ieeexplore.ieee.org/abstract/document/8502783}, doi = {10.1109/JSEN.2018.2877662}, issn = {1530-437X}, year = {2019}, date = {2019-02-01}, journal = {IEEE Sensors Journal}, volume = {19}, number = {4}, pages = {1508-1518}, abstract = {It is evident that recently reported robot-assisted therapy systems for assessment of children with autism spectrum disorder (ASD) lack autonomous interaction abilities and require significant human resources. This paper proposes a sensing system that automatically extracts and fuses sensory features, such as body motion features, facial expressions, and gaze features, further assessing the children behaviors by mapping them to therapist-specified behavioral classes. Experimental results show that the developed system has a capability of interpreting characteristic data of children with ASD, thus has the potential to increase the autonomy of robots under the supervision of a therapist and enhance the quality of the digital description of children with ASD. The research outcomes pave the way to a feasible machine-assisted system for their behavior assessment.}, keywords = {}, pubstate = {published}, tppubtype = {article} } It is evident that recently reported robot-assisted therapy systems for assessment of children with autism spectrum disorder (ASD) lack autonomous interaction abilities and require significant human resources. This paper proposes a sensing system that automatically extracts and fuses sensory features, such as body motion features, facial expressions, and gaze features, further assessing the children behaviors by mapping them to therapist-specified behavioral classes. Experimental results show that the developed system has a capability of interpreting characteristic data of children with ASD, thus has the potential to increase the autonomy of robots under the supervision of a therapist and enhance the quality of the digital description of children with ASD. The research outcomes pave the way to a feasible machine-assisted system for their behavior assessment. |
Li, Jiahan; Fang, Yinfeng; Huang, Yongan; Li, Gongfa; Ju, Zhaojie; Liu, Honghai Towards Active Muscle Pattern Analysis for Dynamic Hand Motions via sEMG Inproceedings In: Lotfi, Ahmad; Bouchachia, Hamid; Gegov, Alexander; Langensiepen, Caroline; McGinnity, Martin (Ed.): Advances in Computational Intelligence Systems, pp. 372–382, Springer International Publishing, Cham, 2019, ISBN: 978-3-319-97982-3. @inproceedings{Li2019, title = {Towards Active Muscle Pattern Analysis for Dynamic Hand Motions via sEMG}, author = {Jiahan Li and Yinfeng Fang and Yongan Huang and Gongfa Li and Zhaojie Ju and Honghai Liu}, editor = {Ahmad Lotfi and Hamid Bouchachia and Alexander Gegov and Caroline Langensiepen and Martin McGinnity}, isbn = {978-3-319-97982-3}, year = {2019}, date = {2019-01-01}, booktitle = {Advances in Computational Intelligence Systems}, pages = {372--382}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {Surface Electromyographys (sEMG) as a widespread human-computer interaction method can reflect the activity of human muscles. When the human forearm finishes different hand motions, there will be strong sEMG signals in different regions of the skin surface. This paper investigates the mapping relationship between sEMG signal patterns and the dynamic hand motions. Four different hand motions are studied based on the extracted signal with mean absolute value (MAV) features and the shape-preserving piecewise cubic interpolation method. In the experiments, a 16-channel electrode sleeve is used to collect 9-subject EMG signals. According to the distribution of electrodes in the forearm, the forearm surface is divided into 8 different muscle regions. The preliminary experimental results show that different hand motions can cause different distribution of sEMG signals in different regions. It confirms that different subjects show similar patterns for the same motions. The experimental results can be applied as new sEMG features with a higher computational speed.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Surface Electromyographys (sEMG) as a widespread human-computer interaction method can reflect the activity of human muscles. When the human forearm finishes different hand motions, there will be strong sEMG signals in different regions of the skin surface. This paper investigates the mapping relationship between sEMG signal patterns and the dynamic hand motions. Four different hand motions are studied based on the extracted signal with mean absolute value (MAV) features and the shape-preserving piecewise cubic interpolation method. In the experiments, a 16-channel electrode sleeve is used to collect 9-subject EMG signals. According to the distribution of electrodes in the forearm, the forearm surface is divided into 8 different muscle regions. The preliminary experimental results show that different hand motions can cause different distribution of sEMG signals in different regions. It confirms that different subjects show similar patterns for the same motions. The experimental results can be applied as new sEMG features with a higher computational speed. |
Billing, Erik; Rosén, Julia; Lindblom, Jessica Expectations of robot technology in welfare Inproceedings In: The 14th ACM/IEEE International Conference onHuman-Robot Interaction, 2019. @inproceedings{billing2019expectations, title = {Expectations of robot technology in welfare}, author = {Erik Billing and Julia Rosén and Jessica Lindblom}, url = {http://www.dream2020.eu/wp-content/uploads/2019/03/Billing-et-al.-2019-Expectations_of_robot_technology_in_welfare.pdf}, year = {2019}, date = {2019-01-01}, booktitle = {The 14th ACM/IEEE International Conference onHuman-Robot Interaction}, abstract = {We report findings from a survey on expectations of robot technology in welfare, within the coming 20 years. 34 assistant nurses answered a questionnaire on which tasks, from their daily work, that they believe robots can perform, already today or in the near future. Additionally, the Negative attitudes toward robots scale (NARS) was used to estimate participants’ attitudes towards robots in general. Results reveal high expecta- tions of robots, where at least half of the participants answered Already today or Within 10 years to 9 out of 10 investigated tasks. Participants were also fairly positive towards robots, reporting low scores on NARS. The obtained results can be interpreted as a serious over-estimation of what robots will be able to do in the near future, but also large varieties in participants’ interpretation of what robots are. We identify challenges in communicating both excitement towards a technology in rapid development and realistic limitations of this technology.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } We report findings from a survey on expectations of robot technology in welfare, within the coming 20 years. 34 assistant nurses answered a questionnaire on which tasks, from their daily work, that they believe robots can perform, already today or in the near future. Additionally, the Negative attitudes toward robots scale (NARS) was used to estimate participants’ attitudes towards robots in general. Results reveal high expecta- tions of robots, where at least half of the participants answered Already today or Within 10 years to 9 out of 10 investigated tasks. Participants were also fairly positive towards robots, reporting low scores on NARS. The obtained results can be interpreted as a serious over-estimation of what robots will be able to do in the near future, but also large varieties in participants’ interpretation of what robots are. We identify challenges in communicating both excitement towards a technology in rapid development and realistic limitations of this technology. |
Cao, Hoang-Long; Jensen, Lars Christian; Nghiem, Xuan Nhan; Vu, Huong; Beir, Albert De; Esteban, Pablo Gomez; de Perre, Greet Van; Lefeber, Dirk; Vanderborght, Bram DualKeepon: a human--robot interaction testbed to study linguistic features of speech Journal Article In: Intelligent Service Robotics, 12 (1), pp. 45–54, 2019, ISSN: 1861-2784. @article{Cao2019b, title = {DualKeepon: a human--robot interaction testbed to study linguistic features of speech}, author = {Hoang-Long Cao and Lars Christian Jensen and Xuan Nhan Nghiem and Huong Vu and Albert De Beir and Pablo Gomez Esteban and Greet Van de Perre and Dirk Lefeber and Bram Vanderborght}, url = {https://doi.org/10.1007/s11370-018-0266-9 http://www.dream2020.eu/wp-content/uploads/2019/05/DualKeepon-1.pdf}, doi = {10.1007/s11370-018-0266-9}, issn = {1861-2784}, year = {2019}, date = {2019-01-01}, journal = {Intelligent Service Robotics}, volume = {12}, number = {1}, pages = {45--54}, abstract = {In this paper, we present a novel dual-robot testbed called DualKeepon for carrying out pairwise comparisons of linguistic features of speech in human--robot interactions. Our solution, using a modified version of the MyKeepon robotic toy developed by Beatbots, is a portable open-source system for researchers to set up experiments quickly, and in an intuitive way. We provide an online tutorial with all required materials to replicate the system. We present two human--robot interaction studies to demonstrate the testbed. The first study investigates the perception of robots using filled pauses. The second study investigates how social roles, realized by different prosodic and lexical speaking profiles, affect trust. Results show that the proposed testbed is a helpful tool for linguistic studies. In addition to the basic setup, advanced users of the system have the ability to connect the system to different robot platforms, i.e., NAO, Pepper.}, keywords = {}, pubstate = {published}, tppubtype = {article} } In this paper, we present a novel dual-robot testbed called DualKeepon for carrying out pairwise comparisons of linguistic features of speech in human--robot interactions. Our solution, using a modified version of the MyKeepon robotic toy developed by Beatbots, is a portable open-source system for researchers to set up experiments quickly, and in an intuitive way. We provide an online tutorial with all required materials to replicate the system. We present two human--robot interaction studies to demonstrate the testbed. The first study investigates the perception of robots using filled pauses. The second study investigates how social roles, realized by different prosodic and lexical speaking profiles, affect trust. Results show that the proposed testbed is a helpful tool for linguistic studies. In addition to the basic setup, advanced users of the system have the ability to connect the system to different robot platforms, i.e., NAO, Pepper. |
Xu, Wan-Fen; Fang, Yinfeng; Zhang, Gongyue; Ju, Zhaojie; Li, Gongfa; Liu, Honghai Surface Emg channel selection for thumb motion classificationsignal Inproceedings In: 2018 International Conference on Machine Learning and Cybernetics, pp. 662–666, IEEE, 2018, ISBN: 978-1-5386-5215-2. @inproceedings{Xu2018, title = {Surface Emg channel selection for thumb motion classificationsignal}, author = {Wan-Fen Xu and Yinfeng Fang and Gongyue Zhang and Zhaojie Ju and Gongfa Li and Honghai Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2019/05/postprint_2_.pdf}, doi = {10.1109/ICMLC.2018.8526988}, isbn = {978-1-5386-5215-2}, year = {2018}, date = {2018-12-12}, booktitle = {2018 International Conference on Machine Learning and Cybernetics}, pages = {662--666}, publisher = {IEEE}, series = {IEEE ICMLC Proceeding Series}, abstract = {Simplifying the interaction between humans and computers has become intensively important. Handgesture contains large amount information that can facilitate the communication among humans, and it can also be utilized to interact with external devices. As a result, this study aims to decode the different hand gestures from sEMG signal. The thumb plays the most important role in hand-based object manipulation, such as touch screen control for smart phones, for which many thumb-based hand involved. Therefore, studying the relationship between EMG signals and the thumb movement has certain value for the future human-computer interaction. In this paper, we focus on the identification of electrode position. The signal from which is not so related to the thumb movement, and thus these sEMG channels can be reduced. In the experiment, a 16-channels sleeve is utilized and a variance-based method was proposedto identify the redundant channels. It is found that there exist three common redundant channelsacross nine subjects., and all located at the inside of the forearm.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Simplifying the interaction between humans and computers has become intensively important. Handgesture contains large amount information that can facilitate the communication among humans, and it can also be utilized to interact with external devices. As a result, this study aims to decode the different hand gestures from sEMG signal. The thumb plays the most important role in hand-based object manipulation, such as touch screen control for smart phones, for which many thumb-based hand involved. Therefore, studying the relationship between EMG signals and the thumb movement has certain value for the future human-computer interaction. In this paper, we focus on the identification of electrode position. The signal from which is not so related to the thumb movement, and thus these sEMG channels can be reduced. In the experiment, a 16-channels sleeve is utilized and a variance-based method was proposedto identify the redundant channels. It is found that there exist three common redundant channelsacross nine subjects., and all located at the inside of the forearm. |
Li, Jinrong; Fang, Yinfeng; Ning, Yong; Jie, Jing; Tan, Ping; Liu, Honghai Relative Confidence Based Information Fusion For Semg-Based Pattern Recognition Inproceedings In: pp. 625-630, 2018. @inproceedings{Li2018, title = {Relative Confidence Based Information Fusion For Semg-Based Pattern Recognition}, author = {Jinrong Li and Yinfeng Fang and Yong Ning and Jing Jie and Ping Tan and Honghai Liu}, url = {https://www.researchgate.net/publication/328903270_Relative_Confidence_Based_Information_Fusion_For_Semg-Based_Pattern_Recognition}, doi = {10.1109/ICMLC.2018.8526931}, year = {2018}, date = {2018-11-01}, pages = {625-630}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Liu, Bangli; Ju, Zhaojie; Kubota, Naoyuki; Liu, Honghai Online action recognition based on skeleton motion distribution Inproceedings In: Proceedings of the British Machine Vision Conference, British Machine Vision Association, 2018. @inproceedings{Liu2018, title = {Online action recognition based on skeleton motion distribution}, author = {Bangli Liu and Zhaojie Ju and Naoyuki Kubota and Honghai Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/0007.pdf}, year = {2018}, date = {2018-09-06}, booktitle = {Proceedings of the British Machine Vision Conference}, publisher = {British Machine Vision Association}, abstract = {Online action recognition which aims to jointly detect and recognize actions from video streams, is an essential step towards a comprehensive understanding of human behavior. However, it is challenging to accurately locate and recognize the occurrence of actions from noisy data streams. This paper proposes a skeleton motion distribution based method for effective online action recognition. Specifically, an adaptive density estimation function is built to calculate the density distribution of skeleton movements. Observing that each action has a unique motion distribution, we detect the occurrence of actions by identifying the transition of the motion distribution in a video stream. Once the starting point of an action is detected, a snippet-based classifier is proposed for online action recognition, which continuously identifies the most likely action class. Experimental results demonstrate that our method outperforms the state-of-the-art methods in terms of both detection accuracy and recognition precision.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Online action recognition which aims to jointly detect and recognize actions from video streams, is an essential step towards a comprehensive understanding of human behavior. However, it is challenging to accurately locate and recognize the occurrence of actions from noisy data streams. This paper proposes a skeleton motion distribution based method for effective online action recognition. Specifically, an adaptive density estimation function is built to calculate the density distribution of skeleton movements. Observing that each action has a unique motion distribution, we detect the occurrence of actions by identifying the transition of the motion distribution in a video stream. Once the starting point of an action is detected, a snippet-based classifier is proposed for online action recognition, which continuously identifies the most likely action class. Experimental results demonstrate that our method outperforms the state-of-the-art methods in terms of both detection accuracy and recognition precision. |
Gao, Dongxu; Ju, Zhaojie; Cao, Jiangtao; Liu, Honghai A novel object tracking method based on a mixture model Journal Article In: International Journal of Intelligent Robotics and Applications, 2 (3), pp. 361–371, 2018, ISSN: 2366-598X. @article{Gao2018b, title = {A novel object tracking method based on a mixture model}, author = {Dongxu Gao and Zhaojie Ju and Jiangtao Cao and Honghai Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/Gao2018_Article_ANovelObjectTrackingMethodBase.pdf}, doi = {10.1007/s41315-018-0062-x}, issn = {2366-598X}, year = {2018}, date = {2018-09-01}, journal = {International Journal of Intelligent Robotics and Applications}, volume = {2}, number = {3}, pages = {361--371}, abstract = {Object tracking has been applied in many fields such as intelligent surveillance and computer vision. Although much progress has been made, there are still many puzzles which pose a huge challenge to object tracking. Currently, the problems are mainly caused by occlusion, similar object appearance and background clutters. A novel method based on a mixture model was proposed for solving these issues. The mixture model was integrated into a Bayes framework with the combination of locally dense contexts feature and the fundamental image information (i.e. the relationship between the object and its surrounding regions). This is because that the tracking problem can be seen as a prediction question, which can be solved using the Bayes method. In addition, both scale variations and templet updating are considered to assure the effectiveness of the proposed algorithm. Furthermore, the Fourier Transform (FT) is used when solving the Bayes equation to make the algorithm run in a real-time system. Therefore, the MMOT (Mixture model for object tracking) can run faster and perform better than existing algorithms on some challenging images sequences in terms of accuracy, quickness and robustness.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Object tracking has been applied in many fields such as intelligent surveillance and computer vision. Although much progress has been made, there are still many puzzles which pose a huge challenge to object tracking. Currently, the problems are mainly caused by occlusion, similar object appearance and background clutters. A novel method based on a mixture model was proposed for solving these issues. The mixture model was integrated into a Bayes framework with the combination of locally dense contexts feature and the fundamental image information (i.e. the relationship between the object and its surrounding regions). This is because that the tracking problem can be seen as a prediction question, which can be solved using the Bayes method. In addition, both scale variations and templet updating are considered to assure the effectiveness of the proposed algorithm. Furthermore, the Fourier Transform (FT) is used when solving the Bayes equation to make the algorithm run in a real-time system. Therefore, the MMOT (Mixture model for object tracking) can run faster and perform better than existing algorithms on some challenging images sequences in terms of accuracy, quickness and robustness. |
Ogenyi, Uchenna Emeoha; Zhang, Gongyue; Yang, Chenguang; Ju, Zhaojie; Liu, Honghai An Intuitive Robot Learning from Human Demonstration Inproceedings In: Chen, Zhiyong; Mendes, Alexandre; Yan, Yamin; Chen, Shifeng (Ed.): Intelligent Robotics and Applications, pp. 176–185, Springer International Publishing, Cham, 2018, ISBN: 978-3-319-97586-3. @inproceedings{10.1007/978-3-319-97586-3_16, title = {An Intuitive Robot Learning from Human Demonstration}, author = {Uchenna Emeoha Ogenyi and Gongyue Zhang and Chenguang Yang and Zhaojie Ju and Honghai Liu}, editor = {Zhiyong Chen and Alexandre Mendes and Yamin Yan and Shifeng Chen}, isbn = {978-3-319-97586-3}, year = {2018}, date = {2018-08-03}, booktitle = {Intelligent Robotics and Applications}, pages = {176--185}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {This paper presents a new way to teach a robot certain motions remotely from human demonstrator. The human and robot interface is built using a Kinect sensor which is connected directly to a remote computer that runs on processing software. The Cartesian coordinates are extracted, converted into joint angles and sent to the workstation for the control of the Sawyer robot. Kinesthetic teaching was used to correct the reproduced demonstrations while only valid resolved joint angles are recorded to ensure consistence in the sent data. The recorded dataset is encoded using GMM while GMR was employed to extract and reproduce generalised trajectory with respect to the associated time-step. To evaluate the proposed approach, an experiment for a robot to follow a human arm motion was performed. This proposed approach could help non-expert users to teach a robot how to perform assembling task in more human like ways.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } This paper presents a new way to teach a robot certain motions remotely from human demonstrator. The human and robot interface is built using a Kinect sensor which is connected directly to a remote computer that runs on processing software. The Cartesian coordinates are extracted, converted into joint angles and sent to the workstation for the control of the Sawyer robot. Kinesthetic teaching was used to correct the reproduced demonstrations while only valid resolved joint angles are recorded to ensure consistence in the sent data. The recorded dataset is encoded using GMM while GMR was employed to extract and reproduce generalised trajectory with respect to the associated time-step. To evaluate the proposed approach, an experiment for a robot to follow a human arm motion was performed. This proposed approach could help non-expert users to teach a robot how to perform assembling task in more human like ways. |
Dai, Lihong; Liu, Jinguo; Ju, Zhaojie; Liu, Yuwang Real-Time HALCON-Based Pose Measurement System for an Astronaut Assistant Robot Inproceedings In: Chen, Zhiyong; Mendes, Alexandre; Yan, Yamin; Chen, Shifeng (Ed.): Intelligent Robotics and Applications, pp. 366–378, Springer International Publishing, Cham, 2018, ISBN: 978-3-319-97586-3. @inproceedings{10.1007/978-3-319-97586-3_33, title = {Real-Time HALCON-Based Pose Measurement System for an Astronaut Assistant Robot}, author = {Lihong Dai and Jinguo Liu and Zhaojie Ju and Yuwang Liu}, editor = {Zhiyong Chen and Alexandre Mendes and Yamin Yan and Shifeng Chen}, isbn = {978-3-319-97586-3}, year = {2018}, date = {2018-08-03}, booktitle = {Intelligent Robotics and Applications}, pages = {366--378}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {The manned space program in China has entered the space station stage. Space astronauts normally have very limited time for certain tasks, so in order to free space astronauts from some repetitive routine tasks, various robots in space appear. Free-flying robots in space can also provide conditions for various scientific experiments with their unique microgravity advantages. The measurement of their positions and attitudes is the premise of their autonomous flight and remote operations. So, the pose measurements system of the Astronaut Assistant Robot is designed in this paper. With the camera calibrated, the image is acquired and processed in real time, and the robot is tracked in real time. In the end poses are estimated by virtue of PnP algorithm. The pose measurements system is modeled and simulated, with an automatic global threshold method to improve the reliability of the system. In addition, a series of measures are adopted to improve its efficiency with a satisfactory accuracy. The experimental results show that the measurement system meets the requirements of both effectiveness and efficiency.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The manned space program in China has entered the space station stage. Space astronauts normally have very limited time for certain tasks, so in order to free space astronauts from some repetitive routine tasks, various robots in space appear. Free-flying robots in space can also provide conditions for various scientific experiments with their unique microgravity advantages. The measurement of their positions and attitudes is the premise of their autonomous flight and remote operations. So, the pose measurements system of the Astronaut Assistant Robot is designed in this paper. With the camera calibrated, the image is acquired and processed in real time, and the robot is tracked in real time. In the end poses are estimated by virtue of PnP algorithm. The pose measurements system is modeled and simulated, with an automatic global threshold method to improve the reliability of the system. In addition, a series of measures are adopted to improve its efficiency with a satisfactory accuracy. The experimental results show that the measurement system meets the requirements of both effectiveness and efficiency. |
Zhou, X; Li, J; Chen, S; Cai, H; Liu, H Multiple Perspective Object Tracking via Context-Aware Correlation Filter Journal Article In: IEEE Access, 6 , pp. 43262-43273, 2018, ISSN: 2169-3536. @article{Zhou2018, title = {Multiple Perspective Object Tracking via Context-Aware Correlation Filter}, author = {X Zhou and J Li and S Chen and H Cai and H Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2019/05/08423623.pdf https://ieeexplore.ieee.org/document/8423623}, doi = {10.1109/ACCESS.2018.2861824}, issn = {2169-3536}, year = {2018}, date = {2018-07-31}, journal = {IEEE Access}, volume = {6}, pages = {43262-43273}, abstract = {Correlation filter (CF)-based tracking methods have demonstrated excellent performance owing to their dense sample strategy and computational efficiency. However, the CF tracker suffers from several drawbacks. First, the training samples are generated by a circulant rigid translation from a fixed viewpoint, which results in less robustness against target appearance variation. Second, the CF tracker derives its optimal solution based on an image patch centered at the previous target position without considering context information, which is prone to suboptimal solutions. Finally, the CF-based tracker cannot handle model degradation resulting from false updating and error accumulation. In this paper, we propose a new tracking method based on two calibratedKinectsensors. We exploit target appearance from two perspectives and the background context to reformulate a CF tracker that is robust to target appearance variation in the tracking process. Meanwhile, our tracker can maximize the margin between the target and background in a unified CF framework. To prevent tracking model degradation resulting from false updating, we propose an adaptive model update strategy by exploiting the response distribution and prior tracking result. Extensive experimental results demonstrate the effectiveness and robustness of the proposed method against the state-of-the-art tracking methods.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Correlation filter (CF)-based tracking methods have demonstrated excellent performance owing to their dense sample strategy and computational efficiency. However, the CF tracker suffers from several drawbacks. First, the training samples are generated by a circulant rigid translation from a fixed viewpoint, which results in less robustness against target appearance variation. Second, the CF tracker derives its optimal solution based on an image patch centered at the previous target position without considering context information, which is prone to suboptimal solutions. Finally, the CF-based tracker cannot handle model degradation resulting from false updating and error accumulation. In this paper, we propose a new tracking method based on two calibratedKinectsensors. We exploit target appearance from two perspectives and the background context to reformulate a CF tracker that is robust to target appearance variation in the tracking process. Meanwhile, our tracker can maximize the margin between the target and background in a unified CF framework. To prevent tracking model degradation resulting from false updating, we propose an adaptive model update strategy by exploiting the response distribution and prior tracking result. Extensive experimental results demonstrate the effectiveness and robustness of the proposed method against the state-of-the-art tracking methods. |
Cai, Haibin; Liu, Bangli; Ju, Zhaojie; Thill, Serge; Belpaeme, Tony; Vanderborght, Bram; Liu, Honghai Accurate Eye Center Localization via Hierarchical Adaptive Convolution Conference British Machine Vision Conference (BMVC), 2018. @conference{Cai2018, title = {Accurate Eye Center Localization via Hierarchical Adaptive Convolution}, author = {Haibin Cai and Bangli Liu and Zhaojie Ju and Serge Thill and Tony Belpaeme and Bram Vanderborght and Honghai Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/1112.pdf}, year = {2018}, date = {2018-07-01}, booktitle = {British Machine Vision Conference (BMVC)}, abstract = {importantbiological properties, which indicates human’s visual focus of attention. However,accurate eye center localization still remains challenging due to the high degree appearancevariation caused by different kinds of viewing angles, illumination conditions, occlusionsand head pose. This paper proposes a hierarchical adaptive convolution method(HAC) to localize the eye center accurately while consuming low computational cost.It mainly utilizes the dramatic illumination changes between the iris and sclera. Morespecifically, novel hierarchical kernels are designed to convolute the eye images and adifferential operation is applied on the adjacent convolution results to generate various responsemaps. The final eye center is localized by searching the maximum response valueamong the response maps. Experimental results on several publicly available datasetsdemonstrate that HAC outperforms the start-of-the-art methods by a large margin. Thecode is made publicly available at https://github.com/myopengit/HAC}, keywords = {}, pubstate = {published}, tppubtype = {conference} } importantbiological properties, which indicates human’s visual focus of attention. However,accurate eye center localization still remains challenging due to the high degree appearancevariation caused by different kinds of viewing angles, illumination conditions, occlusionsand head pose. This paper proposes a hierarchical adaptive convolution method(HAC) to localize the eye center accurately while consuming low computational cost.It mainly utilizes the dramatic illumination changes between the iris and sclera. Morespecifically, novel hierarchical kernels are designed to convolute the eye images and adifferential operation is applied on the adjacent convolution results to generate various responsemaps. The final eye center is localized by searching the maximum response valueamong the response maps. Experimental results on several publicly available datasetsdemonstrate that HAC outperforms the start-of-the-art methods by a large margin. Thecode is made publicly available at https://github.com/myopengit/HAC |
Zhao, Shuwen; Cai, Haibin; Liu, Honghai; Zhang, Jianhua; Chen, Shengyong Feature Selection Mechanism in CNNs for Facial Expression Recognition Conference British Machine Vision Conference (BMVC), 2018. @conference{Zhao2018, title = {Feature Selection Mechanism in CNNs for Facial Expression Recognition}, author = {Shuwen Zhao and Haibin Cai and Honghai Liu and Jianhua Zhang and Shengyong Chen}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/0011.pdf}, year = {2018}, date = {2018-07-01}, booktitle = {British Machine Vision Conference (BMVC)}, abstract = {Facial Expression Recognition (FER) has been a challenging problem in computer vi- sion for many decades, mainly due to the high-level variation of face geometry and facial appearance. In this paper, we propose a feature selection network (FSN) to automatically extract and filter facial features by embedding a feature selection mechanism inside the AlexNet. The designed feature selection mechanism effectively filters irrelevant features and emphasises correlated features according to learned feature maps. Experiment re- sults on several databases demonstrate that the FSN outperforms the AlexNet by a large margin and achieves comparable results with the state-of-the-art methods. Furthermore, the FSN also shows improved generalisation ability over the AlexNet in the cross valida- tion experiment of different datasets.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Facial Expression Recognition (FER) has been a challenging problem in computer vi- sion for many decades, mainly due to the high-level variation of face geometry and facial appearance. In this paper, we propose a feature selection network (FSN) to automatically extract and filter facial features by embedding a feature selection mechanism inside the AlexNet. The designed feature selection mechanism effectively filters irrelevant features and emphasises correlated features according to learned feature maps. Experiment re- sults on several databases demonstrate that the FSN outperforms the AlexNet by a large margin and achieves comparable results with the state-of-the-art methods. Furthermore, the FSN also shows improved generalisation ability over the AlexNet in the cross valida- tion experiment of different datasets. |
Jiang, Y; Yang, C; Ju, Z; Wang, M; Liu, H Null Space Based Robot Compliant Control with Prescribed Motion Performance Inproceedings In: 2018 Eighth International Conference on Information Science and Technology (ICIST), pp. 294-300, 2018, ISSN: 2573-3311. @inproceedings{8426069, title = {Null Space Based Robot Compliant Control with Prescribed Motion Performance}, author = {Y Jiang and C Yang and Z Ju and M Wang and H Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/08426069.pdf}, doi = {10.1109/ICIST.2018.8426069}, issn = {2573-3311}, year = {2018}, date = {2018-06-01}, booktitle = {2018 Eighth International Conference on Information Science and Technology (ICIST)}, pages = {294-300}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Richardson, K; Coeckelbergh, M; Wakunuma, K; Billing, E; Ziemke, T; Gomez, P; Vanderborght, B; Belpaeme, T Robot Enhanced Therapy for Children with Autism (DREAM): A Social Model of Autism Journal Article In: IEEE Technology and Society Magazine, 37 (1), pp. 30-39, 2018, ISSN: 0278-0097. @article{Richardson2018, title = {Robot Enhanced Therapy for Children with Autism (DREAM): A Social Model of Autism}, author = {K Richardson and M Coeckelbergh and K Wakunuma and E Billing and T Ziemke and P Gomez and B Vanderborght and T Belpaeme}, doi = {10.1109/MTS.2018.2795096}, issn = {0278-0097}, year = {2018}, date = {2018-03-01}, journal = {IEEE Technology and Society Magazine}, volume = {37}, number = {1}, pages = {30-39}, abstract = {The development of social robots for children with autism has been a growth field for the past 15 years. This article reviews studies in robots and autism as a neurodevelopmental disorder that impacts socialcommunication development, and the ways social robots could help children with autism develop social skills. Drawing on ethics research from the EU-funded Development of Robot-Enhanced Therapy for Children with Autism (DREAM) project (framework 7), this paper explores how ethics evolves and developed in this European project.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The development of social robots for children with autism has been a growth field for the past 15 years. This article reviews studies in robots and autism as a neurodevelopmental disorder that impacts socialcommunication development, and the ways social robots could help children with autism develop social skills. Drawing on ethics research from the EU-funded Development of Robot-Enhanced Therapy for Children with Autism (DREAM) project (framework 7), this paper explores how ethics evolves and developed in this European project. |
Liu, B; Cai, H; Ji, X; Liu, H Human-human interaction recognition based on spatial and motion trend feature Inproceedings In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 4547-4551, 2018, ISSN: 2381-8549. @inproceedings{Liu2017, title = {Human-human interaction recognition based on spatial and motion trend feature}, author = {B Liu and H Cai and X Ji and H Liu}, url = {https://ieeexplore.ieee.org/document/8297143}, doi = {10.1109/ICIP.2017.8297143}, issn = {2381-8549}, year = {2018}, date = {2018-02-22}, booktitle = {2017 IEEE International Conference on Image Processing (ICIP)}, pages = {4547-4551}, abstract = {Human-human interaction recognition has attracted increasing attention in recent years due to its wide applications in computer vision fields. Currently there are few publicly available RGBD-based human-human interaction datasets collected. This paper introduces a new dataset for human-human interaction recognition. Furthermore, a novel feature descriptor based on spatial relationship and semantic motion trend similarity between body parts is proposed for human-human interaction recognition. The motion trend of each skeleton joint is firstly quantified into the specific semantic word and then a Kernel is built for measuring the similarity of either intra or inter body parts by histogram interaction. Finally, the proposed feature descriptor is evaluated on the SBU interaction dataset and the collected dataset. Experimental results demonstrate the outperformance of our method over the state-of-the-art methods.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Human-human interaction recognition has attracted increasing attention in recent years due to its wide applications in computer vision fields. Currently there are few publicly available RGBD-based human-human interaction datasets collected. This paper introduces a new dataset for human-human interaction recognition. Furthermore, a novel feature descriptor based on spatial relationship and semantic motion trend similarity between body parts is proposed for human-human interaction recognition. The motion trend of each skeleton joint is firstly quantified into the specific semantic word and then a Kernel is built for measuring the similarity of either intra or inter body parts by histogram interaction. Finally, the proposed feature descriptor is evaluated on the SBU interaction dataset and the collected dataset. Experimental results demonstrate the outperformance of our method over the state-of-the-art methods. |
Xue, Y; Ju, Z; Xiang, K; Chen, J; Liu, H Multimodal Human Hand Motion Sensing and Analysis -A Review Journal Article In: IEEE Transactions on Cognitive and Developmental Systems, pp. 1-1, 2018, ISSN: 2379-8920. @article{Xue2018, title = {Multimodal Human Hand Motion Sensing and Analysis -A Review}, author = {Y Xue and Z Ju and K Xiang and J Chen and H Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/e0b98b5d7ec6a4df47a5afef5987364d2289.pdf}, doi = {10.1109/TCDS.2018.2800167}, issn = {2379-8920}, year = {2018}, date = {2018-01-31}, journal = {IEEE Transactions on Cognitive and Developmental Systems}, pages = {1-1}, abstract = {Human hand motion analysis is an essential research topic in recent applications, especially for dexterous robot hand manipulation learning from human hand skills. It provides important information about the gestures, tactile, speed and contact force, captured via multiple sensing technologies. This paper introduces a comprehensive survey of current hand motion sensing technologies and analysis approaches in recent emerging applications. Firstly, the nature of human hand motions is discussed in terms of simple motions, such as grasps and gestures, and complex motions, e.g. in-hand manipulations and re-grasps; secondly, different techniques for hand motion sensing, including contact-based and non-contact-based approaches, are discussed with comparisons with their pros and cons; then, the state-of-the-art analysis methods are introduced, with a particular focus on the multimodal hand motion sensing and analysis; finally, cutting-edge applications of hand motion analysis are reviewed, with further discussion on facing challenges and new future directions.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Human hand motion analysis is an essential research topic in recent applications, especially for dexterous robot hand manipulation learning from human hand skills. It provides important information about the gestures, tactile, speed and contact force, captured via multiple sensing technologies. This paper introduces a comprehensive survey of current hand motion sensing technologies and analysis approaches in recent emerging applications. Firstly, the nature of human hand motions is discussed in terms of simple motions, such as grasps and gestures, and complex motions, e.g. in-hand manipulations and re-grasps; secondly, different techniques for hand motion sensing, including contact-based and non-contact-based approaches, are discussed with comparisons with their pros and cons; then, the state-of-the-art analysis methods are introduced, with a particular focus on the multimodal hand motion sensing and analysis; finally, cutting-edge applications of hand motion analysis are reviewed, with further discussion on facing challenges and new future directions. |
Liu, Bangli; Ju, Zhaojie; Liu, Honghai A structured multi-feature representation for recognizing human action and interaction Journal Article In: Neurocomputing, 318 , pp. 287 - 296, 2018, ISSN: 0925-2312. @article{LIU2018287, title = {A structured multi-feature representation for recognizing human action and interaction}, author = {Bangli Liu and Zhaojie Ju and Honghai Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/1-s2.0-S0925231218310282-main.pdfhttp://www.sciencedirect.com/science/article/pii/S0925231218310282}, doi = {https://doi.org/10.1016/j.neucom.2018.08.066}, issn = {0925-2312}, year = {2018}, date = {2018-01-27}, journal = {Neurocomputing}, volume = {318}, pages = {287 - 296}, abstract = {Active research has been carried out for human action recognition using 3D human skeleton joints with the release of cost-efficient RGB-D sensors. However, extracting discriminative features from noisy skeleton sequences to effectively distinguish various human action or interaction categories still remains challenging. This paper proposes a structured multi-feature representation for human action and interaction recognition. Specifically, a novel kernel enhanced bag of semantic words (BSW) is designed to represent the dynamic property of skeleton trajectories. By aggregating BSW with the geometric feature, a GBSW representation is constructed for human action recognition. For human interaction recognition where the cooperation of each subject matters, a GBSWC representation is proposed via combining the GBSW feature with a correlation feature which addresses the intrinsic relationship between interactive persons. Experimental results on several human action and interaction datasets demonstrate the superior performances of the proposed features over the state-of-the-art methods.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Active research has been carried out for human action recognition using 3D human skeleton joints with the release of cost-efficient RGB-D sensors. However, extracting discriminative features from noisy skeleton sequences to effectively distinguish various human action or interaction categories still remains challenging. This paper proposes a structured multi-feature representation for human action and interaction recognition. Specifically, a novel kernel enhanced bag of semantic words (BSW) is designed to represent the dynamic property of skeleton trajectories. By aggregating BSW with the geometric feature, a GBSW representation is constructed for human action recognition. For human interaction recognition where the cooperation of each subject matters, a GBSWC representation is proposed via combining the GBSW feature with a correlation feature which addresses the intrinsic relationship between interactive persons. Experimental results on several human action and interaction datasets demonstrate the superior performances of the proposed features over the state-of-the-art methods. |
Richardson, Kathleen Springer, 2018, ISBN: 978-3-319-74753-8. @book{Richardson2018c, title = {Challenging Sociality}, author = { Kathleen Richardson}, url = {https://link.springer.com/book/10.1007/978-3-319-74754-5}, doi = {https://doi.org/10.1007/978-3-319-74754-5}, isbn = {978-3-319-74753-8}, year = {2018}, date = {2018-01-01}, booktitle = {Challenging Sociality}, pages = {1--156}, publisher = {Springer}, abstract = {This book explores the development of humanoid robots for helping children with autism develop social skills based on fieldwork in the UK and the USA. Robotic scientists propose that robots can therapeutically help children with autism because there is a “special” affinity between them and mechanical things. This idea is supported by autism experts that claim those with autism have a preference for things over other persons. Autism is also seen as a gendered condition, with men considered less social and therefore more likely to have the condition. The author explores how these experiments in cultivating social skills in children with autism using robots, while focused on a unique subsection, is the model for a new kind of human-thing relationship for wider society across the capitalist world where machines can take on the role of the “you” in the relational encounter. Moreover, underscoring this is a form of consciousness that arises out of specific forms of attachment styles. }, keywords = {}, pubstate = {published}, tppubtype = {book} } This book explores the development of humanoid robots for helping children with autism develop social skills based on fieldwork in the UK and the USA. Robotic scientists propose that robots can therapeutically help children with autism because there is a “special” affinity between them and mechanical things. This idea is supported by autism experts that claim those with autism have a preference for things over other persons. Autism is also seen as a gendered condition, with men considered less social and therefore more likely to have the condition. The author explores how these experiments in cultivating social skills in children with autism using robots, while focused on a unique subsection, is the model for a new kind of human-thing relationship for wider society across the capitalist world where machines can take on the role of the “you” in the relational encounter. Moreover, underscoring this is a form of consciousness that arises out of specific forms of attachment styles. |
Gao, Qing; Liu, Jinguo; Ju, Zhaojie; Zhang, Lu; Li, Yangmin; Liu, Yuwang Hand Detection and Location Based on Improved SSD for Space Human-Robot Interaction Inproceedings In: Chen, Zhiyong; Mendes, Alexandre; Yan, Yamin; Chen, Shifeng (Ed.): Intelligent Robotics and Applications, pp. 164–175, Springer International Publishing, Cham, 2018, ISBN: 978-3-319-97586-3. @inproceedings{Gao2018, title = {Hand Detection and Location Based on Improved SSD for Space Human-Robot Interaction}, author = {Qing Gao and Jinguo Liu and Zhaojie Ju and Lu Zhang and Yangmin Li and Yuwang Liu}, editor = {Zhiyong Chen and Alexandre Mendes and Yamin Yan and Shifeng Chen}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/C18-Gao-Liu-Li-ICIRA.pdf}, isbn = {978-3-319-97586-3}, year = {2018}, date = {2018-01-01}, booktitle = {Intelligent Robotics and Applications}, pages = {164--175}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {In the astronaut-space robot interaction based on hand gestures, the detection and location of hands are the premise and basis of vision-based hand gesture recognition and hand tracking. In this paper, the SSD (Single Shot Multibox Detector) which is a kind of deep learning model is utilized to detect and locate astronaut's hands for space human-robot interaction (SHRI) based on hand gestures. First of all, in order to meet the needs of hand detection and location, an improved SSD model is designed to detect hands when they are shown as small targets in images. Then, a platform for SHRI is built and a set of hand gestures for SHRI are designed. Finally, the proposed SSD model is validated experimentally on a homemade hand gesture database for proving the superiority of this improved SSD model to small target hands detection.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In the astronaut-space robot interaction based on hand gestures, the detection and location of hands are the premise and basis of vision-based hand gesture recognition and hand tracking. In this paper, the SSD (Single Shot Multibox Detector) which is a kind of deep learning model is utilized to detect and locate astronaut's hands for space human-robot interaction (SHRI) based on hand gestures. First of all, in order to meet the needs of hand detection and location, an improved SSD model is designed to detect hands when they are shown as small targets in images. Then, a platform for SHRI is built and a set of hand gestures for SHRI are designed. Finally, the proposed SSD model is validated experimentally on a homemade hand gesture database for proving the superiority of this improved SSD model to small target hands detection. |
Zhou, Dalin; Fang, Yinfeng; Ju, Zhaojie; Liu, Honghai Multi-length Windowed Feature Selection for Surface EMG Based Hand Motion Recognition Inproceedings In: Chen, Zhiyong; Mendes, Alexandre; Yan, Yamin; Chen, Shifeng (Ed.): Intelligent Robotics and Applications, pp. 264–274, Springer International Publishing, Cham, 2018, ISBN: 978-3-319-97586-3. @inproceedings{10.1007/978-3-319-97586-3_24, title = {Multi-length Windowed Feature Selection for Surface EMG Based Hand Motion Recognition}, author = {Dalin Zhou and Yinfeng Fang and Zhaojie Ju and Honghai Liu}, editor = {Zhiyong Chen and Alexandre Mendes and Yamin Yan and Shifeng Chen}, isbn = {978-3-319-97586-3}, year = {2018}, date = {2018-01-01}, booktitle = {Intelligent Robotics and Applications}, pages = {264--274}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {Feature selection for surface electromyography based hand motion recognition has been seen to retrieve an optimal or quasi-optimal feature subset for classification. This work aims to consider the influence of channel, feature and window length simultaneously with an emphasis on the multiple segmentation. The bacterial memetic algorithm is applied to select the feature candidates from time domain and autoregressive coefficients, which is measured by the inter-day hand motion recognition accuracy. The evaluation is conducted on a case study of 3 able-bodied subjects performing 9 hand motions in consecutive 7 days with 4 different window lengths adopted for the electromyographic data segmentation. Classification in combination with the multi-length windowed feature selection achieved an improved recognition accuracy in comparison with using solely the single-length windowed features in inter-day scenarios and indicated that complementary information to full length segmentation resides in the sub-windows, thus providing feasible feature combinations for conventional pattern recognition based solutions to prosthetic control.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Feature selection for surface electromyography based hand motion recognition has been seen to retrieve an optimal or quasi-optimal feature subset for classification. This work aims to consider the influence of channel, feature and window length simultaneously with an emphasis on the multiple segmentation. The bacterial memetic algorithm is applied to select the feature candidates from time domain and autoregressive coefficients, which is measured by the inter-day hand motion recognition accuracy. The evaluation is conducted on a case study of 3 able-bodied subjects performing 9 hand motions in consecutive 7 days with 4 different window lengths adopted for the electromyographic data segmentation. Classification in combination with the multi-length windowed feature selection achieved an improved recognition accuracy in comparison with using solely the single-length windowed features in inter-day scenarios and indicated that complementary information to full length segmentation resides in the sub-windows, thus providing feasible feature combinations for conventional pattern recognition based solutions to prosthetic control. |
Xue, Y; Ju, Z; Xiang, K; Chen, J; Liu, H Multimodal Human Hand Motion Sensing and Analysis -A Review Journal Article In: IEEE Transactions on Cognitive and Developmental Systems, pp. 1-1, 2018, ISSN: 2379-8920. @article{8277185, title = {Multimodal Human Hand Motion Sensing and Analysis -A Review}, author = {Y Xue and Z Ju and K Xiang and J Chen and H Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/08277185.pdf}, doi = {10.1109/TCDS.2018.2800167}, issn = {2379-8920}, year = {2018}, date = {2018-01-01}, journal = {IEEE Transactions on Cognitive and Developmental Systems}, pages = {1-1}, abstract = {Human hand motion analysis is an essential research topic in recent applications, especially for dexterous robot hand manipulation learning from human hand skills. It provides important information about the gestures, tactile, speed and contact force, captured via multiple sensing technologies. This paper introduces a comprehensive survey of current hand motion sensing technologies and analysis approaches in recent emerging applications. Firstly, the nature of human hand motions is discussed in terms of simple motions, such as grasps and gestures, and complex motions, e.g. in-hand manipulations and re-grasps; secondly, different techniques for hand motion sensing, including contact-based and non-contact-based approaches, are discussed with comparisons with their pros and cons; then, the state-of-the-art analysis methods are introduced, with a particular focus on the multimodal hand motion sensing and analysis; finally, cutting-edge applications of hand motion analysis are reviewed, with further discussion on facing challenges and new future directions.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Human hand motion analysis is an essential research topic in recent applications, especially for dexterous robot hand manipulation learning from human hand skills. It provides important information about the gestures, tactile, speed and contact force, captured via multiple sensing technologies. This paper introduces a comprehensive survey of current hand motion sensing technologies and analysis approaches in recent emerging applications. Firstly, the nature of human hand motions is discussed in terms of simple motions, such as grasps and gestures, and complex motions, e.g. in-hand manipulations and re-grasps; secondly, different techniques for hand motion sensing, including contact-based and non-contact-based approaches, are discussed with comparisons with their pros and cons; then, the state-of-the-art analysis methods are introduced, with a particular focus on the multimodal hand motion sensing and analysis; finally, cutting-edge applications of hand motion analysis are reviewed, with further discussion on facing challenges and new future directions. |
Wang, Yiming; Yu, Hui; Dong, Junyu; Stevens, Brett; Liu, Honghai Facial Expression-Aware Face Frontalization Inproceedings In: Lai, Shang-Hong; Lepetit, Vincent; Nishino, Ko; Sato, Yoichi (Ed.): Computer Vision -- ACCV 2016, pp. 375–388, Springer International Publishing, Cham, 2017, ISBN: 978-3-319-54187-7. @inproceedings{Wang2017, title = {Facial Expression-Aware Face Frontalization}, author = {Yiming Wang and Hui Yu and Junyu Dong and Brett Stevens and Honghai Liu}, editor = {Shang-Hong Lai and Vincent Lepetit and Ko Nishino and Yoichi Sato}, url = {https://link.springer.com/chapter/10.1007/978-3-319-54187-7_25}, isbn = {978-3-319-54187-7}, year = {2017}, date = {2017-11-03}, booktitle = {Computer Vision -- ACCV 2016}, pages = {375--388}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {Face frontalization is a rising technique for view-invariant face analysis. It enables a non-frontal facial image to recover its general facial appearances to frontal view. A few pioneering works have been proposed very recently. However, face frontalization with detailed facial expression recovering is still very challenging due to the non-linear relationships between head-pose and expression variations. In this paper, we propose a novel facial expression-aware face frontalization method aiming at reconstructing the frontal view while maintaining vivid appearances with regards to facial expressions. First of all, we design multiple face shape models as the reference templates in order to fit in with various shape of facial expressions. Each template describes a set of typical facial actions referred to Facial Action Coding System (FACS). Then a template matching strategy is applied by measuring a weighted Chi Square error such that the input image can be matched with the most approximate template. Finally, Robust Statistical face Frontalization (RSF) method is employed for the task of frontal view recovery. This method is validated on a spontaneous facial expression database and the experimental results show that the proposed method outperforms the state-of-the-art methods.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Face frontalization is a rising technique for view-invariant face analysis. It enables a non-frontal facial image to recover its general facial appearances to frontal view. A few pioneering works have been proposed very recently. However, face frontalization with detailed facial expression recovering is still very challenging due to the non-linear relationships between head-pose and expression variations. In this paper, we propose a novel facial expression-aware face frontalization method aiming at reconstructing the frontal view while maintaining vivid appearances with regards to facial expressions. First of all, we design multiple face shape models as the reference templates in order to fit in with various shape of facial expressions. Each template describes a set of typical facial actions referred to Facial Action Coding System (FACS). Then a template matching strategy is applied by measuring a weighted Chi Square error such that the input image can be matched with the most approximate template. Finally, Robust Statistical face Frontalization (RSF) method is employed for the task of frontal view recovery. This method is validated on a spontaneous facial expression database and the experimental results show that the proposed method outperforms the state-of-the-art methods. |
Fang, Y; Zhou, D; Li, K; Liu, H Interface Prostheses With Classifier-Feedback-Based User Training Journal Article In: IEEE Transactions on Biomedical Engineering, 64 (11), pp. 2575-2583, 2017, ISSN: 0018-9294. @article{Fang2017, title = {Interface Prostheses With Classifier-Feedback-Based User Training}, author = {Y Fang and D Zhou and K Li and H Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/07792576.pdf}, doi = {10.1109/TBME.2016.2641584}, issn = {0018-9294}, year = {2017}, date = {2017-11-01}, journal = {IEEE Transactions on Biomedical Engineering}, volume = {64}, number = {11}, pages = {2575-2583}, abstract = {It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.}, keywords = {}, pubstate = {published}, tppubtype = {article} } It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation. |
Fang, Y; Zhou, D; Li, K; Ju, Z; Liu, H A force-driven granular model for EMG based grasp recognition Inproceedings In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2939-2944, 2017. @inproceedings{Fang2017b, title = {A force-driven granular model for EMG based grasp recognition}, author = {Y Fang and D Zhou and K Li and Z Ju and H Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/11/08123074.pdf}, doi = {10.1109/SMC.2017.8123074}, year = {2017}, date = {2017-10-01}, booktitle = {2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)}, pages = {2939-2944}, abstract = {It is a challenge to precisely predict hand grasps based on EMG signals given practical scenarios, due to its inherent nature. This paper proposes a solution to tackle the challenge with a force-driven granular model (FDGM). The problem of n-class hand grasp classification has been represented as force-based granular modelling, in which a number of granules are constructed for each class relying on the synchronically captured grasping force. A rule based mechanism is formed for granule generation of each class, and a cross-testing algorithm is proposed to optimise the number of granules. The experiment based on 8-case grasp recognition reveals that the proposed method performs better in terms of motion recognition accuracy of multiple EMG channel combination, and is more insensitive to signal interferences. In comparison with other rules of information granulation, it is confirmed that the force-driven rule is of the most efficiency with comparable classification accuracy. The research outcomes pave the way for real-time prediction of grasps and corresponding force in human-centred environments. }, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } It is a challenge to precisely predict hand grasps based on EMG signals given practical scenarios, due to its inherent nature. This paper proposes a solution to tackle the challenge with a force-driven granular model (FDGM). The problem of n-class hand grasp classification has been represented as force-based granular modelling, in which a number of granules are constructed for each class relying on the synchronically captured grasping force. A rule based mechanism is formed for granule generation of each class, and a cross-testing algorithm is proposed to optimise the number of granules. The experiment based on 8-case grasp recognition reveals that the proposed method performs better in terms of motion recognition accuracy of multiple EMG channel combination, and is more insensitive to signal interferences. In comparison with other rules of information granulation, it is confirmed that the force-driven rule is of the most efficiency with comparable classification accuracy. The research outcomes pave the way for real-time prediction of grasps and corresponding force in human-centred environments. |
Liao, Y; Li, G; Ju, Z; Liu, H; Jiang, D Joint kinect and multiple external cameras simultaneous calibration Inproceedings In: 2017 2nd International Conference on Advanced Robotics and Mechatronics (ICARM), pp. 305-310, 2017. @inproceedings{Liao2017, title = {Joint kinect and multiple external cameras simultaneous calibration}, author = {Y Liao and G Li and Z Ju and H Liu and D Jiang}, url = {https://ieeexplore.ieee.org/abstract/document/8273179}, doi = {10.1109/ICARM.2017.8273179}, year = {2017}, date = {2017-08-02}, booktitle = {2017 2nd International Conference on Advanced Robotics and Mechatronics (ICARM)}, pages = {305-310}, abstract = {Camera calibration applications are used in many places, such as visual estimation, 3D reconstruction and object tracking. Over the past few decades, many methods have been proposed and applied to these areas. However, with the diversification of sensor technology, the problem of multi-sensor joint calibration becomes more and more obvious. In this paper, we propose a new method that can simultaneously calibrate relative poses of three external cameras and a Kinect. And the final cost function is optimized for joint calibration. In addition, the practical platform has also tested the method, and the experimental results show that the accuracy of the proposed method significantly improved.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Camera calibration applications are used in many places, such as visual estimation, 3D reconstruction and object tracking. Over the past few decades, many methods have been proposed and applied to these areas. However, with the diversification of sensor technology, the problem of multi-sensor joint calibration becomes more and more obvious. In this paper, we propose a new method that can simultaneously calibrate relative poses of three external cameras and a Kinect. And the final cost function is optimized for joint calibration. In addition, the practical platform has also tested the method, and the experimental results show that the accuracy of the proposed method significantly improved. |
de Perre, Greet Van; Cao, Hoang-Long; Beir, Albert De; 'o, Pablo G; Lefeber, Dirk; Vanderborght, Bram Generic method for generating blended gestures and affective functional behaviors for social robots Journal Article In: Autonomous Robots, pp. 1–12, 2017. @article{van2017generic, title = {Generic method for generating blended gestures and affective functional behaviors for social robots}, author = { Greet Van de Perre and Hoang-Long Cao and Albert De Beir and Pablo G{'o}mez Esteban and Dirk Lefeber and Bram Vanderborght}, url = {https://link.springer.com/article/10.1007/s10514-017-9650-0}, year = {2017}, date = {2017-06-05}, journal = {Autonomous Robots}, pages = {1--12}, publisher = {Springer}, abstract = {Gesturing is an important modality in human–robot interaction. Up to date, gestures are often implemented for a specific robot configuration and therefore not easily transferable to other robots. To cope with this issue, we presented a generic method to calculate gestures for social robots. The method was designed to work in two modes to allow the calculation of different types of gestures. In this paper, we present the new developments of the method. We discuss how the two working modes can be combined to generate blended emotional expressions and deictic gestures. In certain situations, it is desirable to express an emotional condition through an ongoing functional behavior. Therefore, we implemented the possibility of modulating a pointing or reaching gesture into an affective gesture by influencing the motion speed and amplitude of the posture. The new implementations were validated on virtual models with different configurations, including those of the robots NAO and Justin.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Gesturing is an important modality in human–robot interaction. Up to date, gestures are often implemented for a specific robot configuration and therefore not easily transferable to other robots. To cope with this issue, we presented a generic method to calculate gestures for social robots. The method was designed to work in two modes to allow the calculation of different types of gestures. In this paper, we present the new developments of the method. We discuss how the two working modes can be combined to generate blended emotional expressions and deictic gestures. In certain situations, it is desirable to express an emotional condition through an ongoing functional behavior. Therefore, we implemented the possibility of modulating a pointing or reaching gesture into an affective gesture by influencing the motion speed and amplitude of the posture. The new implementations were validated on virtual models with different configurations, including those of the robots NAO and Justin. |
Cao, Hoang-Long; Gomez Esteban, Pablo ; Beir, Albert De; Simut Vanderborght, Ramona ; {Van De Perre}, Greet ; Lefeber, Dirk; Vanderboght, Bram A survey on behavior control architectures for social robots in healthcare interventions Journal Article In: International Journal of Humanoid Robotics, 2017, ISSN: 0219-8436. @article{Cao2017, title = {A survey on behavior control architectures for social robots in healthcare interventions}, author = {Hoang-Long Cao and Gomez Esteban, Pablo and Albert De Beir and Simut Vanderborght, Ramona and {Van De Perre}, Greet and Dirk Lefeber and Bram Vanderboght }, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/a-survey-on-behavior-control-architectures-for-social-robots-in-healthcare-interventions(188d5d89-e7df-44db-940d-d731726daaa8).html}, issn = {0219-8436}, year = {2017}, date = {2017-06-01}, journal = {International Journal of Humanoid Robotics}, publisher = {World Scientific Publishing Co. Pte Ltd}, abstract = {The use of Robot-Assisted Therapy (RAT) in health-care interventions has increasingly received research attention. However, a lot of RAT studies are conducted under Wizard of Oz (WoZ) techniques in which the robots are teleoperated or preprogrammed. The trend of RAT is moving towards (partially) autonomous control in which the robot behavior control architecture plays a significant role in creating effective human-robot interaction by engaging and motivating human users into the therapeutic processes. This paper describes the state of the art of the autonomous behavior control architectures currently developed for social robots in health-care interventions, considering both clinical and exploratory studies. We also present certain requirements that an architecture used in RAT study should acquire, which provide roboticists and therapists an inspiration to orient their designs and implementations on the basis of their targeted RAT applications.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The use of Robot-Assisted Therapy (RAT) in health-care interventions has increasingly received research attention. However, a lot of RAT studies are conducted under Wizard of Oz (WoZ) techniques in which the robots are teleoperated or preprogrammed. The trend of RAT is moving towards (partially) autonomous control in which the robot behavior control architecture plays a significant role in creating effective human-robot interaction by engaging and motivating human users into the therapeutic processes. This paper describes the state of the art of the autonomous behavior control architectures currently developed for social robots in health-care interventions, considering both clinical and exploratory studies. We also present certain requirements that an architecture used in RAT study should acquire, which provide roboticists and therapists an inspiration to orient their designs and implementations on the basis of their targeted RAT applications. |
Zhou, X; Cai, H; Li, Y; Liu, H Two-eye model-based gaze estimation from a Kinect sensor Inproceedings In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1646-1653, 2017. @inproceedings{7989194, title = {Two-eye model-based gaze estimation from a Kinect sensor}, author = {X Zhou and H Cai and Y Li and H Liu}, url = {http://www.dream2020.eu/wp-content/uploads/2018/09/ICRA2017.pdf}, doi = {10.1109/ICRA.2017.7989194}, year = {2017}, date = {2017-05-01}, booktitle = {2017 IEEE International Conference on Robotics and Automation (ICRA)}, pages = {1646-1653}, abstract = {In this paper, we present an effective and accurate gaze estimation method based on two-eye model of a subject with the tolerance of free head movement from a Kinect sensor. To accurately and efficiently determine the point of gaze, i) we employ two-eye model to improve the estimation accuracy; ii) we propose an improved convolution-based means of gradients method to localize the iris center in 3D space; iii) we present a new personal calibration method that only needs one calibration point. The method approximates the visual axis as a line from the iris center to the gaze point to determine the eyeball centers and the Kappa angles. The final point of gaze can be calculated by using the calibrated personal eye parameters. We experimentally evaluate the proposed gaze estimation method on eleven subjects. Experimental results demonstrate that our gaze estimation method has an average estimation accuracy around 1.99◦, which outperforms many leading methods in the state-of-the-art.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In this paper, we present an effective and accurate gaze estimation method based on two-eye model of a subject with the tolerance of free head movement from a Kinect sensor. To accurately and efficiently determine the point of gaze, i) we employ two-eye model to improve the estimation accuracy; ii) we propose an improved convolution-based means of gradients method to localize the iris center in 3D space; iii) we present a new personal calibration method that only needs one calibration point. The method approximates the visual axis as a line from the iris center to the gaze point to determine the eyeball centers and the Kappa angles. The final point of gaze can be calculated by using the calibrated personal eye parameters. We experimentally evaluate the proposed gaze estimation method on eleven subjects. Experimental results demonstrate that our gaze estimation method has an average estimation accuracy around 1.99◦, which outperforms many leading methods in the state-of-the-art. |
Hoang-Long, Cao ; 'o, G; , Albert De Beir; , Simut Ramona; de , Van Perre Greet; , Lefeber Dirk; Bram, Vanderborght A Collaborative Homeostatic-Based Behavior Controller for Social Robots in Human--Robot Interaction Experiments Journal Article In: International Journal of Social Robotics, 2017, ISSN: 1875-4805. @article{Cao2017b, title = {A Collaborative Homeostatic-Based Behavior Controller for Social Robots in Human--Robot Interaction Experiments}, author = {Hoang-Long, Cao and G{'o}mez Esteban Pablo and Albert De Beir and Simut Ramona and Van de Perre Greet and Lefeber Dirk and Vanderborght Bram}, url = {http://dx.doi.org/10.1007/s12369-017-0405-z https://link.springer.com/article/10.1007/s12369-017-0405-z?no-access=true}, doi = {10.1007/s12369-017-0405-z}, issn = {1875-4805}, year = {2017}, date = {2017-04-11}, journal = {International Journal of Social Robotics}, abstract = {Robots have been gradually leaving laboratory and factory environments and moving into human populated environments. Various social robots have been developed with the ability to exhibit social behaviors and collaborate with non-expert users in different situations. In order to increase the degree of collaboration between humans and the robots in human--robot joint action systems, these robots need to achieve higher levels of interaction with humans. However, many social robots are operated under teleoperation modes or pre-programmed scenarios. Based on homeostatic drive theory, this paper presents the development of a novel collaborative behavior controller for social robots to jointly perform tasks with users in human--robot interaction (HRI) experiments. Manual work during the experiments is reduced, and the experimenters can focus more on the interaction. We propose a hybrid concept for the behavior decision-making process, which combines the hierarchical approach and parallel-rooted, ordered, slip-stack hierarchical architecture. Emotions are associated with behaviors by using the two-dimensional space model of valence and arousal. We validate the usage of the behavior controller by a joint attention HRI scenario in which the NAO robot and a therapist jointly interact with children.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Robots have been gradually leaving laboratory and factory environments and moving into human populated environments. Various social robots have been developed with the ability to exhibit social behaviors and collaborate with non-expert users in different situations. In order to increase the degree of collaboration between humans and the robots in human--robot joint action systems, these robots need to achieve higher levels of interaction with humans. However, many social robots are operated under teleoperation modes or pre-programmed scenarios. Based on homeostatic drive theory, this paper presents the development of a novel collaborative behavior controller for social robots to jointly perform tasks with users in human--robot interaction (HRI) experiments. Manual work during the experiments is reduced, and the experimenters can focus more on the interaction. We propose a hybrid concept for the behavior decision-making process, which combines the hierarchical approach and parallel-rooted, ordered, slip-stack hierarchical architecture. Emotions are associated with behaviors by using the two-dimensional space model of valence and arousal. We validate the usage of the behavior controller by a joint attention HRI scenario in which the NAO robot and a therapist jointly interact with children. |
Senft, Emmanuel; Baxter, Paul; Kennedy, James; 'e, S; Belpaeme, Tony Supervised autonomy for online learning in human-robot interaction Journal Article In: Pattern Recognition Letters, 2017. @article{senft2017supervised, title = {Supervised autonomy for online learning in human-robot interaction}, author = { Emmanuel Senft and Paul Baxter and James Kennedy and S{'e}verin Lemaignan and Tony Belpaeme}, editor = {2017-03-18}, url = {https://pearl.plymouth.ac.uk/bitstream/handle/10026.1/9123/Senft%20et%20al%20-%20Supervised%20autonomy%20for%20online%20learning%20in%20human-robot%20interaction%20-%202017.pdf?sequence=1 http://www.dream2020.eu/wp-content/uploads/2017/07/2017-ES-PRL.pdf}, doi = {10.1016/j.patrec.2017.03.015}, year = {2017}, date = {2017-03-18}, journal = {Pattern Recognition Letters}, publisher = {Elsevier}, abstract = {When a robot is learning it needs to explore its environment and how its environment responds on its actions. When the environment is large and there are a large number of possible actions the robot can take, this exploration phase can take prohibitively long. However, exploration can often be optimised by letting a human expert guide the robot during its learning. Interactive machine learning, in which a human user interactively guides the robot as it learns, has been shown to be an effective way to teach a robot. It requires an intuitive control mechanism to allow the human expert to provide feedback on the robot’s progress. This paper presents a novel method which combines Reinforcement Learning and Supervised Progressively Autonomous Robot Competencies (SPARC). By allowing the user to fully control the robot and by treating rewards as implicit, SPARC aims to learn an action policy while maintaining human supervisory oversight of the robot’s behaviour. This method is evaluated and compared to Interactive Reinforcement Learning in a robot teaching task. Qualitative and quantitative results indicate that SPARC allows for safer and faster learning by the robot, whilst not placing a high workload on the human teacher.}, keywords = {}, pubstate = {published}, tppubtype = {article} } When a robot is learning it needs to explore its environment and how its environment responds on its actions. When the environment is large and there are a large number of possible actions the robot can take, this exploration phase can take prohibitively long. However, exploration can often be optimised by letting a human expert guide the robot during its learning. Interactive machine learning, in which a human user interactively guides the robot as it learns, has been shown to be an effective way to teach a robot. It requires an intuitive control mechanism to allow the human expert to provide feedback on the robot’s progress. This paper presents a novel method which combines Reinforcement Learning and Supervised Progressively Autonomous Robot Competencies (SPARC). By allowing the user to fully control the robot and by treating rewards as implicit, SPARC aims to learn an action policy while maintaining human supervisory oversight of the robot’s behaviour. This method is evaluated and compared to Interactive Reinforcement Learning in a robot teaching task. Qualitative and quantitative results indicate that SPARC allows for safer and faster learning by the robot, whilst not placing a high workload on the human teacher. |
Zhou, X; Cai, H; Shao, Z; Yu, H; Liu, H 3D eye model-based gaze estimation from a depth sensor Inproceedings In: 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 369-374, 2017. @inproceedings{Zhou2016, title = {3D eye model-based gaze estimation from a depth sensor}, author = {X Zhou and H Cai and Z Shao and H Yu and H Liu}, url = {https://ieeexplore.ieee.org/document/7866350 http://www.dream2020.eu/wp-content/uploads/2019/05/77049498.pdf}, doi = {10.1109/ROBIO.2016.7866350}, year = {2017}, date = {2017-02-03}, booktitle = {2016 IEEE International Conference on Robotics and Biomimetics (ROBIO)}, pages = {369-374}, abstract = {In this paper, we address the 3D eye gaze estimation problem using a low-cost, simple-setup, and non-intrusive consumer depth sensor (Kinect sensor). We present an effective and accurate method based on 3D eye model to estimate the point of gaze of a subject with the tolerance of free head movement. To determine the parameters involved in the proposed eye model, we propose i) an improved convolution-based means of gradients iris center localization method to accurately and efficiently locate the iris center in 3D space; ii) a geometric constraints-based method to estimate the eyeball center under the constraints that all the iris center points are distributed on a sphere originated from the eyeball center and the sizes of two eyeballs of a subject are identical; iii) an effective Kappa angle calculation method based on the fact that the visual axes of both eyes intersect at a same point with the screen plane. The final point of gaze is calculated by using the estimated eye model parameters. We experimentally evaluate our gaze estimation method on five subjects. The experimental results show the good performance of the proposed method with an average estimation accuracy of 3.78°, which outperforms several state-of-the-arts.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In this paper, we address the 3D eye gaze estimation problem using a low-cost, simple-setup, and non-intrusive consumer depth sensor (Kinect sensor). We present an effective and accurate method based on 3D eye model to estimate the point of gaze of a subject with the tolerance of free head movement. To determine the parameters involved in the proposed eye model, we propose i) an improved convolution-based means of gradients iris center localization method to accurately and efficiently locate the iris center in 3D space; ii) a geometric constraints-based method to estimate the eyeball center under the constraints that all the iris center points are distributed on a sphere originated from the eyeball center and the sizes of two eyeballs of a subject are identical; iii) an effective Kappa angle calculation method based on the fact that the visual axes of both eyes intersect at a same point with the screen plane. The final point of gaze is calculated by using the estimated eye model parameters. We experimentally evaluate our gaze estimation method on five subjects. The experimental results show the good performance of the proposed method with an average estimation accuracy of 3.78°, which outperforms several state-of-the-arts. |
Gao, D; Ju, Z; Fang, Y; Cao, J; Yang, C; Liu, H Activity recognition for asd children based on joints estimation Inproceedings In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2951-2956, 2017. @inproceedings{Gao2017, title = {Activity recognition for asd children based on joints estimation}, author = {D Gao and Z Ju and Y Fang and J Cao and C Yang and H Liu}, url = {https://ieeexplore.ieee.org/document/8123076}, doi = {10.1109/SMC.2017.8123076}, year = {2017}, date = {2017-01-12}, booktitle = {2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)}, pages = {2951-2956}, abstract = {Human motion recognition is a trending topic and could be applied in many areas, the motion estimation of ASD children is more challenging because of the high uncertainty of their activities, we thus introduced a novel method which is designed for estimating the upper joints and recognising their special motions, we verified the proposed method on our recorded ASD children dataset and adult dataset, the experimental results show the proposed method is effective on the dataset.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Human motion recognition is a trending topic and could be applied in many areas, the motion estimation of ASD children is more challenging because of the high uncertainty of their activities, we thus introduced a novel method which is designed for estimating the upper joints and recognising their special motions, we verified the proposed method on our recorded ASD children dataset and adult dataset, the experimental results show the proposed method is effective on the dataset. |
Cai, H; Lee, D; Joonkoo, H; Fang, Y; Li, S; Liu, H Embedded vision based automotive interior intrusion detection system Inproceedings In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2909-2914, 2017. @inproceedings{Cai2017, title = {Embedded vision based automotive interior intrusion detection system}, author = {H Cai and D Lee and H Joonkoo and Y Fang and S Li and H Liu}, url = {https://ieeexplore.ieee.org/document/8123069}, doi = {10.1109/SMC.2017.8123069}, year = {2017}, date = {2017-01-12}, booktitle = {2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)}, pages = {2909-2914}, abstract = {Motor vehicle theft has caused massive economic loss over the world. This paper proposes an embedded vision system to detect automotive interior intrusion. The system uses a fusion of an acceleration module and a vision module to meet the requirement of low power consumption for most motor vehicles. Furthermore, an effective intrusion detection algorithm is developed for the on-board vision module. The vision system is able to detect the intrusion even in the dark night due to the employment of infrared lights. Experimental evaluation is conducted under a variety of illumination conditions, such as day time, night time and even shining light. An intrusion detection accuracy of 91.7% is achieved, which shows that the developed embedded vision system is reliable for motor vehicle intrusion detection.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Motor vehicle theft has caused massive economic loss over the world. This paper proposes an embedded vision system to detect automotive interior intrusion. The system uses a fusion of an acceleration module and a vision module to meet the requirement of low power consumption for most motor vehicles. Furthermore, an effective intrusion detection algorithm is developed for the on-board vision module. The vision system is able to detect the intrusion even in the dark night due to the employment of infrared lights. Experimental evaluation is conducted under a variety of illumination conditions, such as day time, night time and even shining light. An intrusion detection accuracy of 91.7% is achieved, which shows that the developed embedded vision system is reliable for motor vehicle intrusion detection. |
Phiri, Charles C; Botzheim, János; Valle, Cristina; Ju, Zhaojie; Liu, Honghai Dynamical System Algorithm Specification Analysis and Stabilization Inproceedings In: Huang, YongAn; Wu, Hao; Liu, Honghai; Yin, Zhouping (Ed.): Intelligent Robotics and Applications, pp. 560–569, Springer International Publishing, Cham, 2017, ISBN: 978-3-319-65289-4. @inproceedings{Phiri2017, title = {Dynamical System Algorithm Specification Analysis and Stabilization}, author = {Charles C Phiri and János Botzheim and Cristina Valle and Zhaojie Ju and Honghai Liu}, editor = {YongAn Huang and Hao Wu and Honghai Liu and Zhouping Yin}, url = {https://link.springer.com/chapter/10.1007/978-3-319-65289-4_53}, isbn = {978-3-319-65289-4}, year = {2017}, date = {2017-01-09}, booktitle = {Intelligent Robotics and Applications}, pages = {560--569}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {This paper investigates approaches to deliberately designing systems whose controllability can be quantified. Preliminary findings of ongoing research are presented on complex dynamical system control algorithms. The specification analysis and quality of the pressure control algorithm applied to a Topical Negative Pressure Wound Therapy device are conducted, with further discussion on self-regulation mechanism and characterization of both the partially observable and partially controllable workspace represented by the negative pressure chamber. Statistical methods are employed to understand the device physics and fuzzy logic and bacterial memetic algorithm are utilised to explore and optimize the existing algorithms and also extract the rule base.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } This paper investigates approaches to deliberately designing systems whose controllability can be quantified. Preliminary findings of ongoing research are presented on complex dynamical system control algorithms. The specification analysis and quality of the pressure control algorithm applied to a Topical Negative Pressure Wound Therapy device are conducted, with further discussion on self-regulation mechanism and characterization of both the partially observable and partially controllable workspace represented by the negative pressure chamber. Statistical methods are employed to understand the device physics and fuzzy logic and bacterial memetic algorithm are utilised to explore and optimize the existing algorithms and also extract the rule base. |
Zhang, Shu; Yu, Hui; Wang, Ting; Qi, Lin; Dong, Junyu; Liu, Honghai Dense 3D facial reconstruction from a single depth image in unconstrained environment Journal Article In: Virtual Reality, 22 (1), pp. 37–46, 2017, ISSN: 1434-9957. @article{Zhang2018, title = {Dense 3D facial reconstruction from a single depth image in unconstrained environment}, author = {Shu Zhang and Hui Yu and Ting Wang and Lin Qi and Junyu Dong and Honghai Liu}, url = {https://doi.org/10.1007/s10055-017-0311-6 https://link.springer.com/article/10.1007/s10055-017-0311-6}, doi = {10.1007/s10055-017-0311-6}, issn = {1434-9957}, year = {2017}, date = {2017-01-04}, journal = {Virtual Reality}, volume = {22}, number = {1}, pages = {37--46}, abstract = {With the increasing demands of applications in virtual reality such as 3D films, virtual human--machine interactions and virtual agents, the analysis of 3D human face is considered to be more and more important as a fundamental step in these tasks. Due to information provided by the additional dimension, 3D facial reconstruction enables aforementioned tasks to be achieved with higher accuracy than those based on 2D facial analysis. The denser the 3D facial model is, the more information it could provide. However, most existing dense 3D facial reconstruction methods require complicated processing and high system cost. To this end, this paper presents a novel method that simplifies the process of dense 3D facial reconstruction by employing only one frame of depth data obtained with an off-the-shelf RGB-D sensor. The proposed method is composed of two main stages: (a) the acquisition of the initial 3D facial point cloud with automatically 3D facial region cropping, and (b) the generating of the dense facial point cloud with RBF-based adaptive 3D point interpolation. Experiments reported in this paper demonstrate the competitive results with real-world data.}, keywords = {}, pubstate = {published}, tppubtype = {article} } With the increasing demands of applications in virtual reality such as 3D films, virtual human--machine interactions and virtual agents, the analysis of 3D human face is considered to be more and more important as a fundamental step in these tasks. Due to information provided by the additional dimension, 3D facial reconstruction enables aforementioned tasks to be achieved with higher accuracy than those based on 2D facial analysis. The denser the 3D facial model is, the more information it could provide. However, most existing dense 3D facial reconstruction methods require complicated processing and high system cost. To this end, this paper presents a novel method that simplifies the process of dense 3D facial reconstruction by employing only one frame of depth data obtained with an off-the-shelf RGB-D sensor. The proposed method is composed of two main stages: (a) the acquisition of the initial 3D facial point cloud with automatically 3D facial region cropping, and (b) the generating of the dense facial point cloud with RBF-based adaptive 3D point interpolation. Experiments reported in this paper demonstrate the competitive results with real-world data. |
Kennedy, James; Lemaignan, Severin; Montassier, Caroline; Lavalade, Pauline; Irfan, Bahar; Papadopoulos, Fotios; Senft, Emmanuel; Belpaeme, Tony Child Speech Recognition in Human-Robot Interaction: Evaluations and Recommendations Inproceedings In: Proceedings of the 12th ACM/IEEE International Conference on Human-Robot Interaction, Vienna, Austria, 2017. @inproceedings{Kennedy2017, title = {Child Speech Recognition in Human-Robot Interaction: Evaluations and Recommendations}, author = { James Kennedy and Severin Lemaignan and Caroline Montassier and Pauline Lavalade and Bahar Irfan and Fotios Papadopoulos and Emmanuel Senft and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2017/03/Kennedy2017HRIFull.pdf}, year = {2017}, date = {2017-01-01}, booktitle = {Proceedings of the 12th ACM/IEEE International Conference on Human-Robot Interaction}, address = {Vienna, Austria}, abstract = {An increasing number of human-robot interaction (HRI) studies are now taking place in applied settings with chil- dren. These interactions often hinge on verbal interaction to effectively achieve their goals. Great advances have been made in adult speech recognition and it is often assumed that these advances will carry over to the HRI domain and to interactions with children. In this paper, we evaluate a number of automatic speech recognition (ASR) engines un- der a variety of conditions, inspired by real-world social HRI conditions. Using the data collected we demonstrate that there is still much work to be done in ASR for child speech, with interactions relying solely on this modality still out of reach. However, we also make recommendations for child- robot interaction design in order to maximise the capability that does currently exist.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } An increasing number of human-robot interaction (HRI) studies are now taking place in applied settings with chil- dren. These interactions often hinge on verbal interaction to effectively achieve their goals. Great advances have been made in adult speech recognition and it is often assumed that these advances will carry over to the HRI domain and to interactions with children. In this paper, we evaluate a number of automatic speech recognition (ASR) engines un- der a variety of conditions, inspired by real-world social HRI conditions. Using the data collected we demonstrate that there is still much work to be done in ASR for child speech, with interactions relying solely on this modality still out of reach. However, we also make recommendations for child- robot interaction design in order to maximise the capability that does currently exist. |
Kennedy, James; Baxter, Paul; Belpaeme, Tony Nonverbal Immediacy as a Characterisation of Social Behaviour for Human-Robot Interaction Journal Article In: International Journal of Social Robotics, 9 (1), pp. 109–128, 2017. @article{Kennedy2017b, title = {Nonverbal Immediacy as a Characterisation of Social Behaviour for Human-Robot Interaction}, author = { James Kennedy and Paul Baxter and Tony Belpaeme}, url = {http://bit.ly/2m1v8Pm}, doi = {10.1007/s12369-016-0378-3}, year = {2017}, date = {2017-01-01}, journal = {International Journal of Social Robotics}, volume = {9}, number = {1}, pages = {109--128}, publisher = {Springer}, abstract = {An increasing amount of research has started toexplore the impact of robot social behaviour on the outcomeof a goal for a human interaction partner, such as cognitivelearning gains. However, it remains unclear from what prin-ciples the social behaviour for such robots should be derived.Human models are often used, but in this paper an alterna-tive approach is proposed. First, the concept of nonverbalimmediacy from the communication literature is introduced,with a focus on how it can provide a characterisation of socialbehaviour, and the subsequent outcomes of such behaviour. Aliterature review is conducted to explore the impact on learn-ing of the social cues which form the nonverbal immediacymeasure. This leads to the production of a series of guide-lines for social robot behaviour. The resulting behaviour isevaluated in a more general context, where both children andadults judge the immediacy of humans and robots in a similarmanner, and their recall of a short story is tested. Childrenrecall more of the story when the robot is more immedi-ate, which demonstrates an effect predicted by the literature.This study provides validation for the application of nonver-bal immediacy to child–robot interaction. It is proposed thatnonverbal immediacy measures could be used as a means ofcharacterising robot social behaviour for human–robot interaction.}, keywords = {}, pubstate = {published}, tppubtype = {article} } An increasing amount of research has started toexplore the impact of robot social behaviour on the outcomeof a goal for a human interaction partner, such as cognitivelearning gains. However, it remains unclear from what prin-ciples the social behaviour for such robots should be derived.Human models are often used, but in this paper an alterna-tive approach is proposed. First, the concept of nonverbalimmediacy from the communication literature is introduced,with a focus on how it can provide a characterisation of socialbehaviour, and the subsequent outcomes of such behaviour. Aliterature review is conducted to explore the impact on learn-ing of the social cues which form the nonverbal immediacymeasure. This leads to the production of a series of guide-lines for social robot behaviour. The resulting behaviour isevaluated in a more general context, where both children andadults judge the immediacy of humans and robots in a similarmanner, and their recall of a short story is tested. Childrenrecall more of the story when the robot is more immedi-ate, which demonstrates an effect predicted by the literature.This study provides validation for the application of nonver-bal immediacy to child–robot interaction. It is proposed thatnonverbal immediacy measures could be used as a means ofcharacterising robot social behaviour for human–robot interaction. |
Senft, Emmanuel; Lemaignan, Severin; Baxter, Paul E; Belpaeme, Tony Leveraging Human Inputs in Interactive Machine Learning for Human Robot Interaction Inproceedings In: Proceedings of the 12th ACM/IEEE International Conference on Human-Robot Interaction, 2017. @inproceedings{senft2017leveraging, title = {Leveraging Human Inputs in Interactive Machine Learning for Human Robot Interaction}, author = { Emmanuel Senft and Severin Lemaignan and Paul E Baxter and Tony Belpaeme }, url = {http://www.dream2020.eu/wp-content/uploads/2017/03/2017-ES-HRI_LBR.pdf}, year = {2017}, date = {2017-01-01}, booktitle = {Proceedings of the 12th ACM/IEEE International Conference on Human-Robot Interaction}, series = {HRI '17}, abstract = {A key challenge of HRI is allowing robots to be adaptable, especially as robots are expected to penetrate society at large and to interact in unexpected environments with non- technical users. One way of providing this adaptability is to use Interactive Machine Learning, i.e. having a human supervisor included in the learning process who can steer the action selection and the learning in the desired direction. We ran a study exploring how people use numeric rewards to evaluate a robot’s behaviour and guide its learning. From the results we derive a number of challenges when design- ing learning robots: what kind of input should the human provide? How should the robot communicate its state or its intention? And how can the teaching process by made easier for human supervisors?}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } A key challenge of HRI is allowing robots to be adaptable, especially as robots are expected to penetrate society at large and to interact in unexpected environments with non- technical users. One way of providing this adaptability is to use Interactive Machine Learning, i.e. having a human supervisor included in the learning process who can steer the action selection and the learning in the desired direction. We ran a study exploring how people use numeric rewards to evaluate a robot’s behaviour and guide its learning. From the results we derive a number of challenges when design- ing learning robots: what kind of input should the human provide? How should the robot communicate its state or its intention? And how can the teaching process by made easier for human supervisors? |
Kennedy, James; Baxter, Paul; Belpaeme, Tony The Impact of Robot Tutor Nonverbal Social Behaviour on Child Learning Journal Article In: Frontiers in ICT, 4 (6), 2017, ISSN: 2297-198X. @article{kennedy2017tutor, title = {The Impact of Robot Tutor Nonverbal Social Behaviour on Child Learning}, author = { James Kennedy and Paul Baxter and Tony Belpaeme}, url = {http://journal.frontiersin.org/article/10.3389/fict.2017.00006/abstract http://www.dream2020.eu/wp-content/uploads/2017/04/251879_Kennedy_ProvisionalPDF.pdf}, doi = {10.3389/fict.2017.00006}, issn = {2297-198X}, year = {2017}, date = {2017-01-01}, journal = {Frontiers in ICT}, volume = {4}, number = {6}, publisher = {Frontiers Media}, abstract = {Several studies have indicated that interacting with social robots in educational contexts may lead to greater learning than interactions with computers or virtual agents. As such, an increasing amount of social human-robot interaction research is being conducted in the learning domain, particularly with children. However, it is unclear precisely what social behaviour a robot should employ in such interactions. Inspiration can be taken from human-human studies; this often leads to an assumption that the more social behaviour an agent utilises, the better the learning outcome will be. We apply a nonverbal behaviour metric to a series of studies in which children are taught how to identify prime numbers by a robot with various behavioural manipulations. We find a trend which generally agrees with the pedagogy literature, but also that overt nonverbal behaviour does not account for all learning differences. We discuss the impact of novelty, child expectations, and responses to social cues to further the understanding of the relationship between robot social behaviour and learning. We suggest that the combination of nonverbal behaviour and social cue congruency is necessary to facilitate learning.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Several studies have indicated that interacting with social robots in educational contexts may lead to greater learning than interactions with computers or virtual agents. As such, an increasing amount of social human-robot interaction research is being conducted in the learning domain, particularly with children. However, it is unclear precisely what social behaviour a robot should employ in such interactions. Inspiration can be taken from human-human studies; this often leads to an assumption that the more social behaviour an agent utilises, the better the learning outcome will be. We apply a nonverbal behaviour metric to a series of studies in which children are taught how to identify prime numbers by a robot with various behavioural manipulations. We find a trend which generally agrees with the pedagogy literature, but also that overt nonverbal behaviour does not account for all learning differences. We discuss the impact of novelty, child expectations, and responses to social cues to further the understanding of the relationship between robot social behaviour and learning. We suggest that the combination of nonverbal behaviour and social cue congruency is necessary to facilitate learning. |
Esteban, {Pablo G }; Baxter, Paul; Belpaeme, Tony; Billing, Erik; Cai, Haibin; Cao, Hoang-Long; Coeckelbergh, Mark; Costescu, Cristina; David, Daniel; Beir, {Albert De} ; Fang, Yinfeng; Ju, Zhaojie; Kennedy, James; Liu, Honghai; Mazel, Alexandre; Pandey, Amit; Richardson, Kathleen; Senft, Emmanuel; Thill, Serge; de} Perre, {Greet Van ; Vanderborght, Bram; Vernon, David; Yu, Hui; Ziemke, Tom How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder Journal Article In: Paladyn: journal of behavioral robotics, pp. 18-38, 2017, ISSN: 2081-4836. @article{Esteban2017, title = {How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder}, author = {Esteban, {Pablo G.} and Paul Baxter and Tony Belpaeme and Erik Billing and Haibin Cai and Hoang-Long Cao and Mark Coeckelbergh and Cristina Costescu and Daniel David and Beir, {Albert De} and Yinfeng Fang and Zhaojie Ju and James Kennedy and Honghai Liu and Alexandre Mazel and Amit Pandey and Kathleen Richardson and Emmanuel Senft and Serge Thill and Perre, {Greet Van de} and Bram Vanderborght and David Vernon and Hui Yu and Tom Ziemke}, url = {http://www.dream2020.eu/wp-content/uploads/2017/05/Paladyn-Journal-of-Behavioral-Robotics-How-to-Build-a-Supervised-Autonomous-System-for-Robot-Enhanced-Therapy-for-Children-with-Autism-Spectrum-Disorder.pdf}, issn = {2081-4836}, year = {2017}, date = {2017-01-01}, journal = {Paladyn: journal of behavioral robotics}, pages = {18-38}, abstract = {Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD) through remote control of the robot in so-called Wizard of Oz (WoZ) paradigms. However, there is a need to increase the autonomy of the robot both to lighten the burden on human therapists (who have to remain in control and, importantly, supervise the robot) and to provide a consistent therapeutic experience. This paper seeks to provide insight into increasing the autonomy level of social robots in therapy to move beyond WoZ. With the final aim of improved human-human social interaction for the children, this multidisciplinary research seeks to facilitate the use of social robots as tools in clinical situations by addressing the challenge of increasing robot autonomy. We introduce the clinical framework in which the developments are tested, alongside initial data obtained from patients in a first phase of the project using a WoZ set-up mimicking the targeted supervised-autonomy behaviour. We further describe the implemented system architecture capable of providing the robot with supervised autonomy.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD) through remote control of the robot in so-called Wizard of Oz (WoZ) paradigms. However, there is a need to increase the autonomy of the robot both to lighten the burden on human therapists (who have to remain in control and, importantly, supervise the robot) and to provide a consistent therapeutic experience. This paper seeks to provide insight into increasing the autonomy level of social robots in therapy to move beyond WoZ. With the final aim of improved human-human social interaction for the children, this multidisciplinary research seeks to facilitate the use of social robots as tools in clinical situations by addressing the challenge of increasing robot autonomy. We introduce the clinical framework in which the developments are tested, alongside initial data obtained from patients in a first phase of the project using a WoZ set-up mimicking the targeted supervised-autonomy behaviour. We further describe the implemented system architecture capable of providing the robot with supervised autonomy. |
Hoang-Long, Cao; Albert, De Beir; 'o, G; Ramona, Simut; de Greet, Van Perre; Dirk, Lefeber; Bram, Vanderborght An End-User Interface to Generate Homeostatic Behavior for NAO Robot in Robot-Assisted Social Therapies Book Chapter In: Rojas, Ignacio ; Joya, Gonzalo ; Catala, Andreu (Ed.): pp. 609–619, Springer International Publishing, Cham, 2017, ISBN: 978-3-319-59147-6. @inbook{Cao2017, title = {An End-User Interface to Generate Homeostatic Behavior for NAO Robot in Robot-Assisted Social Therapies}, author = {Cao Hoang-Long and De Beir Albert and G{'o}mez Esteban Pablo and Simut Ramona and Van de Perre Greet and Lefeber Dirk and Vanderborght Bram }, editor = {Rojas, Ignacio and Joya, Gonzalo and Catala, Andreu}, url = {http://dx.doi.org/10.1007/978-3-319-59147-6_52 https://link.springer.com/chapter/10.1007/978-3-319-59147-6_52}, doi = {10.1007/978-3-319-59147-6_52}, isbn = {978-3-319-59147-6}, year = {2017}, date = {2017-01-01}, pages = {609--619}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {Homeostatic drive theory is a popular approach for decision-making of robot behavior in social robotic research. It is potentially to be used in social therapies. To increase the involvement of end-users in the robot's control, we present an end-user interface allowing the therapists to generate homeostatic behavior for NAO robot in social skills training for children. We demonstrate the system by two interactions in which the robot homeostatic behavior is adapted to children's behavior. The result shows that the system provides a practical solution for therapists to implement interaction scenarios to robot behavior.}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } Homeostatic drive theory is a popular approach for decision-making of robot behavior in social robotic research. It is potentially to be used in social therapies. To increase the involvement of end-users in the robot's control, we present an end-user interface allowing the therapists to generate homeostatic behavior for NAO robot in social skills training for children. We demonstrate the system by two interactions in which the robot homeostatic behavior is adapted to children's behavior. The result shows that the system provides a practical solution for therapists to implement interaction scenarios to robot behavior. |
Liu, Bangli; Yu, H; Zhou, X; Tang, Dan; Liu, H Combining 3D joints Moving Trend and Geometry property for human action recognition Inproceedings In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 000332-000337, 2016. @inproceedings{Liu2016, title = {Combining 3D joints Moving Trend and Geometry property for human action recognition}, author = {Bangli Liu and H Yu and X Zhou and Dan Tang and H Liu}, url = {https://ieeexplore.ieee.org/document/7844262}, doi = {10.1109/SMC.2016.7844262}, year = {2016}, date = {2016-08-06}, booktitle = {2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC)}, pages = {000332-000337}, abstract = {Depth image based human action recognition has attracted many attentions due to the popularity of the depth sensors. However, accurate recognition still remains a challenge because of various object appearances, poses and video sequences. In this paper, a novel skeleton joints descriptor based on 3D Moving Trend and Geometry (3DMTG) property is proposed for human action recognition. Specifically, a histogram of 3D moving directions between consecutive frames for each joint is constructed to represent the 3D moving trend feature in spatial domain. The geometry information of joints in each frame is modelled by the relative motion with the initial status. The proposed feature descriptor is evaluated on two popular datasets. The experimental results demonstrate the superior performance of our method over the state-of-the-art methods, especially the higher recognition rates for complex actions.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Depth image based human action recognition has attracted many attentions due to the popularity of the depth sensors. However, accurate recognition still remains a challenge because of various object appearances, poses and video sequences. In this paper, a novel skeleton joints descriptor based on 3D Moving Trend and Geometry (3DMTG) property is proposed for human action recognition. Specifically, a histogram of 3D moving directions between consecutive frames for each joint is constructed to represent the 3D moving trend feature in spatial domain. The geometry information of joints in each frame is modelled by the relative motion with the initial status. The proposed feature descriptor is evaluated on two popular datasets. The experimental results demonstrate the superior performance of our method over the state-of-the-art methods, especially the higher recognition rates for complex actions. |
Toda, Yuichiro; Ju, Zhaojie; Yu, Hui; Takesue, Naoyuki; Wada, Kazuyoshi; Kubota, Naoyuki Real-time 3D point cloud segmentation using Growing Neural Gas with Utility Inproceedings In: 2016 9th International Conference on Human System Interactions (HSI), pp. 418–422, IEEE, 2016, ISBN: 978-1-5090-1730-0. @inproceedings{Toda2016, title = {Real-time 3D point cloud segmentation using Growing Neural Gas with Utility}, author = {Yuichiro Toda and Zhaojie Ju and Hui Yu and Naoyuki Takesue and Kazuyoshi Wada and Naoyuki Kubota}, url = {https://researchportal.port.ac.uk/portal/en/publications/realtime-3d-point-cloud-segmentation-using-growing-neural-gas-with-utility(c4759a05-12f9-4ae0-9003-859640d0e3f9).html http://www.dream2020.eu/wp-content/uploads/2019/05/Real_time_3D_point_cloud_segmentation_using_Growing_Neural_Gas_with_Utility.pdf}, doi = {10.1109/HSI.2016.7529667}, isbn = {978-1-5090-1730-0}, year = {2016}, date = {2016-04-08}, booktitle = {2016 9th International Conference on Human System Interactions (HSI)}, pages = {418--422}, publisher = {IEEE}, abstract = {This paper proposes a real-time feature extraction and segmentation method for a 3D point cloud. First of all, we apply Growing Neural Gas with Utility (GNG-U) to the point cloud for learning a topological structure. However, the standard GNG-U cannot learn the topological structure of 3D space environment and color information simultaneously. To this end, we then modify the GNG-U algorithm by using a weight vector. we propose a surface feature extraction and segmentation method by efficiently utilizing the topological structure. Our segmentation method is based on a region growing method whose similarity value uses the inner value of two normal vectors connected by the topological structure. We show experimental results of the proposed method and discuss the effectiveness of the proposed method.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } This paper proposes a real-time feature extraction and segmentation method for a 3D point cloud. First of all, we apply Growing Neural Gas with Utility (GNG-U) to the point cloud for learning a topological structure. However, the standard GNG-U cannot learn the topological structure of 3D space environment and color information simultaneously. To this end, we then modify the GNG-U algorithm by using a weight vector. we propose a surface feature extraction and segmentation method by efficiently utilizing the topological structure. Our segmentation method is based on a region growing method whose similarity value uses the inner value of two normal vectors connected by the topological structure. We show experimental results of the proposed method and discuss the effectiveness of the proposed method. |
Phiri, Charles C; Ju, Zhaojie; Liu, Honghai Accelerating Humanoid Robot Learning from Human Action Skills Using Context-Aware Middleware Inproceedings In: Kubota, Naoyuki; Kiguchi, Kazuo; Liu, Honghai; Obo, Takenori (Ed.): Intelligent Robotics and Applications, pp. 563–574, Springer International Publishing, Cham, 2016, ISBN: 978-3-319-43506-0. @inproceedings{Phiri2016b, title = {Accelerating Humanoid Robot Learning from Human Action Skills Using Context-Aware Middleware}, author = {Charles C Phiri and Zhaojie Ju and Honghai Liu}, editor = {Naoyuki Kubota and Kazuo Kiguchi and Honghai Liu and Takenori Obo}, url = {https://link.springer.com/chapter/10.1007/978-3-319-43506-0_49}, isbn = {978-3-319-43506-0}, year = {2016}, date = {2016-03-08}, booktitle = {Intelligent Robotics and Applications}, pages = {563--574}, publisher = {Springer International Publishing}, address = {Cham}, abstract = {In this paper we propose the creation of context-aware middleware to solve the challenge of integrating disparate incompatible systems involved in the teaching of human action skills to robots. Context-aware middleware provides the solution to retrofitting capabilities onto existing robots (agents) and bridges the technology differences between systems. The experimental results demonstrate a framework for handling situational and contextual data for robot Learning from Demonstration.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In this paper we propose the creation of context-aware middleware to solve the challenge of integrating disparate incompatible systems involved in the teaching of human action skills to robots. Context-aware middleware provides the solution to retrofitting capabilities onto existing robots (agents) and bridges the technology differences between systems. The experimental results demonstrate a framework for handling situational and contextual data for robot Learning from Demonstration. |
Cai, Haibin; Yu, Hui; Zhou, Xiaolong; Liu, Honghai Robust gaze estimation via normalized iris center-eye corner vector Inproceedings In: Kubota, Naoyuki; Kiguchi, Kazuo; Liu, Honghai; Obo, Takenori (Ed.): Intelligent Robotics and Applications, pp. 300–309, Springer, 2016. @inproceedings{Cai2016, title = {Robust gaze estimation via normalized iris center-eye corner vector}, author = {Haibin Cai and Hui Yu and Xiaolong Zhou and Honghai Liu}, editor = {Naoyuki Kubota and Kazuo Kiguchi and Honghai Liu and Takenori Obo}, url = {https://researchportal.port.ac.uk/portal/en/publications/robust-gaze-estimation-via-normalized-iris-centereye-corner-vector(bbd75d75-d7b0-41ba-9281-f90e70d393e2).html http://www.dream2020.eu/wp-content/uploads/2019/05/Robust_Gaze_ICIRA2016.pdf}, doi = {10.1007/978-3-319-43506-0_26}, year = {2016}, date = {2016-03-08}, booktitle = {Intelligent Robotics and Applications}, pages = {300--309}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, abstract = {Gaze estimation plays an important role in many practical scenarios such as human robot interaction. Although high accurate gaze estimation could be obtained in constrained settings with additional IR sources or depth sensors, single web-cam based gaze estimation still remains challenging. This paper propose a normalized iris center-eye corner (NIC-EC) vector based gaze estimation methods using a single, low cost web-cam. Firstly, reliable facial features and pupil centers are extracted. Then, the NIC-EC vector is proposed to enhance the robustness and accuracy for pupil center-eye corner vector based gaze estimations. Finally, an interpolation method is employed for the mapping between constructed vectors and points of regard. Experimental results showed that the proposed method has significantly improved the accuracy over the pupil center-eye corner vector based gaze estimation method with average accuracy of 1.66∘ under slight head movements.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Gaze estimation plays an important role in many practical scenarios such as human robot interaction. Although high accurate gaze estimation could be obtained in constrained settings with additional IR sources or depth sensors, single web-cam based gaze estimation still remains challenging. This paper propose a normalized iris center-eye corner (NIC-EC) vector based gaze estimation methods using a single, low cost web-cam. Firstly, reliable facial features and pupil centers are extracted. Then, the NIC-EC vector is proposed to enhance the robustness and accuracy for pupil center-eye corner vector based gaze estimations. Finally, an interpolation method is employed for the mapping between constructed vectors and points of regard. Experimental results showed that the proposed method has significantly improved the accuracy over the pupil center-eye corner vector based gaze estimation method with average accuracy of 1.66∘ under slight head movements. |
Baxter, Paul Memory-centred cognitive architectures for robots interacting socially with humans Inproceedings In: Proceedings of the 2nd Workshop on Cognitive Architecture for Social Human-Robot Interaction, at HRI'16, Christchurch, New Zealand, 2016. @inproceedings{baxter2016memory, title = {Memory-centred cognitive architectures for robots interacting socially with humans}, author = { Paul Baxter}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Baxter2016Memory.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 2nd Workshop on Cognitive Architecture for Social Human-Robot Interaction, at HRI'16}, address = {Christchurch, New Zealand}, abstract = {The Memory-Centred Cognition perspective places an active association substrate at the heart of cognition, rather than as a passive adjunct. Consequently, it places prediction and priming on the basis of prior experience to be inherent and fundamental aspects of processing. Social interaction is taken here to minimally require contingent and co-adaptive behaviours from the interacting parties. In this contribution, I seek to show how the memory-centred cognition approach to cognitive architectures can provide an means of addressing these functions. A number of example implementations are briefly reviewed, particularly focusing on multi-modal alignment as a function of experience-based priming. While there is further refinement required to the theory, and implementations based thereon, this approach provides an interesting alternative perspective on the foundations of cognitive architectures to support robots engage in social interactions with humans.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The Memory-Centred Cognition perspective places an active association substrate at the heart of cognition, rather than as a passive adjunct. Consequently, it places prediction and priming on the basis of prior experience to be inherent and fundamental aspects of processing. Social interaction is taken here to minimally require contingent and co-adaptive behaviours from the interacting parties. In this contribution, I seek to show how the memory-centred cognition approach to cognitive architectures can provide an means of addressing these functions. A number of example implementations are briefly reviewed, particularly focusing on multi-modal alignment as a function of experience-based priming. While there is further refinement required to the theory, and implementations based thereon, this approach provides an interesting alternative perspective on the foundations of cognitive architectures to support robots engage in social interactions with humans. |
Baxter, Paul; Lemaignan, Séverin; Trafton, Gregory J Cognitive Architectures for Social Human-Robot Interaction (workshop abstract) Inproceedings In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction, Christchurch, New Zealand, 2016. @inproceedings{baxter2016cognitive, title = {Cognitive Architectures for Social Human-Robot Interaction (workshop abstract)}, author = { Paul Baxter and Séverin Lemaignan and J. Gregory Trafton}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction}, address = {Christchurch, New Zealand}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Baxter, Paul; Belpaeme, Tony A cautionary note on personality (extroversion) assessments in child-robot interaction studies Inproceedings In: Proceedings of the 2nd Workshop on Evaluating Child-Robot Interaction, at HRI'16, Christchurch, New Zealand, 2016. @inproceedings{baxter2016cautionary, title = {A cautionary note on personality (extroversion) assessments in child-robot interaction studies}, author = { Paul Baxter and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Baxter2016Cautionary.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 2nd Workshop on Evaluating Child-Robot Interaction, at HRI'16}, address = {Christchurch, New Zealand}, abstract = {The relationship between personality and social human-robot interaction is a topic of increasing interest. There are further some indications from the literature that there is an association between personality dimensions and various aspects of educational behaviour and performance. This brief contribution seeks to explore the single personality dimension of extroversion/introversion: specifically, how children rate them- selves with a validated questionnaire in comparison to how teachers rate them using a relative scale. In an exploratory study conducted in a primary school, we find a non-significant association between these two ratings. We suggest that this mismatch is related to the context in which the respective ratings were made. In order to facilitate generalisation of personality- related results across studies, we propose two general reporting recommendations. Based on our results, we suggest that the application of personality assessments in a child-robot interaction context may be more complex than initially envisaged, with some dependence on context.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The relationship between personality and social human-robot interaction is a topic of increasing interest. There are further some indications from the literature that there is an association between personality dimensions and various aspects of educational behaviour and performance. This brief contribution seeks to explore the single personality dimension of extroversion/introversion: specifically, how children rate them- selves with a validated questionnaire in comparison to how teachers rate them using a relative scale. In an exploratory study conducted in a primary school, we find a non-significant association between these two ratings. We suggest that this mismatch is related to the context in which the respective ratings were made. In order to facilitate generalisation of personality- related results across studies, we propose two general reporting recommendations. Based on our results, we suggest that the application of personality assessments in a child-robot interaction context may be more complex than initially envisaged, with some dependence on context. |
Kennedy, James; Baxter, Paul; Senft, Emmanuel; Belpaeme, Tony Heart vs Hard Drive: Children Learn More From a Human Tutor Than a Social Robot Inproceedings In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction, pp. 451–452, Christchurch, New Zealand, 2016. @inproceedings{kennedy2016heart, title = {Heart vs Hard Drive: Children Learn More From a Human Tutor Than a Social Robot}, author = { James Kennedy and Paul Baxter and Emmanuel Senft and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Kennedy2016HeartvsHard.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction}, pages = {451--452}, address = {Christchurch, New Zealand}, series = {HRI '16}, abstract = {The field of Human-Robot Interaction (HRI) is increasingly exploring the use of social robots for educating children. Commonly, non-academic audiences will ask how robots compare to humans in terms of learning outcomes. This question is also interesting for social roboticists as humans are often assumed to be an upper benchmark for social behaviour, which influences learning. This paper presents a study in which learning gains of children are compared when taught the same math- ematics material by a robot tutor and a non-expert human tutor. Significant learning occurs in both conditions, but the children improve more with the human tutor. This difference is not statistically significant, but the effect sizes fall in line with findings from other literature showing that humans outperform technology for tutoring. We discuss these findings in the context of applying social robots in child education.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The field of Human-Robot Interaction (HRI) is increasingly exploring the use of social robots for educating children. Commonly, non-academic audiences will ask how robots compare to humans in terms of learning outcomes. This question is also interesting for social roboticists as humans are often assumed to be an upper benchmark for social behaviour, which influences learning. This paper presents a study in which learning gains of children are compared when taught the same math- ematics material by a robot tutor and a non-expert human tutor. Significant learning occurs in both conditions, but the children improve more with the human tutor. This difference is not statistically significant, but the effect sizes fall in line with findings from other literature showing that humans outperform technology for tutoring. We discuss these findings in the context of applying social robots in child education. |
Kennedy, James; Baxter, Paul; Senft, Emmanuel; Belpaeme, Tony Social Robot Tutoring for Child Second Language Learning Inproceedings In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction, pp. 231–238, Christchurch, New Zealand, 2016. @inproceedings{kennedy2016language, title = {Social Robot Tutoring for Child Second Language Learning}, author = { James Kennedy and Paul Baxter and Emmanuel Senft and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Kennedy2016SocialRobot.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction}, pages = {231--238}, address = {Christchurch, New Zealand}, series = {HRI '16}, abstract = {An increasing amount of research is being conducted to determine how a robot tutor should behave socially in educa- tional interactions with children. Both human-human and human- robot interaction literature predicts an increase in learning with increased social availability of a tutor, where social availability has verbal and nonverbal components. Prior work has shown that greater availability in the nonverbal behaviour of a robot tutor has a positive impact on child learning. This paper presents a study with 67 children to explore how social aspects of a tutor robot’s speech influences their perception of the robot and their language learning in an interaction. Children perceive the difference in social behaviour between ‘low’ and ‘high’ verbal availability conditions, and improve significantly between a pre- and a post-test in both conditions. A longer-term retention test taken the following week showed that the children had retained almost all of the information they had learnt. However, learning was not affected by which of the robot behaviours they had been exposed to. It is suggested that in this short-term interaction context, additional effort in developing social aspects of a robot’s verbal behaviour may not return the desired positive impact on learning gains.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } An increasing amount of research is being conducted to determine how a robot tutor should behave socially in educa- tional interactions with children. Both human-human and human- robot interaction literature predicts an increase in learning with increased social availability of a tutor, where social availability has verbal and nonverbal components. Prior work has shown that greater availability in the nonverbal behaviour of a robot tutor has a positive impact on child learning. This paper presents a study with 67 children to explore how social aspects of a tutor robot’s speech influences their perception of the robot and their language learning in an interaction. Children perceive the difference in social behaviour between ‘low’ and ‘high’ verbal availability conditions, and improve significantly between a pre- and a post-test in both conditions. A longer-term retention test taken the following week showed that the children had retained almost all of the information they had learnt. However, learning was not affected by which of the robot behaviours they had been exposed to. It is suggested that in this short-term interaction context, additional effort in developing social aspects of a robot’s verbal behaviour may not return the desired positive impact on learning gains. |
Baxter, Paul; Kennedy, James; Senft, Emmanuel; Lemaignan, Séverin; Belpaeme, Tony From Characterising Three Years of HRI to Methodology and Reporting Recommendations Inproceedings In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (alt.HRI), pp. 391–398, Christchurch, New Zealand, 2016. @inproceedings{baxter2016althri, title = {From Characterising Three Years of HRI to Methodology and Reporting Recommendations}, author = { Paul Baxter and James Kennedy and Emmanuel Senft and Séverin Lemaignan and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Baxter2016AltHRI.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (alt.HRI)}, pages = {391--398}, address = {Christchurch, New Zealand}, series = {HRI '16}, abstract = {Human-Robot Interaction (HRI) research requires the integration and cooperation of multiple disciplines, technical and social, in order to make progress. In many cases using different motivations, each of these disciplines bring with them different assumptions and methodologies. We assess recent trends in the field of HRI by examining publications in the HRI conference over the past three years (over 100 full papers), and characterise them according to 14 categories. We focus primarily on aspects of methodology. From this, a series of practical rec- ommendations based on rigorous guidelines from other research fields that have not yet become common practice in HRI are proposed. Furthermore, we explore the primary implications of the observed recent trends for the field more generally, in terms of both methodology and research directions. We propose that the interdisciplinary nature of HRI must be maintained, but that a common methodological approach provides a much needed frame of reference to facilitate rigorous future progress.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Human-Robot Interaction (HRI) research requires the integration and cooperation of multiple disciplines, technical and social, in order to make progress. In many cases using different motivations, each of these disciplines bring with them different assumptions and methodologies. We assess recent trends in the field of HRI by examining publications in the HRI conference over the past three years (over 100 full papers), and characterise them according to 14 categories. We focus primarily on aspects of methodology. From this, a series of practical rec- ommendations based on rigorous guidelines from other research fields that have not yet become common practice in HRI are proposed. Furthermore, we explore the primary implications of the observed recent trends for the field more generally, in terms of both methodology and research directions. We propose that the interdisciplinary nature of HRI must be maintained, but that a common methodological approach provides a much needed frame of reference to facilitate rigorous future progress. |
Wills, Paul; Baxter, Paul; Kennedy, James; Senft, Emmanuel; Belpaeme, Tony Socially Contingent Humanoid Robot Head Behaviour Results in Increased Charity Donations Inproceedings In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction, pp. 533–534, Christchurch, New Zealand, 2016. @inproceedings{wills2016contingent, title = {Socially Contingent Humanoid Robot Head Behaviour Results in Increased Charity Donations}, author = { Paul Wills and Paul Baxter and James Kennedy and Emmanuel Senft and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Wills2016SociallyContingent.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction}, pages = {533--534}, address = {Christchurch, New Zealand}, series = {HRI '16}, abstract = {The role of robot social behaviour in changing people’s behaviour is an interesting and yet still open question, with the general assumption that social behaviour is beneficial. In this study, we examine the effect of socially contingent robot behaviours on a charity collection task. Manipulating only behavioural cues (maintaining the same verbal content), we show that when the robot exhibits contingent behaviours consistent with those observable in humans, this results in a 32% increase in money collected over a non-reactive robot. These results suggest that apparent social agency on the part of the robot, even when subtle behavioural cues are used, can result in behavioural change on the part of the interacting human.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The role of robot social behaviour in changing people’s behaviour is an interesting and yet still open question, with the general assumption that social behaviour is beneficial. In this study, we examine the effect of socially contingent robot behaviours on a charity collection task. Manipulating only behavioural cues (maintaining the same verbal content), we show that when the robot exhibits contingent behaviours consistent with those observable in humans, this results in a 32% increase in money collected over a non-reactive robot. These results suggest that apparent social agency on the part of the robot, even when subtle behavioural cues are used, can result in behavioural change on the part of the interacting human. |
Senft, Emmanuel; Baxter, Paul; Kennedy, James; Lemaignan, Séverin; Belpaeme, Tony Providing a Robot with Learning Abilities Improves its Perception by Users Inproceedings In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction, pp. 513–514, Christchurch, New Zealand, 2016. @inproceedings{senft2016providing, title = {Providing a Robot with Learning Abilities Improves its Perception by Users}, author = {Emmanuel Senft and Paul Baxter and James Kennedy and Séverin Lemaignan and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Senft2016Providing.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction}, pages = {513--514}, address = {Christchurch, New Zealand}, series = {HRI '16}, abstract = {Subjective appreciation and performance evaluation of a robot by users are two important dimensions for Human- Robot Interaction, especially as increasing numbers of people become involved with robots. As roboticists we have to carefully design robots to make the interaction as smooth and enjoyable as possible for the users, while maintaining good performance in the task assigned to the robot. In this paper, we examine the impact of providing a robot with learning capabilities on how users report the quality of the interaction in relation to objective performance. We show that humans tend to prefer interacting with a learning robot and will rate its capabilities higher even if the actual performance in the task was lower. We suggest that adding learning to a robot could reduce the apparent load felt by a user for a new task and improve the user’s evaluation of the system, thus facilitating the integration of such robots into existing work flows.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Subjective appreciation and performance evaluation of a robot by users are two important dimensions for Human- Robot Interaction, especially as increasing numbers of people become involved with robots. As roboticists we have to carefully design robots to make the interaction as smooth and enjoyable as possible for the users, while maintaining good performance in the task assigned to the robot. In this paper, we examine the impact of providing a robot with learning capabilities on how users report the quality of the interaction in relation to objective performance. We show that humans tend to prefer interacting with a learning robot and will rate its capabilities higher even if the actual performance in the task was lower. We suggest that adding learning to a robot could reduce the apparent load felt by a user for a new task and improve the user’s evaluation of the system, thus facilitating the integration of such robots into existing work flows. |
Simut Vanderborght, Ramona ; Vanderfaeillie, Johan; Cao, Hoang-Long; Lefeber, Dirk; Vanderborght, Bram Are social robots social enough for preschoolers with autism spectrum disorder? Book Chapter In: XI Autism Europe International Conference, 2016. @inbook{dafb1027148e4322912d1baf93c0d2ee, title = {Are social robots social enough for preschoolers with autism spectrum disorder?}, author = {Simut Vanderborght, Ramona and Johan Vanderfaeillie and Hoang-Long Cao and Dirk Lefeber and Bram Vanderborght}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/are-social-robots-social-enough-for-preschoolers-with-autism-spectrum-disorder(dafb1027-148e-4322-912d-1baf93c0d2ee).html}, year = {2016}, date = {2016-01-01}, booktitle = {XI Autism Europe International Conference}, abstract = {Children with autism spectrum disorder (ASD) can often experience an information overload when they interact with people. Therefore, they can find social interaction threatening, making them withdraw, even from family members. It has been declared that robots are able to engage children with ASD in social tasks. These outcomes are hypothesized to be the result of several strengths the robots have, in comparison with human interaction partners. They express human-like behaviors, but these behaviors can be break down in small pieces and presented only partially, repeatable and less intense to the children with ASD. This makes communication, with all its complex and frightening subtleties and nuances, less complicated and more comfortable for them. Aim was to investigate if a social robot is able to trigger joint attention (JA) in young children with ASD. 25 children were exposed to a JA task with respectively, a therapist and a robot. It was examined if the children differed in the way how they follow the gaze, alternate eye-contact, share social information with the two partners. Participants achieved significantly higher JA performance with the human partner compared to the robot. Our outcomes are in contradiction with existing research and suggest to be more cautious in promoting or using robotic tools when training JA for children with ASD. When, how, for which purpose robots can be used in the treatment of children with ASD remains still unclear. Despite of the so declared potential of robots, rigorous studies are needed to provide answers to these questions.}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } Children with autism spectrum disorder (ASD) can often experience an information overload when they interact with people. Therefore, they can find social interaction threatening, making them withdraw, even from family members. It has been declared that robots are able to engage children with ASD in social tasks. These outcomes are hypothesized to be the result of several strengths the robots have, in comparison with human interaction partners. They express human-like behaviors, but these behaviors can be break down in small pieces and presented only partially, repeatable and less intense to the children with ASD. This makes communication, with all its complex and frightening subtleties and nuances, less complicated and more comfortable for them. Aim was to investigate if a social robot is able to trigger joint attention (JA) in young children with ASD. 25 children were exposed to a JA task with respectively, a therapist and a robot. It was examined if the children differed in the way how they follow the gaze, alternate eye-contact, share social information with the two partners. Participants achieved significantly higher JA performance with the human partner compared to the robot. Our outcomes are in contradiction with existing research and suggest to be more cautious in promoting or using robotic tools when training JA for children with ASD. When, how, for which purpose robots can be used in the treatment of children with ASD remains still unclear. Despite of the so declared potential of robots, rigorous studies are needed to provide answers to these questions. |
Cao, Hoang-Long; Gomez Esteban, Pablo ; De Beir, Albert ; Simut Vanderborght, Ramona ; Van De Perre, Greet ; Vanderborght, Bram A platform-independent robot control architecture for multiple therapeutic scenarios Book Chapter In: Proceedings of the Fifth International Symposium on New Frontiers in Human-Robot Interaction, 2016. @inbook{98167b0e4f024137a78692f7cff1c8b3, title = {A platform-independent robot control architecture for multiple therapeutic scenarios}, author = {Hoang-Long Cao and Gomez Esteban, Pablo and De Beir, Albert and Simut Vanderborght, Ramona and Van De Perre, Greet and Bram Vanderborght}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/toward-a-platformindependent-social-behavior-architecture-for-multiple-therapeutic-scenarios(2ff3b2d9-4092-4232-bf8b-a4c700ab3111).html}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the Fifth International Symposium on New Frontiers in Human-Robot Interaction}, abstract = {While social robots are developed to provide assistance to users through social interactions, their behaviors are dominantly pre-programmed and remote-controlled. Despite the numerous robot control architectures being developed, very few offer reutilization opportunities in various therapeutic contexts. To bridge this gap, we propose a robot control architecture to be applied in different scenarios taking into account requirements from both therapeutic and robotic perspectives. As robot behaviors are kept at an abstract level and afterward mapped with the robot's morphology, the proposed architecture accommodates its applicability to a variety of social robot platforms. AB - While social robots are developed to provide assistance to users through social interactions, their behaviors are dominantly pre-programmed and remote-controlled. Despite the numerous robot control architectures being developed, very few offer reutilization opportunities in various therapeutic contexts. To bridge this gap, we propose a robot control architecture to be applied in different scenarios taking into account requirements from both therapeutic and robotic perspectives. As robot behaviors are kept at an abstract level and afterward mapped with the robot's morphology, the proposed architecture accommodates its applicability to a variety of social robot platforms.}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } While social robots are developed to provide assistance to users through social interactions, their behaviors are dominantly pre-programmed and remote-controlled. Despite the numerous robot control architectures being developed, very few offer reutilization opportunities in various therapeutic contexts. To bridge this gap, we propose a robot control architecture to be applied in different scenarios taking into account requirements from both therapeutic and robotic perspectives. As robot behaviors are kept at an abstract level and afterward mapped with the robot's morphology, the proposed architecture accommodates its applicability to a variety of social robot platforms. AB - While social robots are developed to provide assistance to users through social interactions, their behaviors are dominantly pre-programmed and remote-controlled. Despite the numerous robot control architectures being developed, very few offer reutilization opportunities in various therapeutic contexts. To bridge this gap, we propose a robot control architecture to be applied in different scenarios taking into account requirements from both therapeutic and robotic perspectives. As robot behaviors are kept at an abstract level and afterward mapped with the robot's morphology, the proposed architecture accommodates its applicability to a variety of social robot platforms. |
Gomez Esteban, Pablo ; Cao, Hoang-Long; De Beir, Albert ; Van De Perre, Greet ; Lefeber, Dirk; Vanderborght, Bram A multilayer reactive system for robots interacting with children with autism Book Chapter In: Proceedings of the Fifth International Symposium on New Frontiers in Human-Robot Interaction, 2016. @inbook{53260d446f6946369c2661f56f643c3d, title = {A multilayer reactive system for robots interacting with children with autism}, author = {Gomez Esteban, Pablo and Hoang-Long Cao and De Beir, Albert and Van De Perre, Greet and Dirk Lefeber and Bram Vanderborght}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/a-multilayer-reactive-system-for-robots-interacting-with-children-with-autism(53260d44-6f69-4636-9c26-61f56f643c3d).html}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the Fifth International Symposium on New Frontiers in Human-Robot Interaction}, abstract = {There is a lack of autonomy on traditional Robot-Assisted Therapy systems interacting with children with autism. To overcome this limitation a supervised autonomous robot controller is being built. In this paper we present a multilayer reactive system within such controller. The goal of this Reactive system is to allow the robot to appropriately react to the child’s behavior creating the illusion of being alive.}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } There is a lack of autonomy on traditional Robot-Assisted Therapy systems interacting with children with autism. To overcome this limitation a supervised autonomous robot controller is being built. In this paper we present a multilayer reactive system within such controller. The goal of this Reactive system is to allow the robot to appropriately react to the child’s behavior creating the illusion of being alive. |
Peca, Andreea; Simut Vanderborght, Ramona ; Cao, Hoang-Long; Vanderborght, Bram Do infants perceive the social robot Keepon as a communicative partner? Journal Article In: Infant Behavior and Development, 42 , pp. 157 - 167, 2016, ISSN: 0163-6383. @article{Peca2016157, title = {Do infants perceive the social robot Keepon as a communicative partner?}, author = {Andreea Peca and Ramona { Simut Vanderborght } and Hoang-Long Cao and Bram Vanderborght}, url = {http://www.sciencedirect.com/science/article/pii/S0163638315300138}, doi = {http://dx.doi.org/10.1016/j.infbeh.2015.10.005}, issn = {0163-6383}, year = {2016}, date = {2016-01-01}, journal = {Infant Behavior and Development}, volume = {42}, pages = {157 - 167}, abstract = {Abstract This study investigates if infants perceive an unfamiliar agent, such as the robot Keepon, as a social agent after observing an interaction between the robot and a human adult. 23 infants, aged 9–17 month, were exposed, in a first phase, to either a contingent interaction between the active robot and an active human adult, or to an interaction between an active human adult and the non-active robot, followed by a second phase, in which infants were offered the opportunity to initiate a turn-taking interaction with Keepon. The measured variables were: (1) the number of social initiations the infant directed toward the robot, and (2) the number of anticipatory orientations of attention to the agent that follows in the conversation. The results indicate a significant higher level of initiations in the interactive robot condition compared to the non-active robot condition, while the difference between the frequencies of anticipations of turn-taking behaviors was not significant.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Abstract This study investigates if infants perceive an unfamiliar agent, such as the robot Keepon, as a social agent after observing an interaction between the robot and a human adult. 23 infants, aged 9–17 month, were exposed, in a first phase, to either a contingent interaction between the active robot and an active human adult, or to an interaction between an active human adult and the non-active robot, followed by a second phase, in which infants were offered the opportunity to initiate a turn-taking interaction with Keepon. The measured variables were: (1) the number of social initiations the infant directed toward the robot, and (2) the number of anticipatory orientations of attention to the agent that follows in the conversation. The results indicate a significant higher level of initiations in the interactive robot condition compared to the non-active robot condition, while the difference between the frequencies of anticipations of turn-taking behaviors was not significant. |
Beir, Albert De; Vanderboght, Bram Evolutionary Method for Robot Morphology: Case Study of Social Robot Probo Inproceedings In: The Eleventh ACM/IEEE International Conference on Human Robot Interaction, pp. 609–610, IEEE Press, Christchurch, New Zealand, 2016, ISBN: 978-1-4673-8370-7. @inproceedings{DeBeir:2016:EMR:2906831.2907005, title = {Evolutionary Method for Robot Morphology: Case Study of Social Robot Probo}, author = { Albert De Beir and Bram Vanderboght}, url = {http://dl.acm.org/citation.cfm?id=2906831.2907005}, isbn = {978-1-4673-8370-7}, year = {2016}, date = {2016-01-01}, booktitle = {The Eleventh ACM/IEEE International Conference on Human Robot Interaction}, pages = {609--610}, publisher = {IEEE Press}, address = {Christchurch, New Zealand}, series = {HRI '16}, abstract = {The appearance of robots is often made arbitrary as it relies more on guidelines than on a rigorous methodology. This paper presents a novel method of using genetic algorithms (GA) to improve the appearance of social robots with human feedback. Such general methods are interesting as they do not require prior artistic experience from the designer and can integrate the end-user in the loop. As a proof of concept, we carry out a case study by applying this method to the new design of the social robot Probo. Using designer feedback, the robot is evolved from its original design over five populations composed of 15 individuals. An online survey shows that the evolved designs are significantly improved compared to the original. These results indicate the feasibility of the method employed and gives rise to the possibility of non-technical end-users influencing the design of robot morphologies adapted for specific human-robot interaction requirements.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The appearance of robots is often made arbitrary as it relies more on guidelines than on a rigorous methodology. This paper presents a novel method of using genetic algorithms (GA) to improve the appearance of social robots with human feedback. Such general methods are interesting as they do not require prior artistic experience from the designer and can integrate the end-user in the loop. As a proof of concept, we carry out a case study by applying this method to the new design of the social robot Probo. Using designer feedback, the robot is evolved from its original design over five populations composed of 15 individuals. An online survey shows that the evolved designs are significantly improved compared to the original. These results indicate the feasibility of the method employed and gives rise to the possibility of non-technical end-users influencing the design of robot morphologies adapted for specific human-robot interaction requirements. |
Peca, Andreea; Coeckelbergh, Mark; Simut Vanderborght, Ramona ; Pop, Cristina; Pintea, Sebastian; David, Daniel D; Vanderborght, Bram A scale measuring ethical acceptability of Robot Enhanced Therapy for children with Autism Spectrum Disorders Journal Article In: IEEE Technology And Society Magazine, 2016, ISSN: 1932-4529. @article{7bea31308aaf4ecba4f75ba37e701390, title = {A scale measuring ethical acceptability of Robot Enhanced Therapy for children with Autism Spectrum Disorders}, author = {Andreea Peca and Mark Coeckelbergh and Simut Vanderborght, Ramona and Cristina Pop and Sebastian Pintea and David, Daniel D and Bram Vanderborght}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/a-scale-measuring-ethical-acceptability-of-robot-enhanced-therapy-for-children-with-autism-spectrum-disorders(7bea3130-8aaf-4ecb-a4f7-5ba37e701390)/export.html}, issn = {1932-4529}, year = {2016}, date = {2016-01-01}, journal = {IEEE Technology And Society Magazine}, abstract = {A survey measuring the ethical acceptability of social robots in robot enhanced therapy (RET) for children with autism spectrum disorder (ASD) was developed. Three components of ethical acceptability were revealed: for use, for human-like interaction and for non-human appearance. The results indicate a high level of ethical acceptability on all the three factors. Men had a higher level of ethical acceptability of social robots on all the three factors, and younger respondents were more willing to accept the use of social robots in RET for ASD compared to older ones. Furthermore, respondents who had previous experience with robots had higher levels of ethical acceptability on all the three factors. Parents show a good acceptance level of using social robots in therapy of ASD, but a lower level compared with the other group. Education did not influence the level of ethical acceptability.}, keywords = {}, pubstate = {published}, tppubtype = {article} } A survey measuring the ethical acceptability of social robots in robot enhanced therapy (RET) for children with autism spectrum disorder (ASD) was developed. Three components of ethical acceptability were revealed: for use, for human-like interaction and for non-human appearance. The results indicate a high level of ethical acceptability on all the three factors. Men had a higher level of ethical acceptability of social robots on all the three factors, and younger respondents were more willing to accept the use of social robots in RET for ASD compared to older ones. Furthermore, respondents who had previous experience with robots had higher levels of ethical acceptability on all the three factors. Parents show a good acceptance level of using social robots in therapy of ASD, but a lower level compared with the other group. Education did not influence the level of ethical acceptability. |
de Perre, Greet Van; Beir, Albert De; Cao, Hoang-Long; Esteban, Pablo G'omez; Lefeber, Dirk; Vanderborght, Bram Reaching and pointing gestures calculated by a generic gesture system for social robots Journal Article In: Robotics and Autonomous Systems, 83 , pp. 32-43, 2016. @article{van2016reaching, title = {Reaching and pointing gestures calculated by a generic gesture system for social robots}, author = {Greet Van de Perre and Albert De Beir and Hoang-Long Cao and Pablo G'omez Esteban and Dirk Lefeber and Bram Vanderborght}, url = {http://www.dream2020.eu/wp-content/uploads/2016/08/Reaching-and-pointing-gestures_final.pdf}, doi = {doi:10.1016/j.robot.2016.06.006}, year = {2016}, date = {2016-01-01}, journal = {Robotics and Autonomous Systems}, volume = {83}, pages = {32-43}, publisher = {Elsevier}, abstract = {Since the implementation of gestures for a certain robot generally involves the use of specific information about its morphology, these gestures are not easily transferable to other robots. To cope with this problem, we proposed a generic method to generate gestures, constructed independently of any configuration and therefore useable for different robots. In this paper, we discuss the novel end-effector mode of the method, which can be used to calculate gestures whereby the position of the end-effector is important, for example for reaching for or pointing towards an object. The interesting and innovative feature of our method is its high degree of flexibility in both the possible configurations wherefore the method can be used, as in the gestures to be calculated. The method was validated on several configurations, including those of the robots ASIMO, NAO and Justin. In this paper, the working principles of the end-effector mode are discussed and a number of results are presented.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Since the implementation of gestures for a certain robot generally involves the use of specific information about its morphology, these gestures are not easily transferable to other robots. To cope with this problem, we proposed a generic method to generate gestures, constructed independently of any configuration and therefore useable for different robots. In this paper, we discuss the novel end-effector mode of the method, which can be used to calculate gestures whereby the position of the end-effector is important, for example for reaching for or pointing towards an object. The interesting and innovative feature of our method is its high degree of flexibility in both the possible configurations wherefore the method can be used, as in the gestures to be calculated. The method was validated on several configurations, including those of the robots ASIMO, NAO and Justin. In this paper, the working principles of the end-effector mode are discussed and a number of results are presented. |
Baxter, Paul; Kennedy, James; Ashurst, Emily; Belpaeme, Tony The Effect of Repeating Tasks on Performance Levels in Mediated Child-Robot Interactions. Inproceedings In: In Proceedings of the Robots 4 Learning Workshop, at RO-MAN 2016, 2016. @inproceedings{Baxter2016, title = {The Effect of Repeating Tasks on Performance Levels in Mediated Child-Robot Interactions.}, author = {Paul Baxter and James Kennedy and Emily Ashurst and Tony Belpaeme}, url = {http://www.dream2020.eu/baxter2016romanr4l/}, year = {2016}, date = {2016-01-01}, booktitle = {In Proceedings of the Robots 4 Learning Workshop, at RO-MAN 2016}, abstract = {That “practice makes perfect” is a powerful heuris- tic for improving performance through repetition. This is widely used in educational contexts, and as such it provides a poten- tially useful feature for application to child-robot educational interactions. While this effect may intuitively appear to be present, we here describe data to provide evidence in support of this supposition. Conducting a descriptive analysis of data from a wider study, we specifically examine the effect on child performance of repeating a previously performed collaborative task with a peer robot (i.e. not an expert agent), if initial performance is low. The results generally indicate a positive effect on performance through repetition, and a number of other correlation effects that highlight the role of individual differences. This outcome provides evidence for the variable utility of repetition between individuals, but also indicates that this is driven by the individual, which can nevertheless result in performance improvements even in the context of peer-peer interactions with relatively sparse feedback.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } That “practice makes perfect” is a powerful heuris- tic for improving performance through repetition. This is widely used in educational contexts, and as such it provides a poten- tially useful feature for application to child-robot educational interactions. While this effect may intuitively appear to be present, we here describe data to provide evidence in support of this supposition. Conducting a descriptive analysis of data from a wider study, we specifically examine the effect on child performance of repeating a previously performed collaborative task with a peer robot (i.e. not an expert agent), if initial performance is low. The results generally indicate a positive effect on performance through repetition, and a number of other correlation effects that highlight the role of individual differences. This outcome provides evidence for the variable utility of repetition between individuals, but also indicates that this is driven by the individual, which can nevertheless result in performance improvements even in the context of peer-peer interactions with relatively sparse feedback. |
Kennedy, James; Lemaignan, Severin; Belpaeme, Tony The Cautious Attitude of Teachers Towards Social Robots in Schools Inproceedings In: In Proceedings of the Robots 4 Learning Workshop, at RO-MAN 2016, 2016. @inproceedings{Kennedy2016, title = {The Cautious Attitude of Teachers Towards Social Robots in Schools}, author = {James Kennedy and Severin Lemaignan and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/09/Kennedy2016ROMANR4L.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {In Proceedings of the Robots 4 Learning Workshop, at RO-MAN 2016}, abstract = {Social robots are increasingly being applied in educa- tional environments such as schools. It is important to understand the views of the general public as social acceptance will likely play a role in the adoption of such technology. Other literature suggests that teacher attitudes are a strong predictor of technol- ogy use in classrooms, so willingness to engage with social robots will influence application in practice. In this paper we present the results of a rigorously-framed survey used to gather the views of both the general public and education professionals towards the use of robots in schools. Overall, we find that the attitude towards social robots in schools is cautious, but potentially accepting. We discuss the reported set of perceived obstacles for the broader adoption of robots in the classroom in this context. Interestingly, concerns about appropriate social skills for the robots dominate over practical and ethical concerns, suggesting that this should remain a focus for child-robot interaction research.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Social robots are increasingly being applied in educa- tional environments such as schools. It is important to understand the views of the general public as social acceptance will likely play a role in the adoption of such technology. Other literature suggests that teacher attitudes are a strong predictor of technol- ogy use in classrooms, so willingness to engage with social robots will influence application in practice. In this paper we present the results of a rigorously-framed survey used to gather the views of both the general public and education professionals towards the use of robots in schools. Overall, we find that the attitude towards social robots in schools is cautious, but potentially accepting. We discuss the reported set of perceived obstacles for the broader adoption of robots in the classroom in this context. Interestingly, concerns about appropriate social skills for the robots dominate over practical and ethical concerns, suggesting that this should remain a focus for child-robot interaction research. |
Lemaignan, Severin; Kennedy, James; Baxter, Paul; Belpaeme, Tony Towards "Machine-Learnable" Child-Robot Interactions: the PInSoRo Dataset Journal Article In: In Proceedings of the Long-Term Child-Robot Interaction Workshop, at RO-MAN 2016, 2016. @article{Lemaignan2016, title = {Towards "Machine-Learnable" Child-Robot Interactions: the PInSoRo Dataset}, author = {Severin Lemaignan and James Kennedy and Paul Baxter and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/09/Lemaignan2016ROMANWorkshop.pdf}, year = {2016}, date = {2016-01-01}, journal = {In Proceedings of the Long-Term Child-Robot Interaction Workshop, at RO-MAN 2016}, abstract = {Child-robot interactions are increasingly being explored in domains which require longer-term application, such as healthcare and education. In order for a robot to behave in an appropriate manner over longer timescales, its behaviours should be coterminous with that of the interacting children. Generating such sustained and engaging social behaviours is an on-going research challenge, and we argue here that the recent progress of deep machine learning opens new perspectives that the HRI community should embrace. As an initial step in that direction, we propose the creation of a large open dataset of child-robot social interactions. We detail our proposed methodology for data acquisition: children interact with a robot puppeted by an expert adult during a range of playful face-to- face social tasks. By doing so, we seek to capture a rich set of human-like behaviours occurring in natural social interactions, that are explicitly mapped to the robot’s embodiment and affordances.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Child-robot interactions are increasingly being explored in domains which require longer-term application, such as healthcare and education. In order for a robot to behave in an appropriate manner over longer timescales, its behaviours should be coterminous with that of the interacting children. Generating such sustained and engaging social behaviours is an on-going research challenge, and we argue here that the recent progress of deep machine learning opens new perspectives that the HRI community should embrace. As an initial step in that direction, we propose the creation of a large open dataset of child-robot social interactions. We detail our proposed methodology for data acquisition: children interact with a robot puppeted by an expert adult during a range of playful face-to- face social tasks. By doing so, we seek to capture a rich set of human-like behaviours occurring in natural social interactions, that are explicitly mapped to the robot’s embodiment and affordances. |
Stahl, Bernd Carsten; Coeckelbergh, Mark Ethics of Healthcare Robotics: Towards Responsible Research and Innovation Journal Article In: Robotics and Autonomous Systems, 86 , pp. 152 - 161, 2016, ISSN: 0921-8890. @article{Stahl2016152, title = {Ethics of Healthcare Robotics: Towards Responsible Research and Innovation}, author = {Bernd Carsten Stahl and Mark Coeckelbergh}, url = {http://www.dream2020.eu/wp-content/uploads/2016/11/1-s2.0-S0921889016305292-main.pdf}, doi = {http://dx.doi.org/10.1016/j.robot.2016.08.018}, issn = {0921-8890}, year = {2016}, date = {2016-01-01}, journal = {Robotics and Autonomous Systems}, volume = {86}, pages = {152 - 161}, abstract = {Abstract How can we best identify, understand, and deal with ethical and societal issues raised by healthcare robotics? This paper argues that next to ethical analysis, classic technology assessment, and philosophical speculation we need forms of reflection, dialogue, and experiment that come, quite literally, much closer to innovation practices and contexts of use. The authors discuss a number of ways how to achieve that. Informed by their experience with “embedded” ethics in technical projects and with various tools and methods of responsible research and innovation, the paper identifies “internal” and “external” forms of dialogical research and innovation, reflections on the possibilities and limitations of these forms of ethical–technological innovation, and explores a number of ways how they can be supported by policy at national and supranational level.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Abstract How can we best identify, understand, and deal with ethical and societal issues raised by healthcare robotics? This paper argues that next to ethical analysis, classic technology assessment, and philosophical speculation we need forms of reflection, dialogue, and experiment that come, quite literally, much closer to innovation practices and contexts of use. The authors discuss a number of ways how to achieve that. Informed by their experience with “embedded” ethics in technical projects and with various tools and methods of responsible research and innovation, the paper identifies “internal” and “external” forms of dialogical research and innovation, reflections on the possibilities and limitations of these forms of ethical–technological innovation, and explores a number of ways how they can be supported by policy at national and supranational level. |
Senft, Emmanuel; Lemaignan, Severin; Baxter, Paul E; Belpaeme, Tony SPARC: an efficient way to combine reinforcement learning and supervised autonomy. Inproceedings In: FILM Workshop at NIPS'16, 2016. @inproceedings{senft2016sparc, title = {SPARC: an efficient way to combine reinforcement learning and supervised autonomy.}, author = { Emmanuel Senft and Severin Lemaignan and Paul E Baxter and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2017/03/2016-ES-FILM.pdf}, year = {2016}, date = {2016-01-01}, booktitle = {FILM Workshop at NIPS'16}, abstract = {Shortcomings of reinforcement learning for robot control include the sparsity of the environmental reward function, the high number of trials required before reaching an efficient action policy and the reliance on exploration to gather information about the environment, potentially resulting in undesired actions. These limits can be overcome by adding a human in the loop to provide additional information during the learning phase. In this paper, we propose a novel way to combine human inputs and reinforcement by following the Supervised Progressively Autonomous Robot Competencies (SPARC) approach. We compare this method to the principles of Interactive Reinforcement Learning as proposed by Thomaz and Breazeal. Results from a study involving 40 participants show that using SPARC increases the performance of the learning, reduces the time and number of inputs required for teaching and faces fewer errors during the learning process. These results support the use of SPARC as an efficient method to teach a robot to interact with humans.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Shortcomings of reinforcement learning for robot control include the sparsity of the environmental reward function, the high number of trials required before reaching an efficient action policy and the reliance on exploration to gather information about the environment, potentially resulting in undesired actions. These limits can be overcome by adding a human in the loop to provide additional information during the learning phase. In this paper, we propose a novel way to combine human inputs and reinforcement by following the Supervised Progressively Autonomous Robot Competencies (SPARC) approach. We compare this method to the principles of Interactive Reinforcement Learning as proposed by Thomaz and Breazeal. Results from a study involving 40 participants show that using SPARC increases the performance of the learning, reduces the time and number of inputs required for teaching and faces fewer errors during the learning process. These results support the use of SPARC as an efficient method to teach a robot to interact with humans. |
Beckerle, Philipp; Beir, Albert De; ü, Tim Sch; Caspar, Emilie A Human body schema exploration: Analyzing design requirements of Robotic Hand and Leg Illusions Inproceedings In: Robot and Human Interactive Communication (RO-MAN), 2016 25th IEEE International Symposium on, pp. 763–768, IEEE 2016. @inproceedings{beckerle2016human, title = {Human body schema exploration: Analyzing design requirements of Robotic Hand and Leg Illusions}, author = { Philipp Beckerle and Albert De Beir and Tim Sch{ü}rmann and Emilie A Caspar}, url = {http://ieeexplore.ieee.org/abstract/document/7745205/}, year = {2016}, date = {2016-01-01}, booktitle = {Robot and Human Interactive Communication (RO-MAN), 2016 25th IEEE International Symposium on}, pages = {763--768}, organization = {IEEE}, abstract = {Understanding the integration of user-proximal robots in the body schema of their human users has a distinct potential to improve human-robot interaction. Robotic devices can help to investigate the psychological fundamentals of body schema integration. While the Rubber Hand Illusion experiment indicates how artifacts can be perceived as a part of the own body, it relies on a passive limb that does not perform motions during the examinations. Novel setups aim at Robotic Hand/Leg Illusions induced by robotic devices which imitate human motions. Although such devices distinctly extend experimental possibilities, their design is rather proprietary and unstructured up to now. This paper analyzes the requirements of robotic hand and leg illusion setups based on systematic discussion of a multidisciplinary team of researchers from engineering and psychology. In a comparative study, requirements are collected and structured, their similarities and differences are determined, and the most important ones are extracted yielding design implications. The requirements with the highest priority are setup characteristics that concern the occurrence and quality of the illusion, i.e., hiding the real limb, anatomical plausibility, visual appearance, temporal delay, and software-controlled experimental conditions. Based on the results, the design of future robotic devices for the exploration of human body schema integration might be guided and supported.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Understanding the integration of user-proximal robots in the body schema of their human users has a distinct potential to improve human-robot interaction. Robotic devices can help to investigate the psychological fundamentals of body schema integration. While the Rubber Hand Illusion experiment indicates how artifacts can be perceived as a part of the own body, it relies on a passive limb that does not perform motions during the examinations. Novel setups aim at Robotic Hand/Leg Illusions induced by robotic devices which imitate human motions. Although such devices distinctly extend experimental possibilities, their design is rather proprietary and unstructured up to now. This paper analyzes the requirements of robotic hand and leg illusion setups based on systematic discussion of a multidisciplinary team of researchers from engineering and psychology. In a comparative study, requirements are collected and structured, their similarities and differences are determined, and the most important ones are extracted yielding design implications. The requirements with the highest priority are setup characteristics that concern the occurrence and quality of the illusion, i.e., hiding the real limb, anatomical plausibility, visual appearance, temporal delay, and software-controlled experimental conditions. Based on the results, the design of future robotic devices for the exploration of human body schema integration might be guided and supported. |
Beir, Albert De; Cao, Hoang-Long; 'o, Pablo G; de Perre, Greet Van; Lefeber, Dirk; Vanderborght, Bram Enhancing emotional facial expressiveness on NAO Journal Article In: International Journal of Social Robotics, 8 (4), pp. 513–521, 2016. @article{de2016enhancing, title = {Enhancing emotional facial expressiveness on NAO}, author = { Albert De Beir and Hoang-Long Cao and Pablo G{'o}mez Esteban and Greet Van de Perre and Dirk Lefeber and Bram Vanderborght}, url = {https://link.springer.com/article/10.1007/s12369-016-0363-x}, year = {2016}, date = {2016-01-01}, journal = {International Journal of Social Robotics}, volume = {8}, number = {4}, pages = {513--521}, publisher = {Springer Netherlands}, abstract = {Robots can express emotions for better human robot interaction. In this field, NAO robot is a platform widely used. This robot mainly expresses emotions by gestures and colored LED eyes, but, due to its white flat and inanimate face, the robot cannot express facial expressions. This work proposes a pluggable eyebrows device enabling NAO to express anger or sadness while performing other tasks. This device is plug-and-play and directly controllable by NAO’s development environments. Additionally, we developed a platform independent mapping of eyebrows angles with emotions. We first conducted an experiment that qualitatively attests the interest of this device. Three following experiments were conducted to: (1) confirm the relation between eyebrows angle and expressed emotion; (2) evaluate different shapes in order to select the most appropriate one; (3) prove that NAO is able to use the eyebrows to express emotions while performing non emotional tasks.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Robots can express emotions for better human robot interaction. In this field, NAO robot is a platform widely used. This robot mainly expresses emotions by gestures and colored LED eyes, but, due to its white flat and inanimate face, the robot cannot express facial expressions. This work proposes a pluggable eyebrows device enabling NAO to express anger or sadness while performing other tasks. This device is plug-and-play and directly controllable by NAO’s development environments. Additionally, we developed a platform independent mapping of eyebrows angles with emotions. We first conducted an experiment that qualitatively attests the interest of this device. Three following experiments were conducted to: (1) confirm the relation between eyebrows angle and expressed emotion; (2) evaluate different shapes in order to select the most appropriate one; (3) prove that NAO is able to use the eyebrows to express emotions while performing non emotional tasks. |
Vernon, David; Thill, Serge; Ziemke, Tom; Hemeren, Paul; Billing, Erik An Architecture-oriented Approach to System Integration in Collaborative Robotics Research Projects — An Experience Report Journal Article In: Journal of Software Engineering for Robotics, 6 (1), pp. 15 - 32, 2015, ISSN: 2035-3928. @article{DVernon2015, title = {An Architecture-oriented Approach to System Integration in Collaborative Robotics Research Projects — An Experience Report}, author = {David Vernon and Serge Thill and Tom Ziemke and Paul Hemeren and Erik Billing}, url = {http://www.dream2020.eu/wp-content/uploads/2016/02/15_Vernon_et_al_JOSER.pdf}, issn = {2035-3928}, year = {2015}, date = {2015-12-06}, journal = {Journal of Software Engineering for Robotics}, volume = {6}, number = {1}, pages = {15 - 32}, abstract = {Effective system integration requires strict adherence to strong software engineering standards, a practice not much favoured in many collaborative research projects. We argue that component-based software engineering (CBSE) provides a way to overcome this problem because it provides flexibility for developers while requiring the adoption of only a modest number of software engineering practices. This focus on integration complements software re-use, the more usual motivation for adopting CBSE. We illustrate our argument by showing how a large-scale system architecture for an application in the domain of robot-enhanced therapy for children with autism spectrum disorder (ASD) has been implemented. We highlight the manner in which the integration process is facilitated by the architecture implementation of a set of placeholder components that comprise stubs for all functional primitives, as well as the complete implementation of all inter-component communications. We focus on the component-port-connector meta-model and show that the YARP robot platform is a well-matched middleware framework for the implementation of this model. To facilitate the validation of port-connector communication, we configure the initial placeholder implementation of the system architecture as a discrete event simulation and control the invocation of each component’s stub primitives probabilistically. This allows the system integrator to adjust the rate of inter-component communication while respecting its asynchronous and concurrent character. Also, individual ports and connectors can be periodically selected as the simulator cycles through each primitive in each sub-system component. This ability to control the rate of connector communication considerably eases the task of validating component-port-connector behaviour in a large system. Ultimately, over and above its well-accepted benefits for software re-use in robotics, CBSE strikes a good balance between software engineering best practice and the socio-technical problem of managing effective integration in collaborative robotics research projects.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Effective system integration requires strict adherence to strong software engineering standards, a practice not much favoured in many collaborative research projects. We argue that component-based software engineering (CBSE) provides a way to overcome this problem because it provides flexibility for developers while requiring the adoption of only a modest number of software engineering practices. This focus on integration complements software re-use, the more usual motivation for adopting CBSE. We illustrate our argument by showing how a large-scale system architecture for an application in the domain of robot-enhanced therapy for children with autism spectrum disorder (ASD) has been implemented. We highlight the manner in which the integration process is facilitated by the architecture implementation of a set of placeholder components that comprise stubs for all functional primitives, as well as the complete implementation of all inter-component communications. We focus on the component-port-connector meta-model and show that the YARP robot platform is a well-matched middleware framework for the implementation of this model. To facilitate the validation of port-connector communication, we configure the initial placeholder implementation of the system architecture as a discrete event simulation and control the invocation of each component’s stub primitives probabilistically. This allows the system integrator to adjust the rate of inter-component communication while respecting its asynchronous and concurrent character. Also, individual ports and connectors can be periodically selected as the simulator cycles through each primitive in each sub-system component. This ability to control the rate of connector communication considerably eases the task of validating component-port-connector behaviour in a large system. Ultimately, over and above its well-accepted benefits for software re-use in robotics, CBSE strikes a good balance between software engineering best practice and the socio-technical problem of managing effective integration in collaborative robotics research projects. |
Cai, Haibin; Zhou, Xiaolong; Yu, Hui; Liu, Honghai Gaze Estimation Driven Solution for Interacting Children with ASD Inproceedings In: 26th 2015 International Symposium on Micro-NanoMechatronics and Human Science, Nagoya, Japan, 2015. @inproceedings{Micro-NanoMechatronicsandHumanScience, title = {Gaze Estimation Driven Solution for Interacting Children with ASD}, author = {Haibin Cai and Xiaolong Zhou and Hui Yu and Honghai Liu}, year = {2015}, date = {2015-11-23}, booktitle = {26th 2015 International Symposium on Micro-NanoMechatronics and Human Science}, address = {Nagoya, Japan}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Hettiarachchi, N; Ju, Z; Liu, H A New Wearable Ultrasound Muscle Activity Sensing System for Dexterous Prosthetic Control Inproceedings In: 2015 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1415-1420, 2015. @inproceedings{Hettiarachchi2015, title = {A New Wearable Ultrasound Muscle Activity Sensing System for Dexterous Prosthetic Control}, author = {N Hettiarachchi and Z Ju and H Liu}, url = {https://ieeexplore.ieee.org/document/7379383}, doi = {10.1109/SMC.2015.251}, year = {2015}, date = {2015-10-10}, booktitle = {2015 IEEE International Conference on Systems, Man, and Cybernetics}, pages = {1415-1420}, abstract = {In this paper we introduce a novel Wearable Ultrasound Radial Muscle Activity Detection System (WURMADS) and a demonstration of its ability to recognize forearm muscle activities of amputee subjects to control a dexterous prosthetic hand. The system consists of control electronics to capture and record the ultrasound echo signals in real-time and two wearable bands embedded with eight ultrasound transducers. Based on the principles of Sonomyography, we recognized the intended isotonic finger gestures (trans-radial amputees) by analysing the echo patterns captured by eight single element transducers. Conventional non-invasive my electric detection strategy, Surface Electro Myography (sEMG), cannot reliably identify the deeper muscle activations in the forearm due to crosstalk and signal attenuation. It also suffers from signal degradation due to muscle fatigue and non-linearity. For this study we custom designed 1D single element waterproof transducers to be assembled in a radial structure. A real-time capturing electronic system was also designed, which is portable and wearable. The system operates at 5MHz, the frequency that demonstrated optimum performance for forearm applications. Ten trans-radial amputee subjects were employed in this study to capture data by executing five supervised gestures. The results obtained from offline analysis show that good gesture recognition rates can be achieved, while maintaining a decent correlation coefficient margin to distinguish gestures.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In this paper we introduce a novel Wearable Ultrasound Radial Muscle Activity Detection System (WURMADS) and a demonstration of its ability to recognize forearm muscle activities of amputee subjects to control a dexterous prosthetic hand. The system consists of control electronics to capture and record the ultrasound echo signals in real-time and two wearable bands embedded with eight ultrasound transducers. Based on the principles of Sonomyography, we recognized the intended isotonic finger gestures (trans-radial amputees) by analysing the echo patterns captured by eight single element transducers. Conventional non-invasive my electric detection strategy, Surface Electro Myography (sEMG), cannot reliably identify the deeper muscle activations in the forearm due to crosstalk and signal attenuation. It also suffers from signal degradation due to muscle fatigue and non-linearity. For this study we custom designed 1D single element waterproof transducers to be assembled in a radial structure. A real-time capturing electronic system was also designed, which is portable and wearable. The system operates at 5MHz, the frequency that demonstrated optimum performance for forearm applications. Ten trans-radial amputee subjects were employed in this study to capture data by executing five supervised gestures. The results obtained from offline analysis show that good gesture recognition rates can be achieved, while maintaining a decent correlation coefficient margin to distinguish gestures. |
Zhang, Shu; Yu, Hui; Dong, Junyu; Wang, Ting; Liu, Honghai Automatic Reconstruction of Dense 3D Face Point Cloud with A Single Depth Image Inproceedings In: Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China , 2015. @inproceedings{Systems,Man,andCybernetics, title = {Automatic Reconstruction of Dense 3D Face Point Cloud with A Single Depth Image}, author = {Shu Zhang and Hui Yu and Junyu Dong and Ting Wang and Honghai Liu}, url = {http://eprints.port.ac.uk/18857/}, year = {2015}, date = {2015-10-09}, booktitle = {Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics}, address = {Hong Kong, China }, abstract = {Human face analysis is the basis for many other computer vision tasks, such as camera surveillance, entrance authorization and age estimation. With 3D face models, the vision task based on facial analysis can usually achieve a higher accuracy than the 2D cases since it provides more information with the additional dimension. However, most existing 3D face reconstruction methods suffer from complicated processing and high computation. This paper presents a novel method that simplifies the 3D face reconstruction process with only one shot of Kinect data. The output of the system is a high density of 3D face point cloud with smoother surface. This provides rich details of the human face for other computer vision tasks. Experiments with real world data show promising results using the proposed method}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Human face analysis is the basis for many other computer vision tasks, such as camera surveillance, entrance authorization and age estimation. With 3D face models, the vision task based on facial analysis can usually achieve a higher accuracy than the 2D cases since it provides more information with the additional dimension. However, most existing 3D face reconstruction methods suffer from complicated processing and high computation. This paper presents a novel method that simplifies the 3D face reconstruction process with only one shot of Kinect data. The output of the system is a high density of 3D face point cloud with smoother surface. This provides rich details of the human face for other computer vision tasks. Experiments with real world data show promising results using the proposed method |
Cao, Hoang-Long; Gomez Esteban, Pablo ; De Beir, Albert ; Simut Vanderborght, Ramona ; Van de Perre, Greet ; Lefeber, Dirk; Vanderborght, Bram Toward a platform-independent social behavior architecture for multiple therapeutic scenarios Inproceedings In: Proceedings of the 1st International Conference on Social Robots in Therapy and Education, 2015. @inproceedings{cao2015multiple, title = {Toward a platform-independent social behavior architecture for multiple therapeutic scenarios}, author = {Hoang-Long Cao and Pablo { Gomez Esteban } and Albert { De Beir } and Ramona { Simut Vanderborght } and Greet { Van de Perre } and Dirk Lefeber and Bram Vanderborght}, url = {http://newfriends2015.org/Proceedings/Contents/Papers/New_Friends_2015_submission_16.pdf}, year = {2015}, date = {2015-10-01}, booktitle = {Proceedings of the 1st International Conference on Social Robots in Therapy and Education}, abstract = {Researchers in cognitive robotics have developed different architectures to help social robots make decisions and express emotions autonomously in order to replace the pre-programmed and remote-controlled techniques. However, most of the current developments are ad-hoc solutions with no possibility to be utilized in multiple therapeutic situations. In this paper, we propose the design of a social behavior architecture, which aims at helping the social robot to sustain the user’s engagement and motivation, and to achieve the goal of interaction in different scenarios. Representations of behaviors are kept at abstract level and mapped with the robot’s morphology afterward. This approach ensures the architecture to be applicable to a wide range of social robots.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Researchers in cognitive robotics have developed different architectures to help social robots make decisions and express emotions autonomously in order to replace the pre-programmed and remote-controlled techniques. However, most of the current developments are ad-hoc solutions with no possibility to be utilized in multiple therapeutic situations. In this paper, we propose the design of a social behavior architecture, which aims at helping the social robot to sustain the user’s engagement and motivation, and to achieve the goal of interaction in different scenarios. Representations of behaviors are kept at abstract level and mapped with the robot’s morphology afterward. This approach ensures the architecture to be applicable to a wide range of social robots. |
De Beir, Albert ; Cao, Hoang-Long; Gomez Esteban, Pablo ; Van de Perre, Greet ; Vanderborght, Bram Enhancing Nao expression of emotions using pluggable eyebrows Inproceedings In: Proceedings of the 1st International Conference on Social Robots in Therapy and Education, Windesheim Flevoland, 2015. @inproceedings{debeir2015eyebrows, title = {Enhancing Nao expression of emotions using pluggable eyebrows}, author = {Albert { De Beir } and Hoang-Long Cao and Pablo { Gomez Esteban } and Greet { Van de Perre } and Bram Vanderborght}, url = {http://newfriends2015.org/Proceedings/Contents/Papers/New_Friends_2015_submission_10.pdf}, year = {2015}, date = {2015-10-01}, booktitle = {Proceedings of the 1st International Conference on Social Robots in Therapy and Education}, publisher = {Windesheim Flevoland}, abstract = {Robots can express emotions for better Human Robot Interaction (HRI). In this field of HRI, the Nao robot is a platform widely used. This robot mainly expresses emotions by gestures and colored led eyes, but due to its white flat and inanimate face it cannot express facial expressions. In this work we propose a pluggable eyebrows device with two separated degrees of freedom. A short survey is conducted to qualitatively evaluate the usefulness of this device.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Robots can express emotions for better Human Robot Interaction (HRI). In this field of HRI, the Nao robot is a platform widely used. This robot mainly expresses emotions by gestures and colored led eyes, but due to its white flat and inanimate face it cannot express facial expressions. In this work we propose a pluggable eyebrows device with two separated degrees of freedom. A short survey is conducted to qualitatively evaluate the usefulness of this device. |
Tilmans, Roger; Gomez Esteban, Pablo ; Cao, Hoang-Long; Vanderborght, Bram Social and autonomous confabulation architecture for autism therapy Inproceedings In: Proceedings of the 1st International Conference on Social Robots in Therapy and Education, 2015. @inproceedings{tilmans2015confabulation, title = {Social and autonomous confabulation architecture for autism therapy}, author = {Roger Tilmans and Pablo { Gomez Esteban } and Hoang-Long Cao and Bram Vanderborght}, url = {http://newfriends2015.org/Proceedings/Contents/Papers/New_Friends_2015_submission_14.pdf}, year = {2015}, date = {2015-10-01}, booktitle = {Proceedings of the 1st International Conference on Social Robots in Therapy and Education}, abstract = {This paper presents the Social and Autonomous Con- fabulation Architecture: a new cognitive architecture developed for social robots that have to operate under supervised autonomy. With this new cognitive architecture, based on the confabulation theory, robots are able to choose autonomously their behaviors, to have emotions and to have learning abilities. At the end, the architecture is tested with the robot Nao playing different interactive games with a kid.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } This paper presents the Social and Autonomous Con- fabulation Architecture: a new cognitive architecture developed for social robots that have to operate under supervised autonomy. With this new cognitive architecture, based on the confabulation theory, robots are able to choose autonomously their behaviors, to have emotions and to have learning abilities. At the end, the architecture is tested with the robot Nao playing different interactive games with a kid. |
Ju, Zhaojie; Ji, Xiaofei; Li, Jing; Liu, Honghai An Integrative Framework of Human Hand Gesture Segmentation for Human–Robot Interaction Journal Article In: Systems Journal, IEEE, PP (99), pp. 1-11, 2015, ISSN: 1932-8184. @article{IEEESystemsJournal, title = { An Integrative Framework of Human Hand Gesture Segmentation for Human–Robot Interaction}, author = {Zhaojie Ju and Xiaofei Ji and Jing Li and Honghai Liu }, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7244188}, doi = {10.1109/JSYST.2015.2468231}, issn = {1932-8184}, year = {2015}, date = {2015-09-01}, journal = {Systems Journal, IEEE}, volume = {PP}, number = {99}, pages = {1-11}, abstract = {This paper proposes a novel framework to segment hand gestures in RGB-depth (RGB-D) images captured by Kinect using humanlike approaches for human–robot interaction. The goal is to reduce the error of Kinect sensing and, consequently, to improve the precision of hand gesture segmentation for robot NAO. The proposed framework consists of two main novel approaches. First, the depth map and RGB image are aligned by using the genetic algorithm to estimate key points, and the alignment is robust to uncertainties of the extracted point numbers. Then, a novel approach is proposed to refine the edge of the tracked hand gestures in RGB images by applying a modified expectation–maximization (EM) algorithm based on Bayesian networks. The experimental results demonstrate that the proposed alignment method is capable of precisely matching the depth maps with RGB images, and the EM algorithm further effectively adjusts the RGB edges of the segmented hand gestures. The proposed framework has been integrated and validated in a system of human–robot interaction to improve NAO robot's performance of understanding and interpretation. }, keywords = {}, pubstate = {published}, tppubtype = {article} } This paper proposes a novel framework to segment hand gestures in RGB-depth (RGB-D) images captured by Kinect using humanlike approaches for human–robot interaction. The goal is to reduce the error of Kinect sensing and, consequently, to improve the precision of hand gesture segmentation for robot NAO. The proposed framework consists of two main novel approaches. First, the depth map and RGB image are aligned by using the genetic algorithm to estimate key points, and the alignment is robust to uncertainties of the extracted point numbers. Then, a novel approach is proposed to refine the edge of the tracked hand gestures in RGB images by applying a modified expectation–maximization (EM) algorithm based on Bayesian networks. The experimental results demonstrate that the proposed alignment method is capable of precisely matching the depth maps with RGB images, and the EM algorithm further effectively adjusts the RGB edges of the segmented hand gestures. The proposed framework has been integrated and validated in a system of human–robot interaction to improve NAO robot's performance of understanding and interpretation. |
Drejing, Karl; Thill, Serge; Hemeren, Paul Engagement: A traceable motivational concept in human-robot interaction Inproceedings In: Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on, pp. 956-961, 2015. @inproceedings{7344690, title = {Engagement: A traceable motivational concept in human-robot interaction}, author = {Karl Drejing and Serge Thill and Paul Hemeren}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7344690}, doi = {10.1109/ACII.2015.7344690}, year = {2015}, date = {2015-09-01}, booktitle = {Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on}, pages = {956-961}, abstract = {Engagement is essential to meaningful social interaction between humans. Understanding the mechanisms by which we detect engagement of other humans can help us understand how we can build robots that interact socially with humans. However, there is currently a lack of measurable engagement constructs on which to build an artificial system that can reliably support social interaction between humans and robots. This paper proposes a definition, based on motivation theories, and outlines a framework to explore the idea that engagement can be seen as specific behaviors and their attached magnitude or intensity. This is done by the use of data from multiple sources such as observer ratings, kinematic data, audio and outcomes of interactions. We use the domain of human-robot interaction in order to illustrate the application of this approach. The framework further suggests a method to gather and aggregate this data. If certain behaviors and their attached intensities co-occur with various levels of judged engagement, then engagement could be assessed by this framework consequently making it accessible to a robotic platform. This framework could improve the social capabilities of interactive agents by adding the ability to notice when and why an agent becomes disengaged, thereby providing the interactive agent with an ability to reengage him or her. We illustrate and propose validation of our framework with an example from robot-assisted therapy for children with autism spectrum disorder. The framework also represents a general approach that can be applied to other social interactive settings between humans and robots, such as interactions with elderly people.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Engagement is essential to meaningful social interaction between humans. Understanding the mechanisms by which we detect engagement of other humans can help us understand how we can build robots that interact socially with humans. However, there is currently a lack of measurable engagement constructs on which to build an artificial system that can reliably support social interaction between humans and robots. This paper proposes a definition, based on motivation theories, and outlines a framework to explore the idea that engagement can be seen as specific behaviors and their attached magnitude or intensity. This is done by the use of data from multiple sources such as observer ratings, kinematic data, audio and outcomes of interactions. We use the domain of human-robot interaction in order to illustrate the application of this approach. The framework further suggests a method to gather and aggregate this data. If certain behaviors and their attached intensities co-occur with various levels of judged engagement, then engagement could be assessed by this framework consequently making it accessible to a robotic platform. This framework could improve the social capabilities of interactive agents by adding the ability to notice when and why an agent becomes disengaged, thereby providing the interactive agent with an ability to reengage him or her. We illustrate and propose validation of our framework with an example from robot-assisted therapy for children with autism spectrum disorder. The framework also represents a general approach that can be applied to other social interactive settings between humans and robots, such as interactions with elderly people. |
Gao, Dongxu; Ju, Zhaojie; Cao, Jiangtao; Liu, Honghai Real time Object Tracking Via a Mixture Model Inproceedings In: 24th IEEE International Symposium on Robot and Human Interactive Communication, pp. 112-116, IEEE, Kobe, Japan, 2015. @inproceedings{RobotandHumanInteractiveCommunication, title = { Real time Object Tracking Via a Mixture Model}, author = {Dongxu Gao and Zhaojie Ju and Jiangtao Cao and Honghai Liu}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7333701}, doi = {10.1109/ROMAN.2015.7333701}, year = {2015}, date = {2015-08-31}, booktitle = {24th IEEE International Symposium on Robot and Human Interactive Communication}, pages = {112-116}, publisher = {IEEE}, address = {Kobe, Japan}, abstract = {Object tracking has been applied in many fields such as intelligent surveillance and computer vision. Although much progress has been made, there are still many puzzles which pose a huge challenge to object tracking. Currently, the problems are mainly caused by appearance model as well as real-time performance. A novel method was been proposed in this paper to handle both of these problems. Locally dense contexts feature and image information (i.e. the relationship between the object and its surrounding regions) are combined in a Bayes framework. Then the tracking problem can be seen as a prediction question which need to compute the posterior probability. Both scale variations and temple updating are considered in the proposed algorithm to assure the effectiveness. To make the algorithm runs in a real time system, a Fourier Transform (FT) is used when solving the Bayes equation. Therefore, the MMOT (Mixture model for object tracking) runs in real-time and performs better than state-of-the-art algorithms on some challenging image sequences in terms of accuracy, quickness and robustness.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Object tracking has been applied in many fields such as intelligent surveillance and computer vision. Although much progress has been made, there are still many puzzles which pose a huge challenge to object tracking. Currently, the problems are mainly caused by appearance model as well as real-time performance. A novel method was been proposed in this paper to handle both of these problems. Locally dense contexts feature and image information (i.e. the relationship between the object and its surrounding regions) are combined in a Bayes framework. Then the tracking problem can be seen as a prediction question which need to compute the posterior probability. Both scale variations and temple updating are considered in the proposed algorithm to assure the effectiveness. To make the algorithm runs in a real time system, a Fourier Transform (FT) is used when solving the Bayes equation. Therefore, the MMOT (Mixture model for object tracking) runs in real-time and performs better than state-of-the-art algorithms on some challenging image sequences in terms of accuracy, quickness and robustness. |
Vernon, David; Thill, Serge; Ziemke, Tom The Role of Intention in Cognitive Robotics Book Chapter In: Esposito, Anna; Jain, Lakhmi C (Ed.): Socially Believable Behaving Systems - The Quest for Equipping Machines with Human-Level Automaton Intelligence, Springer, 2015. @inbook{Vernon2015, title = {The Role of Intention in Cognitive Robotics}, author = {David Vernon and Serge Thill and Tom Ziemke }, editor = {Anna Esposito and Lakhmi C. Jain}, url = {http://dream2020.eu/wp-content/uploads/2015/08/15_Vernon_Thill_Ziemke_SBBS1.pdf}, year = {2015}, date = {2015-08-29}, booktitle = {Socially Believable Behaving Systems - The Quest for Equipping Machines with Human-Level Automaton Intelligence}, issuetitle = {Intelligent Systems Reference Library}, publisher = {Springer}, abstract = {We argue that the development of robots that can interact effectively with people requires a special focus on building systems that can perceive and compre- hend intentions in other agents. Such a capability is a prerequisite for all pro-social behaviour and underpins in particular the ability to engage in instrumental help- ing and mutual collaboration. We explore the prospective and intentional nature of action, highlighting the importance of joint action, shared goals, shared intentions, and joint attention in facilitating social interaction between two or more cognitive agents. We discuss the link between reading intentions and theory of mind, noting the role played by internal simulation, especially when inferring higher-level action- focussed intentions. Finally, we highlight that pro-social behaviour in humans is the result of a developmental process and we note the implications of this for the chal- lenge of creating cognitive robots that can read intentions.}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } We argue that the development of robots that can interact effectively with people requires a special focus on building systems that can perceive and compre- hend intentions in other agents. Such a capability is a prerequisite for all pro-social behaviour and underpins in particular the ability to engage in instrumental help- ing and mutual collaboration. We explore the prospective and intentional nature of action, highlighting the importance of joint action, shared goals, shared intentions, and joint attention in facilitating social interaction between two or more cognitive agents. We discuss the link between reading intentions and theory of mind, noting the role played by internal simulation, especially when inferring higher-level action- focussed intentions. Finally, we highlight that pro-social behaviour in humans is the result of a developmental process and we note the implications of this for the chal- lenge of creating cognitive robots that can read intentions. |
Thill, Serge; Vernon, David How to design emergent models of cognition for application-driven artificial agents Conference Proc. Neural Computation Psychology Workshop (in press), 2015. @conference{Thill2015, title = {How to design emergent models of cognition for application-driven artificial agents}, author = {Serge Thill and David Vernon}, url = {http://dream2020.eu/wp-content/uploads/2015/08/15_Thill_Vernon_NCPW.pdf}, year = {2015}, date = {2015-08-29}, booktitle = {Proc. Neural Computation Psychology Workshop (in press)}, abstract = {Emergent models of cognition are attractive for artificial cognitive agents be- cause they overcome the brittleness of systems that are fully specified in ax- iomatic terms at design time, increasing, for example, the ability to deal with uncertainty and unforeseen events. When the agent is created to fulfil specific requirements defined by a given application, there is an apparent conflict be- tween the emergent (i.e. self-defining) nature of the agent’s behaviour and the pre-specified (i.e. axiomatically-defined) nature of the requirements. Here, we develop a framework for the design of emergent models of cog- nition whose behaviour can be shaped to fulfil application requirements while retaining the desired characteristics of emergence. We achieve this by viewing the artificial agent as forming an eco-system with the environment in which it is deployed. Consequently, the objective function that determines the agent’s be- haviour is cast in terms that factor in interaction with the environment (while not being controlled by it) and therefore implicitly includes the application requirements. This framework is particularly relevant to application driven research where artificial agents are designed to interact with humans in a certain man- ner. We illustrate this with the example of robot-enhanced therapy for children with autism spectrum disorder.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Emergent models of cognition are attractive for artificial cognitive agents be- cause they overcome the brittleness of systems that are fully specified in ax- iomatic terms at design time, increasing, for example, the ability to deal with uncertainty and unforeseen events. When the agent is created to fulfil specific requirements defined by a given application, there is an apparent conflict be- tween the emergent (i.e. self-defining) nature of the agent’s behaviour and the pre-specified (i.e. axiomatically-defined) nature of the requirements. Here, we develop a framework for the design of emergent models of cog- nition whose behaviour can be shaped to fulfil application requirements while retaining the desired characteristics of emergence. We achieve this by viewing the artificial agent as forming an eco-system with the environment in which it is deployed. Consequently, the objective function that determines the agent’s be- haviour is cast in terms that factor in interaction with the environment (while not being controlled by it) and therefore implicitly includes the application requirements. This framework is particularly relevant to application driven research where artificial agents are designed to interact with humans in a certain man- ner. We illustrate this with the example of robot-enhanced therapy for children with autism spectrum disorder. |
Cao, Hoang-Long; Pop, Cristina A; Simut Vanderborght, Ramona ; Furnemont, Raphael; De Beir, Albert ; Van De Perre, Greet ; Gomez Esteban, Pablo ; Lefeber, Dirk; Vanderborght, Bram Probolino: A Portable low-cost social device for home-based autism therapy Inproceedings In: Tapus, Adriana; André, Elisabeth; Martin, Jean-Claude; Ferland, François; Ammi, Mehdi (Ed.): Proceedings of the 7th International Conference on Social Robotics, pp. 93–102, Springer, 2015, ISBN: 978-3-319-25553-8. @inproceedings{cao2015probolino, title = {Probolino: A Portable low-cost social device for home-based autism therapy}, author = {Hoang-Long Cao and Cristina A. Pop and Ramona { Simut Vanderborght } and Raphael Furnemont and Albert { De Beir } and Greet { Van De Perre } and Pablo { Gomez Esteban } and Dirk Lefeber and Bram Vanderborght}, editor = {Adriana Tapus and Elisabeth André and Jean-Claude Martin and François Ferland and Mehdi Ammi}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/probolino-a-portable-lowcost-social-device-for-homebased-autism-therapy(0ea075a7-6e41-4490-b3ba-2f0ca90613bf).html}, doi = {10.1007/978-3-319-25554-5_10}, isbn = {978-3-319-25553-8}, year = {2015}, date = {2015-08-10}, booktitle = {Proceedings of the 7th International Conference on Social Robotics}, pages = {93--102}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, abstract = {Recent research has shown that social robots are beneficial in therapeutic interventions for children with autism in clinical environment. For the generalization of the skills learned in therapy sessions outside the clinic or laboratory, the therapeutic process needs to be continued at home. Therefore, social robotic devices should be designed with smaller sizes, lower cost, and higher level of autonomy. This paper presents the development of Probolino, a portable and low-cost social robotic device based on the social robot Probo. The system functions as a "robotic cognitive orthotic" which is an intermediate step between a computer and a robot without motion. Interactive games are developed to help children with autism spectrum disorders make social decisions in daily activities. These activities are configured in a time-line by therapists or parents via a web interface. Probolino is expected to enhance the efficiency of current robot-assisted autism therapy. Recent research has shown that social robots are beneficial in therapeutic interventions for children with autism in clinical environment. For the generalization of the skills learned in therapy sessions outside the clinic or laboratory, the therapeutic process needs to be continued at home. Therefore, social robotic devices should be designed with smaller sizes, lower cost, and higher level of autonomy. This paper presents the development of Probolino, a portable and low-cost social robotic device based on the social robot Probo. The system functions as a "robotic cognitive orthotic" which is an intermediate step between a computer and a robot without motion. Interactive games are developed to help children with autism spectrum disorders make social decisions in daily activities. These activities are configured in a time-line by therapists or parents via a web interface. Probolino is expected to enhance the efficiency of current robot-assisted autism therapy.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Recent research has shown that social robots are beneficial in therapeutic interventions for children with autism in clinical environment. For the generalization of the skills learned in therapy sessions outside the clinic or laboratory, the therapeutic process needs to be continued at home. Therefore, social robotic devices should be designed with smaller sizes, lower cost, and higher level of autonomy. This paper presents the development of Probolino, a portable and low-cost social robotic device based on the social robot Probo. The system functions as a "robotic cognitive orthotic" which is an intermediate step between a computer and a robot without motion. Interactive games are developed to help children with autism spectrum disorders make social decisions in daily activities. These activities are configured in a time-line by therapists or parents via a web interface. Probolino is expected to enhance the efficiency of current robot-assisted autism therapy. Recent research has shown that social robots are beneficial in therapeutic interventions for children with autism in clinical environment. For the generalization of the skills learned in therapy sessions outside the clinic or laboratory, the therapeutic process needs to be continued at home. Therefore, social robotic devices should be designed with smaller sizes, lower cost, and higher level of autonomy. This paper presents the development of Probolino, a portable and low-cost social robotic device based on the social robot Probo. The system functions as a "robotic cognitive orthotic" which is an intermediate step between a computer and a robot without motion. Interactive games are developed to help children with autism spectrum disorders make social decisions in daily activities. These activities are configured in a time-line by therapists or parents via a web interface. Probolino is expected to enhance the efficiency of current robot-assisted autism therapy. |
Peca, Andreea; Simut Vanderborght, Ramona ; Pintea, Sebastian; Vanderborght, Bram Are children with ASD more prone to test the intentions of the Robonova robot compared to a human? Journal Article In: International Journal of Social Robotics, 7 (5), pp. 629–639, 2015, ISSN: 1875-4791. @article{peca2015, title = {Are children with ASD more prone to test the intentions of the Robonova robot compared to a human?}, author = {Andreea Peca and Ramona { Simut Vanderborght } and Sebastian Pintea and Bram Vanderborght}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/are-children-with-asd-more-prone-to-test-the-intentions-of-the-robonova-robot-compared-to-a-human(628967bf-bcbe-4fdb-b5c7-b8d1ccc0e8c3).html}, issn = {1875-4791}, year = {2015}, date = {2015-08-09}, journal = {International Journal of Social Robotics}, volume = {7}, number = {5}, pages = {629--639}, abstract = {This study proposes an innovative device specifically designed for investigating the ability of children with autism spectrum disorder (ASD) to test the intention of a partner in a dyadic interactive game. 21 children with ASD were exposed to both a contingent and a noncontingent interaction condition with either a human agent or the Robonova robot as partners. The statistical analysis indicates a strong tendency toward a significant higher frequency of testing behaviors in the robot noncontingent condition and no difference between the two groups (robot/human) for the contingent condition. Children with ASD showed significantly more eye gaze to the robot compared to the human agent in both contingent and noncontingent conditions, but no difference in affect was found. The high level of initiations recorded in all conditions suggests that the game has a high motivational value for ASD children. Further longitudinal studies should investigate if such synchronous interaction games lead to improved shared intentionality in children with ASD. AB - This study proposes an innovative device specifically designed for investigating the ability of children with autism spectrum disorder (ASD) to test the intention of a partner in a dyadic interactive game. 21 children with ASD were exposed to both a contingent and a noncontingent interaction condition with either a human agent or the Robonova robot as partners. The statistical analysis indicates a strong tendency toward a significant higher frequency of testing behaviors in the robot noncontingent condition and no difference between the two groups (robot/human) for the contingent condition. Children with ASD showed significantly more eye gaze to the robot compared to the human agent in both contingent and noncontingent conditions, but no difference in affect was found. The high level of initiations recorded in all conditions suggests that the game has a high motivational value for ASD children. Further longitudinal studies should investigate if such synchronous interaction games lead to improved shared intentionality in children with ASD}, keywords = {}, pubstate = {published}, tppubtype = {article} } This study proposes an innovative device specifically designed for investigating the ability of children with autism spectrum disorder (ASD) to test the intention of a partner in a dyadic interactive game. 21 children with ASD were exposed to both a contingent and a noncontingent interaction condition with either a human agent or the Robonova robot as partners. The statistical analysis indicates a strong tendency toward a significant higher frequency of testing behaviors in the robot noncontingent condition and no difference between the two groups (robot/human) for the contingent condition. Children with ASD showed significantly more eye gaze to the robot compared to the human agent in both contingent and noncontingent conditions, but no difference in affect was found. The high level of initiations recorded in all conditions suggests that the game has a high motivational value for ASD children. Further longitudinal studies should investigate if such synchronous interaction games lead to improved shared intentionality in children with ASD. AB - This study proposes an innovative device specifically designed for investigating the ability of children with autism spectrum disorder (ASD) to test the intention of a partner in a dyadic interactive game. 21 children with ASD were exposed to both a contingent and a noncontingent interaction condition with either a human agent or the Robonova robot as partners. The statistical analysis indicates a strong tendency toward a significant higher frequency of testing behaviors in the robot noncontingent condition and no difference between the two groups (robot/human) for the contingent condition. Children with ASD showed significantly more eye gaze to the robot compared to the human agent in both contingent and noncontingent conditions, but no difference in affect was found. The high level of initiations recorded in all conditions suggests that the game has a high motivational value for ASD children. Further longitudinal studies should investigate if such synchronous interaction games lead to improved shared intentionality in children with ASD |
Cai, Haibin; Yu, Hui; Yao, Chunyan; Chen, Shengyong; Liu, Honghai Conlution-Based Means of Gradient for Fast Eye Centre Localization Inproceedings In: 7th International Conference on Machine Learning and Cybernetics, Guangzhou,China, 2015. @inproceedings{MachineLearningandCybernetics, title = { Conlution-Based Means of Gradient for Fast Eye Centre Localization}, author = {Haibin Cai and Hui Yu and Chunyan Yao and Shengyong Chen and Honghai Liu}, year = {2015}, date = {2015-07-12}, booktitle = {7th International Conference on Machine Learning and Cybernetics}, address = {Guangzhou,China}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Cai, Haibin; Liu, Bangli; Zhang, Jianhua; Chen, Shengyong; Liu, Honghai Visual Focus of Attention Estimation Using Eye Center Localization Journal Article In: Systems Journal,IEEE, PP (99), pp. 1-6, 2015, ISSN: 1932-8184 . @article{SystemsJournal, title = {Visual Focus of Attention Estimation Using Eye Center Localization}, author = {Haibin Cai and Bangli Liu and Jianhua Zhang and Shengyong Chen and Honghai Liu}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=7151779&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel7%2F4267003%2F4357939%2F07151779.pdf%3Farnumber%3D7151779}, issn = {1932-8184 }, year = {2015}, date = {2015-07-07}, journal = { Systems Journal,IEEE}, volume = {PP}, number = {99}, pages = {1-6}, abstract = {Estimating people visual focus of attention (VFOA) plays a crucial role in various practical systems such as human–robot interaction. It is challenging to extract the cue of the VFOA of a person due to the difficulty of recognizing gaze directionality. In this paper, we propose an improved integrodifferential approach to represent gaze via efficiently and accurately localizing the eye center in lower resolution image. The proposed method takes advantage of the drastic intensity changes between the iris and the sclera and the grayscale of the eye center as well. The number of kernels is optimized to convolute the original eye region image, and the eye center is located via searching the maximum ratio derivative of the neighbor curve magnitudes in the convolution image. Experimental results confirm that the algorithm outperforms the state-of-the-art methods in terms of computational cost, accuracy, and robustness to illumination changes.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Estimating people visual focus of attention (VFOA) plays a crucial role in various practical systems such as human–robot interaction. It is challenging to extract the cue of the VFOA of a person due to the difficulty of recognizing gaze directionality. In this paper, we propose an improved integrodifferential approach to represent gaze via efficiently and accurately localizing the eye center in lower resolution image. The proposed method takes advantage of the drastic intensity changes between the iris and the sclera and the grayscale of the eye center as well. The number of kernels is optimized to convolute the original eye region image, and the eye center is located via searching the maximum ratio derivative of the neighbor curve magnitudes in the convolution image. Experimental results confirm that the algorithm outperforms the state-of-the-art methods in terms of computational cost, accuracy, and robustness to illumination changes. |
Simut Vanderborght, Ramona ; Vanderfaeillie, Johan; Peca, Andreea; Van De Perre, Greet ; Vanderborght, Bram Children with autism spectrum disorders make a fruit salad with Probo, the social robot. An interaction study Journal Article In: Journal of Autism and Developmental Disorders, 46 (1), pp. 113–126, 2015, ISSN: 0162-3257. @article{simut2015probo, title = {Children with autism spectrum disorders make a fruit salad with Probo, the social robot. An interaction study}, author = {Ramona { Simut Vanderborght } and Johan Vanderfaeillie and Andreea Peca and Greet { Van De Perre } and Bram Vanderborght}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/children-with-autism-spectrum-disorders-make-a-fruit-salad-with-probo-the-social-robot-an-interaction-study(e32ef87c-9fdd-4999-b810-c51414d586c5).html}, issn = {0162-3257}, year = {2015}, date = {2015-07-05}, journal = {Journal of Autism and Developmental Disorders}, volume = {46}, number = {1}, pages = {113--126}, abstract = {This exploratory study aims to put robotics in a clinical framework. It was investigated how children with ASD perform in a play task with a robotic partner compared with a human partner, which impact these two types of interaction have on the children's social behaviors and, whether the robot triggers social interaction with another adult. A repeated measures design was used with one of the largest sample of children with ASD (N=30) declared until now in this field. Most of our outcomes are interesting in that they contradict commonly held assumptions about HRI with children with ASD. Limitations and future directions are finally presented.}, keywords = {}, pubstate = {published}, tppubtype = {article} } This exploratory study aims to put robotics in a clinical framework. It was investigated how children with ASD perform in a play task with a robotic partner compared with a human partner, which impact these two types of interaction have on the children's social behaviors and, whether the robot triggers social interaction with another adult. A repeated measures design was used with one of the largest sample of children with ASD (N=30) declared until now in this field. Most of our outcomes are interesting in that they contradict commonly held assumptions about HRI with children with ASD. Limitations and future directions are finally presented. |
Senft, Emmanuel; Baxter, Paul; Belpaeme, Tony Human-Guided Learning of Social Action Selection for Robot-Assisted Therapy Journal Article In: Journal of Machine Learning Research, 43 , pp. 15-20, 2015, (Proceedings of The 4th Workshop on Machine Learning for Interactive Systems, 11th July, 2015 (Lille, France)). @article{Senft-MLIS-2015, title = {Human-Guided Learning of Social Action Selection for Robot-Assisted Therapy}, author = {Emmanuel Senft and Paul Baxter and Tony Belpaeme }, editor = {Heriberto Cuayáhuitl and Nina Dethlefs and Lutz Frommberger and Martijn { Van Otterlo } and Olivier Pietquin}, url = {http://jmlr.org/proceedings/papers/v43/senft15.pdf}, year = {2015}, date = {2015-07-01}, booktitle = {Proceedings of The 4th Workshop on Machine Learning for Interactive Systems}, journal = {Journal of Machine Learning Research}, volume = {43}, pages = {15-20}, abstract = {This paper presents a method for progressively increasing autonomous action selection capabilities in sensitive environments, where random exploration-based learning is not desirable, using guidance provided by a human supervisor. We describe the global framework and a simulation case study based on a scenario in Robot Assisted Therapy for children with Autism Spectrum Disorder. This simulation illustrates the functional features of our proposed approach, and demonstrates how a system following these principles adapts to different interaction contexts while maintaining an appropriate behaviour for the system at all times.}, note = {Proceedings of The 4th Workshop on Machine Learning for Interactive Systems, 11th July, 2015 (Lille, France)}, keywords = {}, pubstate = {published}, tppubtype = {article} } This paper presents a method for progressively increasing autonomous action selection capabilities in sensitive environments, where random exploration-based learning is not desirable, using guidance provided by a human supervisor. We describe the global framework and a simulation case study based on a scenario in Robot Assisted Therapy for children with Autism Spectrum Disorder. This simulation illustrates the functional features of our proposed approach, and demonstrates how a system following these principles adapts to different interaction contexts while maintaining an appropriate behaviour for the system at all times. |
Vernon, David; Beetz, Michael; Sandini, Giulio Prospection in cognition: the case for joint episodic-procedural memory in cognitive robotics Journal Article In: Frontiers in Robotics and AI, 2 (19), pp. 1-14, 2015. @article{Vernon2015, title = {Prospection in cognition: the case for joint episodic-procedural memory in cognitive robotics}, author = {David Vernon and Michael Beetz and Giulio Sandini}, url = {http://journal.frontiersin.org/article/10.3389/frobt.2015.00019/abstract}, year = {2015}, date = {2015-07-01}, journal = {Frontiers in Robotics and AI}, volume = {2}, number = {19}, pages = {1-14}, abstract = {Prospection lies at the core of cognition: it is the means by which an agent – a person or a cognitive robot – shifts its perspective from immediate sensory experience to anticipate future events, be they the actions of other agents or the outcome of its own actions. Prospection, accomplished by internal simulation, requires mechanisms for both perceptual imagery and motor imagery. While it is known that these two forms of imagery are tightly entwined in the mirror neuron system, we do not yet have an effective model of the mentalizing network which would provide a framework to integrate declarative episodic and procedural memory systems and to combine experiential knowledge with skillful know-how. Such a framework would be founded on joint perceptuo-motor representations. In this paper, we examine the case for this form of representation, contrasting sensory-motor theory with ideo-motor theory, and we discuss how such a framework could be realized by joint episodic-procedural memory. We argue that such a representation framework has several advantages for cognitive robots. Since episodic memory operates by recombining imperfectly recalled past experience, this allows it to simulate new or unexpected events. Furthermore, by virtue of its associative nature, joint episodic-procedural memory allows the internal simulation to be conditioned by current context, semantic memory, and the agent’s value system. Context and semantics constrain the combinatorial explosion of potential perception-action associations and allow effective action selection in the pursuit of goals, while the value system provides the motives that underpin the agent’s autonomy and cognitive development. This joint episodic-procedural memory framework is neutral regarding the final implementation of these episodic and procedural memories, which can be configured sub-symbolically as associative networks or symbolically as content-addressable image databases and databases of motor-control scripts.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Prospection lies at the core of cognition: it is the means by which an agent – a person or a cognitive robot – shifts its perspective from immediate sensory experience to anticipate future events, be they the actions of other agents or the outcome of its own actions. Prospection, accomplished by internal simulation, requires mechanisms for both perceptual imagery and motor imagery. While it is known that these two forms of imagery are tightly entwined in the mirror neuron system, we do not yet have an effective model of the mentalizing network which would provide a framework to integrate declarative episodic and procedural memory systems and to combine experiential knowledge with skillful know-how. Such a framework would be founded on joint perceptuo-motor representations. In this paper, we examine the case for this form of representation, contrasting sensory-motor theory with ideo-motor theory, and we discuss how such a framework could be realized by joint episodic-procedural memory. We argue that such a representation framework has several advantages for cognitive robots. Since episodic memory operates by recombining imperfectly recalled past experience, this allows it to simulate new or unexpected events. Furthermore, by virtue of its associative nature, joint episodic-procedural memory allows the internal simulation to be conditioned by current context, semantic memory, and the agent’s value system. Context and semantics constrain the combinatorial explosion of potential perception-action associations and allow effective action selection in the pursuit of goals, while the value system provides the motives that underpin the agent’s autonomy and cognitive development. This joint episodic-procedural memory framework is neutral regarding the final implementation of these episodic and procedural memories, which can be configured sub-symbolically as associative networks or symbolically as content-addressable image databases and databases of motor-control scripts. |
Wang, Yiming; Yu, Hui; Stevens, Brett; Liu, Honghai Dynamic Facial Expression Recognition Using Local Patch and LBP-TOP Conference Proceedings of the 8th International Conference on Human System Interaction, IEEE , Warsaw ,Poland, 2015. @conference{HumanSystemInteraction, title = { Dynamic Facial Expression Recognition Using Local Patch and LBP-TOP}, author = {Yiming Wang and Hui Yu and Brett Stevens and Honghai Liu}, url = {http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7170694&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7170694}, doi = {10.1109/HSI.2015.7170694 }, year = {2015}, date = {2015-06-25}, booktitle = {Proceedings of the 8th International Conference on Human System Interaction}, pages = {362-367}, publisher = {IEEE }, address = {Warsaw ,Poland}, abstract = {Local binary pattern on three orthogonal planes (LBP-TOP) is one of the most popular method for dynamic texture analysis and has been successfully applied to facial expression analysis. Yet an effective LBP-TOP operator highly relies on preprocessing. And, like many appearance-based approaches, this approach reserves more identity-related cues rather than expression. In this work, we propose a fully automatic approach for facial expression recognition based on points registration, localized patch extraction and LBP-TOP feature representation. The efficiency of this method is evaluated on CK+ database. Results show that the proposed method has achieved a better performance compared with existing methods.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Local binary pattern on three orthogonal planes (LBP-TOP) is one of the most popular method for dynamic texture analysis and has been successfully applied to facial expression analysis. Yet an effective LBP-TOP operator highly relies on preprocessing. And, like many appearance-based approaches, this approach reserves more identity-related cues rather than expression. In this work, we propose a fully automatic approach for facial expression recognition based on points registration, localized patch extraction and LBP-TOP feature representation. The efficiency of this method is evaluated on CK+ database. Results show that the proposed method has achieved a better performance compared with existing methods. |
Zhang, Shu; Yu, Hui; Dong, Junyu; Wang, Ting; Qi, Lin; Liu, Honghai Combining Kinect and PnP for Camera Pose Estimation Conference In: Proceedings of the 8th International Conference on Human System Interaction, IEEE , Warsaw,Poland, 2015. @conference{HumanSystemInteraction, title = { Combining Kinect and PnP for Camera Pose Estimation}, author = {Shu Zhang and Hui Yu and Junyu Dong and Ting Wang and Lin Qi and Honghai Liu}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7170693}, doi = {10.1109/HSI.2015.7170693}, year = {2015}, date = {2015-06-25}, booktitle = {In: Proceedings of the 8th International Conference on Human System Interaction}, pages = {357-361}, publisher = {IEEE }, address = { Warsaw,Poland}, abstract = {This paper presents a novel method to conduct camera pose estimation though combining Kinect and Perspective-n-points algorithms. Most existing camera pose estimation methods suffer from the errors caused by inevitable outliers between 2D-3D correspondences. To this end, we propose to use a random down sampling process to deal with outliers in this paper. The proposed method is divided into two main steps, which are 2D-3D correspondences generation and pose estimation. The method has been tested in a real project, and the experiment has shown encouraging results compared to the ground truth.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } This paper presents a novel method to conduct camera pose estimation though combining Kinect and Perspective-n-points algorithms. Most existing camera pose estimation methods suffer from the errors caused by inevitable outliers between 2D-3D correspondences. To this end, we propose to use a random down sampling process to deal with outliers in this paper. The proposed method is divided into two main steps, which are 2D-3D correspondences generation and pose estimation. The method has been tested in a real project, and the experiment has shown encouraging results compared to the ground truth. |
Van de Perre, Greet ; Van Damme, Michaël ; Lefeber, Dirk; Vanderborght, Bram Development of a generic method to generate upper-body emotional expressions for different social robots Journal Article In: Advanced Robotics, 29 (9), pp. 597-609, 2015. @article{van2015development, title = {Development of a generic method to generate upper-body emotional expressions for different social robots}, author = {Greet { Van de Perre } and Michaël { Van Damme } and Dirk Lefeber and Bram Vanderborght}, url = {http://www.tandfonline.com/doi/abs/10.1080/01691864.2015.1031697#.VcisIPmqpBc}, year = {2015}, date = {2015-05-05}, journal = {Advanced Robotics}, volume = {29}, number = {9}, pages = {597-609}, abstract = {To investigate the effect of gestures in human–robot interaction, a number of social robots capable of gesturing have been designed. Gestures are often preprogrammed off-line or generated by mapping motion capture data to the robot. Since these gestures are dependent on the robot’s joint configuration, they cannot be used for other robots. Therefore, when using a new robot platform with a different morphology, new joint trajectories to reach the desired postures need to be implemented. This method aims to minimize the workload when implementing gestures on a new robot platform and facilitate the sharing of gestures between different robots. The innovative aspect of this method is that it is constructed independently of any robot configuration, and therefore it can be used to generate gestures for different robot platforms. To calculate a gesture for a certain configuration, the developed method uses a set of target gestures listed in a database and maps them to that specific configuration. The database currently consists of a set of emotional expressions. The method was validated on the virtual model of different robots and an online survey was performed to evaluate the user’s perception of the output of the method. The results of this survey showed that the calculated gestures for a certain robot configuration well resemble the target gestures, and thus that our developed method to map gestures to different robot morphologies gives good results.}, keywords = {}, pubstate = {published}, tppubtype = {article} } To investigate the effect of gestures in human–robot interaction, a number of social robots capable of gesturing have been designed. Gestures are often preprogrammed off-line or generated by mapping motion capture data to the robot. Since these gestures are dependent on the robot’s joint configuration, they cannot be used for other robots. Therefore, when using a new robot platform with a different morphology, new joint trajectories to reach the desired postures need to be implemented. This method aims to minimize the workload when implementing gestures on a new robot platform and facilitate the sharing of gestures between different robots. The innovative aspect of this method is that it is constructed independently of any robot configuration, and therefore it can be used to generate gestures for different robot platforms. To calculate a gesture for a certain configuration, the developed method uses a set of target gestures listed in a database and maps them to that specific configuration. The database currently consists of a set of emotional expressions. The method was validated on the virtual model of different robots and an online survey was performed to evaluate the user’s perception of the output of the method. The results of this survey showed that the calculated gestures for a certain robot configuration well resemble the target gestures, and thus that our developed method to map gestures to different robot morphologies gives good results. |
Ju, Zhaojie; Gao, Dongxu; Cao, Jiangtao; Liu, Honghai A Novel Approach to Extract Hand Gesture Feature in Depth Images Journal Article In: Multimedia Tools and Applications, 2015, ISSN: 1573-7721. @article{MultimediaToolsandApplications, title = { A Novel Approach to Extract Hand Gesture Feature in Depth Images}, author = {Zhaojie Ju and Dongxu Gao and Jiangtao Cao and Honghai Liu }, url = {http://link.springer.com/article/10.1007%2Fs11042-015-2609-2}, doi = {10.1007/s11042-015-2609-2 }, issn = {1573-7721}, year = {2015}, date = {2015-04-24}, journal = {Multimedia Tools and Applications}, abstract = {This paper proposes a novel approach to extract human hand gesture features in real-time from RGB-D images based on the earth mover’s distance and Lasso algorithms. Firstly, hand gestures with hand edge contour are segmented using a contour length information based de-noise method. A modified finger earth mover’s distance algorithm is then applied applied to locate the palm image and extract fingertip features. Lastly and more importantly, a Lasso algorithm is proposed to effectively and efficiently extract the fingertip feature from a hand contour curve. Experimental results are discussed to demonstrate the effectiveness of the proposed approach.}, keywords = {}, pubstate = {published}, tppubtype = {article} } This paper proposes a novel approach to extract human hand gesture features in real-time from RGB-D images based on the earth mover’s distance and Lasso algorithms. Firstly, hand gestures with hand edge contour are segmented using a contour length information based de-noise method. A modified finger earth mover’s distance algorithm is then applied applied to locate the palm image and extract fingertip features. Lastly and more importantly, a Lasso algorithm is proposed to effectively and efficiently extract the fingertip feature from a hand contour curve. Experimental results are discussed to demonstrate the effectiveness of the proposed approach. |
Kennedy, James; Baxter, Paul; Belpaeme, Tony Can Less be More? The Impact of Robot Social Behaviour on Human Learning Inproceedings In: Proceedings of the 4th International Symposium on New Frontiers in HRI at AISB 2015, Canterbury, UK, 2015, ISBN: 3319255541, 9783319255545. @inproceedings{kennedy2015less, title = {Can Less be More? The Impact of Robot Social Behaviour on Human Learning}, author = {James Kennedy and Paul Baxter and Tony Belpaeme}, url = {http://www.cs.kent.ac.uk/events/2015/AISB2015/proceedings/hri/5-Kennedy-canlessbe.pdf}, isbn = {3319255541, 9783319255545}, year = {2015}, date = {2015-04-21}, booktitle = {Proceedings of the 4th International Symposium on New Frontiers in HRI at AISB 2015}, address = {Canterbury, UK}, abstract = {In a large number of human-robot interaction (HRI) stud- ies, the aim is often to improve the social behaviour of a robot in order to provide a better interaction experience. Increasingly, companion robots are not being used merely as interaction partners, but to also help achieve a goal. One such goal is education, which encompasses many other factors such as behaviour change and motivation. In this paper we question whether robot social behaviour helps or hinders in this context, and challenge an often underlying assumption that robot social behaviour and task outcomes are only positively related. Draw- ing on both human-human interaction and human-robot interaction studies we hypothesise a curvilinear relationship between social robot behaviour and human task performance in the short-term, highlighting a possible trade-off between social cues and learning. However, we posit that this relationship is likely to change over time, with longer interaction periods favouring more social robots}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In a large number of human-robot interaction (HRI) stud- ies, the aim is often to improve the social behaviour of a robot in order to provide a better interaction experience. Increasingly, companion robots are not being used merely as interaction partners, but to also help achieve a goal. One such goal is education, which encompasses many other factors such as behaviour change and motivation. In this paper we question whether robot social behaviour helps or hinders in this context, and challenge an often underlying assumption that robot social behaviour and task outcomes are only positively related. Draw- ing on both human-human interaction and human-robot interaction studies we hypothesise a curvilinear relationship between social robot behaviour and human task performance in the short-term, highlighting a possible trade-off between social cues and learning. However, we posit that this relationship is likely to change over time, with longer interaction periods favouring more social robots |
Senft, Emmanuel; Baxter, Paul; Kennedy, James; Belpaeme, Tony When is it better to give up? Towards Autonomous Action Selection for Robot Assisted ASD Therapy Inproceedings In: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts, pp. 197-198, ACM Press, 2015, ISBN: 978-1-4503-3318-4. @inproceedings{Senft2015a, title = {When is it better to give up? Towards Autonomous Action Selection for Robot Assisted ASD Therapy}, author = {Emmanuel Senft and Paul Baxter and James Kennedy and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Senft2015WhenBetterGiveUp.pdf}, doi = {10.1145/2701973.2702715}, isbn = {978-1-4503-3318-4}, year = {2015}, date = {2015-03-02}, booktitle = {Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts}, pages = {197-198}, publisher = {ACM Press}, abstract = {Robot Assisted Therapy (RAT) for children with ASD has found promising applications. In this paper, we outline an autonomous action selection mechanism to extend current RAT approaches. This will include the ability to revert control of the therapeutic intervention to the supervising therapist. We suggest that in order to maintain the goals of therapy, sometimes it is better if the robot gives up.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Robot Assisted Therapy (RAT) for children with ASD has found promising applications. In this paper, we outline an autonomous action selection mechanism to extend current RAT approaches. This will include the ability to revert control of the therapeutic intervention to the supervising therapist. We suggest that in order to maintain the goals of therapy, sometimes it is better if the robot gives up. |
Kennedy, James; Baxter, Paul; Belpaeme, Tony Head Pose Estimation is an Inadequate Replacement for Eye Gaze in Child-Robot Interaction Inproceedings In: Proceedings of the 10th ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts, pp. 35–36, ACM 2015. @inproceedings{kennedy2015head, title = {Head Pose Estimation is an Inadequate Replacement for Eye Gaze in Child-Robot Interaction}, author = { James Kennedy and Paul Baxter and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Kennedy2015HeadPose.pdf}, doi = {http://dx.doi.org/10.1145/2701973.2701988}, year = {2015}, date = {2015-03-02}, booktitle = {Proceedings of the 10th ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts}, pages = {35--36}, organization = {ACM}, abstract = {Gaze analysis of human-robot interactions can reveal much about the dynamics of the interaction and be a useful step in establishing levels of engagement and attention. Currently, much of this work has to be conducted manually through post-hoc video coding due to current limitations in non-invasive, real-time gaze tracking solutions. This paper assesses whether real-time head pose estimation from an RGB-D camera may be used in place of manual post-hoc coding of gaze direction. Using data collected from an experiment `in the wild', it is found that the proposed RGB-D based pose estimation method is neither accurate nor consistent enough to provide a reliable measure of gaze within human-robot interactions.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Gaze analysis of human-robot interactions can reveal much about the dynamics of the interaction and be a useful step in establishing levels of engagement and attention. Currently, much of this work has to be conducted manually through post-hoc video coding due to current limitations in non-invasive, real-time gaze tracking solutions. This paper assesses whether real-time head pose estimation from an RGB-D camera may be used in place of manual post-hoc coding of gaze direction. Using data collected from an experiment `in the wild', it is found that the proposed RGB-D based pose estimation method is neither accurate nor consistent enough to provide a reliable measure of gaze within human-robot interactions. |
Kennedy, James; Baxter, Paul; Belpaeme, Tony The Robot Who Tried Too Hard: Social Behaviour of a Robot Tutor Can Negatively Affect Child Learning Inproceedings In: Proceedings of the 10th ACM/IEEE International Conference on Human-Robot Interaction, pp. 67–74, ACM 2015. @inproceedings{kennedy2015robot, title = {The Robot Who Tried Too Hard: Social Behaviour of a Robot Tutor Can Negatively Affect Child Learning}, author = { James Kennedy and Paul Baxter and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Kennedy2015RobotTriedTooHard.pdf}, doi = {http://dx.doi.org/10.1145/2696454.2696457}, year = {2015}, date = {2015-03-02}, booktitle = {Proceedings of the 10th ACM/IEEE International Conference on Human-Robot Interaction}, pages = {67--74}, organization = {ACM}, abstract = {Social robots are finding increasing application in the domain of education, particularly for children, to support and augment learning opportunities. With an implicit assumption that social and adaptive behaviour is desirable, it is therefore of interest to determine precisely how these aspects of behaviour may be exploited in robots to support children in their learning. In this paper, we explore this issue by evaluating the effect of a social robot tutoring strategy with children learning about prime numbers. It is shown that the tutoring strategy itself leads to improvement, but that the presence of a robot employing this strategy amplifies this effect, resulting in significant learning. However, it was also found that children interacting with a robot using social and adaptive behaviours in addition to the teaching strategy did not learn a significant amount. These results indicate that while the presence of a physical robot leads to improved learning, caution is required when applying social behaviour to a robot in a tutoring context.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Social robots are finding increasing application in the domain of education, particularly for children, to support and augment learning opportunities. With an implicit assumption that social and adaptive behaviour is desirable, it is therefore of interest to determine precisely how these aspects of behaviour may be exploited in robots to support children in their learning. In this paper, we explore this issue by evaluating the effect of a social robot tutoring strategy with children learning about prime numbers. It is shown that the tutoring strategy itself leads to improvement, but that the presence of a robot employing this strategy amplifies this effect, resulting in significant learning. However, it was also found that children interacting with a robot using social and adaptive behaviours in addition to the teaching strategy did not learn a significant amount. These results indicate that while the presence of a physical robot leads to improved learning, caution is required when applying social behaviour to a robot in a tutoring context. |
Ziemke, Tom; Thill, Serge; Vernon, David Embodiment is a Double-Edged Sword in Human-Robot Interaction: Ascribed vs. Intrinsic Intentionality Conference Proc. Workshop on Cognition: A Bridge between Robotics and Interaction, ACM/IEEE Human Robot Interaction Conference (HRI 2015), Portland, USA , 2015. @conference{Ziemke2015, title = {Embodiment is a Double-Edged Sword in Human-Robot Interaction: Ascribed vs. Intrinsic Intentionality}, author = {Tom Ziemke and Serge Thill and David Vernon}, url = {http://dream2020.eu/wp-content/uploads/2015/08/15_Ziemke_Thill_Vernon_HRI.pdf}, year = {2015}, date = {2015-03-01}, booktitle = {Proc. Workshop on Cognition: A Bridge between Robotics and Interaction, ACM/IEEE Human Robot Interaction Conference (HRI 2015)}, pages = {9-10}, address = {Portland, USA }, abstract = {This very short paper makes a relatively simple point: The human embodied cognitive capacity / tendency to attribute intentionality, goals, etc. to others, and to interpret their behavior in intentional terms, is fundamental to many types of social interaction. There are at least two quite different conceptions of embodied cognition though, underlying much research in cognitive robotics and human-robot interaction, which also differ regarding whether robots (a) could actually have their ‘own’ intrinsic intentionality, or (b) could only be ascribed/attributed intentionality, similar to the way cartoon characters are. For robotics research as such the distinction might be secondary, and for philosophy of mind the questions might not be resolvable any time soon. For society and the general public, however, the issue potentially has quite significant social and ethical implications – therefore researchers might need to pay more attention to this than they have so far.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } This very short paper makes a relatively simple point: The human embodied cognitive capacity / tendency to attribute intentionality, goals, etc. to others, and to interpret their behavior in intentional terms, is fundamental to many types of social interaction. There are at least two quite different conceptions of embodied cognition though, underlying much research in cognitive robotics and human-robot interaction, which also differ regarding whether robots (a) could actually have their ‘own’ intrinsic intentionality, or (b) could only be ascribed/attributed intentionality, similar to the way cartoon characters are. For robotics research as such the distinction might be secondary, and for philosophy of mind the questions might not be resolvable any time soon. For society and the general public, however, the issue potentially has quite significant social and ethical implications – therefore researchers might need to pay more attention to this than they have so far. |
Zhang, Xuguang; He, H; Cao, S; Liu, Honghai Flow field texture representation-based motion segmentation for crowd counting Journal Article In: Machine Vision and Applications, 26 (7-8), pp. 871–883, 2015, ISSN: 0932-8092. @article{Zhang2015, title = {Flow field texture representation-based motion segmentation for crowd counting}, author = {Xuguang Zhang and H He and S Cao and Honghai Liu}, url = {https://researchportal.port.ac.uk/portal/en/publications/flow-field-texture-representationbased-motion-segmentation-for-crowd-counting(e2dee5c8-fb58-44a7-b8e2-47eb7e75dc65).html http://www.dream2020.eu/wp-content/uploads/2019/05/96919666.pdf}, doi = {10.1007/s00138-015-0703-0}, issn = {0932-8092}, year = {2015}, date = {2015-01-11}, journal = {Machine Vision and Applications}, volume = {26}, number = {7-8}, pages = {871–883}, publisher = {Springer}, abstract = {It is evident that crowd counting is one of bottlenecks for crowd-related computer vision theory and applications such as surveillance. Since accuracy of estimating crowd size dominantly depends on the performance of motion detection of pedestrians, this paper attacks the challenging problem mainly by proposing a motion segmentation method based on flow field texture representation. Firstly, the motion crowd and background are represented as different texture images by employing line integral convolution. Then information entropy is introduced to quantify the textures as different values so that the texture images can be segmented; further an optimal threshold is obtained via Otsu method to segment the binarization entropy image. Finally, the area of motion foreground pixels is calculated for each image in a crowd motion video. The size of the crowd is estimated by least squares fitting using abundant datum of foreground pixels’ area and the number of individuals in a crowd. Experimental results demonstrate that the proposed crowd counting method outperforms background subtraction, Gaussian mixture model and optical flow-based methods in terms of mean absolute error and mean relative error.}, keywords = {}, pubstate = {published}, tppubtype = {article} } It is evident that crowd counting is one of bottlenecks for crowd-related computer vision theory and applications such as surveillance. Since accuracy of estimating crowd size dominantly depends on the performance of motion detection of pedestrians, this paper attacks the challenging problem mainly by proposing a motion segmentation method based on flow field texture representation. Firstly, the motion crowd and background are represented as different texture images by employing line integral convolution. Then information entropy is introduced to quantify the textures as different values so that the texture images can be segmented; further an optimal threshold is obtained via Otsu method to segment the binarization entropy image. Finally, the area of motion foreground pixels is calculated for each image in a crowd motion video. The size of the crowd is estimated by least squares fitting using abundant datum of foreground pixels’ area and the number of individuals in a crowd. Experimental results demonstrate that the proposed crowd counting method outperforms background subtraction, Gaussian mixture model and optical flow-based methods in terms of mean absolute error and mean relative error. |
Coeckelbergh, Mark; Pop, Cristina A; Simut Vanderborght, Ramona ; Peca, Andreea; Pintea, Sebastian; David, Daniel; Vanderborght, Bram In: Science and Engineering Ethics, pp. 1-19, 2015, ISSN: 1353-3452. @article{Coeckelbergh2015, title = {A Survey of Expectations About the Role of Robots in Robot-Assisted Therapy for Children with ASD: Ethical Acceptability, Trust, Sociability, Appearance, and Attachment}, author = {Mark Coeckelbergh and Cristina A. Pop and Ramona { Simut Vanderborght } and Andreea Peca and Sebastian Pintea and Daniel David and Bram Vanderborght}, url = {http://dx.doi.org/10.1007/s11948-015-9649-x}, issn = {1353-3452}, year = {2015}, date = {2015-01-01}, journal = {Science and Engineering Ethics}, pages = {1-19}, publisher = {Springer Netherlands}, abstract = {The use of robots in therapy for children with autism spectrum disorder (ASD) raises issues concerning the ethical and social acceptability of this technology and, more generally, about human–robot interaction. However, usually philosophical papers on the ethics of human–robot-interaction do not take into account stakeholders’ views; yet it is important to involve stakeholders in order to render the research responsive to concerns within the autism and autism therapy community. To support responsible research and innovation in this field, this paper identifies a range of ethical, social and therapeutic concerns, and presents and discusses the results of an exploratory survey that investigated these issues and explored stakeholders’ expectations about this kind of therapy. We conclude that although in general stakeholders approve of using robots in therapy for children with ASD, it is wise to avoid replacing therapists by robots and to develop and use robots that have what we call supervised autonomy. This is likely to create more trust among stakeholders and improve the quality of the therapy. Moreover, our research suggests that issues concerning the appearance of the robot need to be adequately dealt with by the researchers and therapists. For instance, our survey suggests that zoomorphic robots may be less problematic than robots that look too much like humans.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The use of robots in therapy for children with autism spectrum disorder (ASD) raises issues concerning the ethical and social acceptability of this technology and, more generally, about human–robot interaction. However, usually philosophical papers on the ethics of human–robot-interaction do not take into account stakeholders’ views; yet it is important to involve stakeholders in order to render the research responsive to concerns within the autism and autism therapy community. To support responsible research and innovation in this field, this paper identifies a range of ethical, social and therapeutic concerns, and presents and discusses the results of an exploratory survey that investigated these issues and explored stakeholders’ expectations about this kind of therapy. We conclude that although in general stakeholders approve of using robots in therapy for children with ASD, it is wise to avoid replacing therapists by robots and to develop and use robots that have what we call supervised autonomy. This is likely to create more trust among stakeholders and improve the quality of the therapy. Moreover, our research suggests that issues concerning the appearance of the robot need to be adequately dealt with by the researchers and therapists. For instance, our survey suggests that zoomorphic robots may be less problematic than robots that look too much like humans. |
Kennedy, James; Baxter, Paul; Belpaeme, Tony Comparing Robot Embodiments in a Guided Discovery Learning Interaction with Children Journal Article In: International Journal of Social Robotics, 7 (2), pp. 293–308, 2015. @article{kennedy2015comparing, title = {Comparing Robot Embodiments in a Guided Discovery Learning Interaction with Children}, author = { James Kennedy and Paul Baxter and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Kennedy2015ComparingRobotEmbodiments.pdf}, doi = {http://dx.doi.org/10.1007/s12369-014-0277-4}, year = {2015}, date = {2015-01-01}, journal = {International Journal of Social Robotics}, volume = {7}, number = {2}, pages = {293--308}, publisher = {Springer}, abstract = {The application of social robots to the domain of education is becoming more prevalent. However, there remain a wide range of open issues, such as the effectiveness of robots as tutors on student learning outcomes, the role of social behaviour in teaching interactions, and how the embodiment of a robot influences the interaction. In this paper, we seek to explore children’s behaviour towards a robot tutor for children in a novel guided discovery learning interaction. Since the necessity of real robots (as opposed to virtual agents) in education has not been definitively established in the literature, the effect of robot embodiment is assessed. The results demonstrate that children overcome strong incorrect biases in the material to be learned, but with no significant differences between embodiment conditions. However, the data do suggest that the use of real robots carries an advantage in terms of social presence that could provide educational benefits.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The application of social robots to the domain of education is becoming more prevalent. However, there remain a wide range of open issues, such as the effectiveness of robots as tutors on student learning outcomes, the role of social behaviour in teaching interactions, and how the embodiment of a robot influences the interaction. In this paper, we seek to explore children’s behaviour towards a robot tutor for children in a novel guided discovery learning interaction. Since the necessity of real robots (as opposed to virtual agents) in education has not been definitively established in the literature, the effect of robot embodiment is assessed. The results demonstrate that children overcome strong incorrect biases in the material to be learned, but with no significant differences between embodiment conditions. However, the data do suggest that the use of real robots carries an advantage in terms of social presence that could provide educational benefits. |
Baxter, Paul; Matu, Silviu; Senft, Emmanuel; Costescu, Cristina; Kennedy, James; David, Daniel; Belpaeme, Tony Touchscreen-Mediated Child-Robot Interactions Applied to ASD Therapy Inproceedings In: Proceedings of the 1st International Conference on Social Robots in Therapy and Education, Almere, Netherlands, 2015. @inproceedings{baxter2015touchscreen, title = {Touchscreen-Mediated Child-Robot Interactions Applied to ASD Therapy}, author = { Paul Baxter and Silviu Matu and Emmanuel Senft and Cristina Costescu and James Kennedy and Daniel David and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Baxter2015TouchscreenMediated.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 1st International Conference on Social Robots in Therapy and Education}, address = {Almere, Netherlands}, abstract = {Robots are finding increasing application in the domain of ASD therapy as they provide a number of ad- vantageous properties such as replicability and controllable expressivity. In this abstract we introduce a role for touch- screens that act as mediating devices in therapeutic robot-child interactions. Informed by extensive work with neurotypical children in educational contexts, an initial study using a touchscreen mediator in support of robot-assisted ASD therapy was conducted to examine the feasibility of this approach, in so doing demonstrating how this application provides a number of technical and potentially therapeutic advantages.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Robots are finding increasing application in the domain of ASD therapy as they provide a number of ad- vantageous properties such as replicability and controllable expressivity. In this abstract we introduce a role for touch- screens that act as mediating devices in therapeutic robot-child interactions. Informed by extensive work with neurotypical children in educational contexts, an initial study using a touchscreen mediator in support of robot-assisted ASD therapy was conducted to examine the feasibility of this approach, in so doing demonstrating how this application provides a number of technical and potentially therapeutic advantages. |
Baxter, Paul; Ashurst, Emily; Kennedy, James; Senft, Emmanuel; Lemaignan, Séverin; Belpaeme, Tony The wider supportive role of social robots in the classroom for teachers Inproceedings In: Proceedings of the 1st International Workshop on Educational Robotics, at ICSR 2015, Paris, France, 2015. @inproceedings{baxter2015wider, title = {The wider supportive role of social robots in the classroom for teachers}, author = { Paul Baxter and Emily Ashurst and James Kennedy and Emmanuel Senft and Séverin Lemaignan and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Baxter2015WiderRobotsInClassroom.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 1st International Workshop on Educational Robotics, at ICSR 2015}, address = {Paris, France}, abstract = {Robots are being increasingly used in schools by researchers keen to assess how they may be used to facilitate learning and provide support. Based on 15 school experiment visits at 9 different schools in the U.K., we outline our observations, specifically focusing on the broader implications of robots in the classroom primarily from the perspective of the teacher. We then outline the basis for future research considerations in HRI, centred around the three themes of pedagogy, methodology, and ethics. For further application of robotics to education, we suggest that these three themes need to form a central part of continuing research.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Robots are being increasingly used in schools by researchers keen to assess how they may be used to facilitate learning and provide support. Based on 15 school experiment visits at 9 different schools in the U.K., we outline our observations, specifically focusing on the broader implications of robots in the classroom primarily from the perspective of the teacher. We then outline the basis for future research considerations in HRI, centred around the three themes of pedagogy, methodology, and ethics. For further application of robotics to education, we suggest that these three themes need to form a central part of continuing research. |
Kennedy, James; Baxter, Paul; Senft, Emmanuel; Belpaeme, Tony Using Immediacy to Characterise Robot Social Behaviour in Child-Robot Interactions Inproceedings In: Proceedings of the 1st Workshop on Evaluating Child-Robot Interaction, at ICSR 2015, Paris, France, 2015. @inproceedings{kennedy2015characterise, title = {Using Immediacy to Characterise Robot Social Behaviour in Child-Robot Interactions}, author = { James Kennedy and Paul Baxter and Emmanuel Senft and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Kennedy2015ImmediacyCharacterise.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 1st Workshop on Evaluating Child-Robot Interaction, at ICSR 2015}, address = {Paris, France}, abstract = {With an increasing number of researchers exploring Child-Robot Interaction (CRI), it becomes desirable to compare robot social behaviour between experi- mental scenarios. Such comparisons would allow researchers to better account for differences between studies in child responses to robots or interaction outcomes (such as learning or behaviour change) [4]. However, there are various problems associated with executing such evaluations in CRI contexts [2]. This extended abstract considers the use of immediacy questionnaires to characterise children’s perceptions of robot social behaviour. Challenges faced when using this measure are presented, along with some practical solutions and resources which could be of use to other researchers.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } With an increasing number of researchers exploring Child-Robot Interaction (CRI), it becomes desirable to compare robot social behaviour between experi- mental scenarios. Such comparisons would allow researchers to better account for differences between studies in child responses to robots or interaction outcomes (such as learning or behaviour change) [4]. However, there are various problems associated with executing such evaluations in CRI contexts [2]. This extended abstract considers the use of immediacy questionnaires to characterise children’s perceptions of robot social behaviour. Challenges faced when using this measure are presented, along with some practical solutions and resources which could be of use to other researchers. |
Senft, Emmanuel; Baxter, Paul; Kennedy, James; Belpaeme, Tony SPARC: Supervised Progressively Autonomous Robot Competencies Inproceedings In: Proceedings of the 7th International Conference on Social Robotics, pp. 603–612, Springer International Publishing, Paris, France, 2015. @inproceedings{senft2015sparc, title = {SPARC: Supervised Progressively Autonomous Robot Competencies}, author = { Emmanuel Senft and Paul Baxter and James Kennedy and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Senft2015SPARC.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 7th International Conference on Social Robotics}, pages = {603--612}, publisher = {Springer International Publishing}, address = {Paris, France}, series = {ICSR '15}, abstract = {The Wizard-of-Oz robot control methodology is widely used and typically places a high burden of effort and attention on the human supervisor to ensure appropriate robot behaviour, which may distract from other aspects of the task engaged in. We propose that this load can be reduced by enabling the robot to learn online from the guidance of the supervisor to become progressively more autonomous: Supervised Progressively Autonomous Robot Competencies (SPARC). Applying this concept to the domain of Robot Assisted Therapy (RAT) for children with Autistic Spectrum Disorder, a novel methodology is employed to assess the effect of a learning robot on the workload of the human supervisor. A user study shows that controlling a learning robot enables supervisors to achieve similar task performance as with a non-learning robot, but with both fewer interventions and a reduced perception of workload. These results demonstrate the utility of the SPARC concept and its potential effectiveness to reduce load on human WoZ supervisors.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The Wizard-of-Oz robot control methodology is widely used and typically places a high burden of effort and attention on the human supervisor to ensure appropriate robot behaviour, which may distract from other aspects of the task engaged in. We propose that this load can be reduced by enabling the robot to learn online from the guidance of the supervisor to become progressively more autonomous: Supervised Progressively Autonomous Robot Competencies (SPARC). Applying this concept to the domain of Robot Assisted Therapy (RAT) for children with Autistic Spectrum Disorder, a novel methodology is employed to assess the effect of a learning robot on the workload of the human supervisor. A user study shows that controlling a learning robot enables supervisors to achieve similar task performance as with a non-learning robot, but with both fewer interventions and a reduced perception of workload. These results demonstrate the utility of the SPARC concept and its potential effectiveness to reduce load on human WoZ supervisors. |
Kennedy, James; Baxter, Paul; Senft, Emmanuel; Belpaeme, Tony Higher Nonverbal Immediacy Leads to Greater Learning Gains in Child-Robot Tutoring Interactions Inproceedings In: Proceedings of the 7th International Conference on Social Robotics, pp. 327–336, Springer International Publishing, Paris, France, 2015. @inproceedings{kennedy2015higher, title = {Higher Nonverbal Immediacy Leads to Greater Learning Gains in Child-Robot Tutoring Interactions}, author = { James Kennedy and Paul Baxter and Emmanuel Senft and Tony Belpaeme}, url = {http://www.dream2020.eu/wp-content/uploads/2016/04/Kennedy2015HigherNonverbal.pdf}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 7th International Conference on Social Robotics}, pages = {327--336}, publisher = {Springer International Publishing}, address = {Paris, France}, series = {ICSR '15}, abstract = {Nonverbal immediacy has been positively correlated with cognitive learning gains in human-human interaction, but remains rela- tively under-explored in human-robot interaction contexts. This paper presents a study in which robot behaviour is derived from the principles of nonverbal immediacy. Both high and low immediacy behaviours are evaluated in a tutoring interaction with children where a robot teaches how to work out whether numbers are prime. It is found that children who interact with the robot exhibiting more immediate nonverbal be- haviour make significant learning gains, whereas those interacting with the less immediate robot do not. A strong trend is found suggesting that the children can perceive the differences between conditions, supporting results from existing work with adults.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Nonverbal immediacy has been positively correlated with cognitive learning gains in human-human interaction, but remains rela- tively under-explored in human-robot interaction contexts. This paper presents a study in which robot behaviour is derived from the principles of nonverbal immediacy. Both high and low immediacy behaviours are evaluated in a tutoring interaction with children where a robot teaches how to work out whether numbers are prime. It is found that children who interact with the robot exhibiting more immediate nonverbal be- haviour make significant learning gains, whereas those interacting with the less immediate robot do not. A strong trend is found suggesting that the children can perceive the differences between conditions, supporting results from existing work with adults. |
Zhou, Xiaolong; Yu, Hui; Liu, Honghai; Li, Youfu Tracking Multiple Video Targets with an Improved GM-PHD Tracker Journal Article In: Sensors, 15 (12), pp. 29794, 2015, ISSN: 1424-8220. @article{s151229794, title = {Tracking Multiple Video Targets with an Improved GM-PHD Tracker}, author = { Xiaolong Zhou and Hui Yu and Honghai Liu and Youfu Li}, url = {http://www.mdpi.com/1424-8220/15/12/29794}, doi = {10.3390/s151229794}, issn = {1424-8220}, year = {2015}, date = {2015-01-01}, journal = {Sensors}, volume = {15}, number = {12}, pages = {29794}, abstract = {Tracking multiple moving targets from a video plays an important role in many vision-based robotic applications. In this paper, we propose an improved Gaussian mixture probability hypothesis density (GM-PHD) tracker with weight penalization to effectively and accurately track multiple moving targets from a video. First, an entropy-based birth intensity estimation method is incorporated to eliminate the false positives caused by noisy video data. Then, a weight-penalized method with multi-feature fusion is proposed to accurately track the targets in close movement. For targets without occlusion, a weight matrix that contains all updated weights between the predicted target states and the measurements is constructed, and a simple, but effective method based on total weight and predicted target state is proposed to search the ambiguous weights in the weight matrix. The ambiguous weights are then penalized according to the fused target features that include spatial-colour appearance, histogram of oriented gradient and target area and further re-normalized to form a new weight matrix. With this new weight matrix, the tracker can correctly track the targets in close movement without occlusion. For targets with occlusion, a robust game-theoretical method is used. Finally, the experiments conducted on various video scenarios validate the effectiveness of the proposed penalization method and show the superior performance of our tracker over the state of the art.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Tracking multiple moving targets from a video plays an important role in many vision-based robotic applications. In this paper, we propose an improved Gaussian mixture probability hypothesis density (GM-PHD) tracker with weight penalization to effectively and accurately track multiple moving targets from a video. First, an entropy-based birth intensity estimation method is incorporated to eliminate the false positives caused by noisy video data. Then, a weight-penalized method with multi-feature fusion is proposed to accurately track the targets in close movement. For targets without occlusion, a weight matrix that contains all updated weights between the predicted target states and the measurements is constructed, and a simple, but effective method based on total weight and predicted target state is proposed to search the ambiguous weights in the weight matrix. The ambiguous weights are then penalized according to the fused target features that include spatial-colour appearance, histogram of oriented gradient and target area and further re-normalized to form a new weight matrix. With this new weight matrix, the tracker can correctly track the targets in close movement without occlusion. For targets with occlusion, a robust game-theoretical method is used. Finally, the experiments conducted on various video scenarios validate the effectiveness of the proposed penalization method and show the superior performance of our tracker over the state of the art. |
Cao, Hoang-Long; Gomez Esteban, Pablo ; De Beir, Albert ; Simut Vanderborght, Ramona ; Van De Perre, Greet ; Lefeber, Dirk; Vanderborght, Bram Toward a platform-independent social behavior architecture for multiple therapeutic scenarios Book Chapter In: Proceedings of the 1st International Conference on Social Robots in Therapy and Education, Windesheim Flevoland, 2015. @inbook{2ff3b2d940924232bf8ba4c700ab3111, title = {Toward a platform-independent social behavior architecture for multiple therapeutic scenarios}, author = {Hoang-Long Cao and Gomez Esteban, Pablo and De Beir, Albert and Simut Vanderborght, Ramona and Van De Perre, Greet and Dirk Lefeber and Bram Vanderborght}, url = {https://cris.cumulus.vub.ac.be/portal/en/publications/toward-a-platformindependent-social-behavior-architecture-for-multiple-therapeutic-scenarios(2ff3b2d9-4092-4232-bf8b-a4c700ab3111)/export.html}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 1st International Conference on Social Robots in Therapy and Education}, publisher = {Windesheim Flevoland}, abstract = {Researchers in cognitive robotics have developed different architectures to help social robots make decisions and express emotions autonomously in order to replace the pre-programmed and remote-controlled techniques. However, most of the current developments are ad-hoc solutions with no possibility to be utilized in multiple therapeutic situations. In this paper, we propose the design of a social behavior architecture, which aims at helping the social robot to sustain the user’s engagement and motivation, and to achieve the goal of interaction in different scenarios. Representations of behaviors are kept at abstract level and mapped with the robot’s morphology afterward. This approach ensures the architecture to be applicable to a wide range of social robots. }, keywords = {}, pubstate = {published}, tppubtype = {inbook} } Researchers in cognitive robotics have developed different architectures to help social robots make decisions and express emotions autonomously in order to replace the pre-programmed and remote-controlled techniques. However, most of the current developments are ad-hoc solutions with no possibility to be utilized in multiple therapeutic situations. In this paper, we propose the design of a social behavior architecture, which aims at helping the social robot to sustain the user’s engagement and motivation, and to achieve the goal of interaction in different scenarios. Representations of behaviors are kept at abstract level and mapped with the robot’s morphology afterward. This approach ensures the architecture to be applicable to a wide range of social robots. |
Peca, Andreea; Simut Vanderborght, Ramona ; Pintea, Sebastian; Costescu, Cristina A; Vanderborght, Bram How do typically developing children and children with autism perceive different social robots? Journal Article In: Computers in Human Behavior, 41 , pp. 268-277, 2014. @article{peca2014typically, title = {How do typically developing children and children with autism perceive different social robots?}, author = {Andreea Peca and Ramona { Simut Vanderborght } and Sebastian Pintea and Cristina A. Costescu and Bram Vanderborght}, url = {http://www.sciencedirect.com/science/article/pii/S0747563214004932}, year = {2014}, date = {2014-12-01}, journal = {Computers in Human Behavior}, volume = {41}, pages = {268-277}, abstract = {This study investigates the way children categorize different robots and their preferences for certain robots. For this aim, a matching pictures game in which 6 social robots are to be matched to one of the categories: machines, humans, animals and toys, was developed and implemented on a tablet device. A mixed factorial design with one within-subjects variable (type of robots) and two between-subjects variables (type of development and gender) was used. The data suggest that both TD and ASD children perceive robots mainly as toys, while children with ASD also perceive robots as machines. A high diversity of preferences for different robots was revealed, but also a high preference for simplified designs, with exaggerated facial features. This study provides an innovative instrument for studying children’s perception about social robots, and offers valuable information, with implications on the design of social robots.}, keywords = {}, pubstate = {published}, tppubtype = {article} } This study investigates the way children categorize different robots and their preferences for certain robots. For this aim, a matching pictures game in which 6 social robots are to be matched to one of the categories: machines, humans, animals and toys, was developed and implemented on a tablet device. A mixed factorial design with one within-subjects variable (type of robots) and two between-subjects variables (type of development and gender) was used. The data suggest that both TD and ASD children perceive robots mainly as toys, while children with ASD also perceive robots as machines. A high diversity of preferences for different robots was revealed, but also a high preference for simplified designs, with exaggerated facial features. This study provides an innovative instrument for studying children’s perception about social robots, and offers valuable information, with implications on the design of social robots. |
Yu, Hui; Liu, Honghai Linear Regression for Head Pose Analysis Conference Proceedings of IEEE World Congress on Computational Intelligence, IEEE , Beijing ,China, 2014, ISBN: 978-1-4799-6627-1. @conference{ComputationalIntelligence, title = {Linear Regression for Head Pose Analysis}, author = {Hui Yu and Honghai Liu}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6889676}, doi = {10.1109/IJCNN.2014.6889676}, isbn = {978-1-4799-6627-1}, year = {2014}, date = {2014-07-06}, booktitle = {Proceedings of IEEE World Congress on Computational Intelligence}, pages = {987-992}, publisher = {IEEE }, address = {Beijing ,China}, abstract = {Extensive research has been conducted to estimate and analyze head poses for various applications. Most existing methods tend to detect facial features and locate landmarks on a face for pose estimation. However, the sensitivity to occlusion of some face parts with key features and uncontrolled illumination of face images make the facial feature detection vulnerable. In this paper, we propose a framework for pose estimation without the need of face features or landmarks detection. Specifically, we formulate the pose estimation as a linear regression applied to the pose space. This method is based on the assumption that pose space cannot be linearly approximated in the pose subspace. The experimental results strongly support this assumption. In cases where the database does not obtain various poses in the intraclass, we propose to generate those poses through a 3D reconstruction and projection method. The experiment conducted on the CMU MultiPIE and IMM Face database has shown the effectiveness of the proposed method. }, keywords = {}, pubstate = {published}, tppubtype = {conference} } Extensive research has been conducted to estimate and analyze head poses for various applications. Most existing methods tend to detect facial features and locate landmarks on a face for pose estimation. However, the sensitivity to occlusion of some face parts with key features and uncontrolled illumination of face images make the facial feature detection vulnerable. In this paper, we propose a framework for pose estimation without the need of face features or landmarks detection. Specifically, we formulate the pose estimation as a linear regression applied to the pose space. This method is based on the assumption that pose space cannot be linearly approximated in the pose subspace. The experimental results strongly support this assumption. In cases where the database does not obtain various poses in the intraclass, we propose to generate those poses through a 3D reconstruction and projection method. The experiment conducted on the CMU MultiPIE and IMM Face database has shown the effectiveness of the proposed method. |
Yu, Hui; Ju, Zhaojie; Liu, Honghai Image Factorization and Feature Fusion for Enhancing Robot Vision in Human Face Recognition Conference Proceedings of IEEE World Congress on Computational Intelligence, IEEE , Beijing, China, 2014, ISBN: 978-1-4799-6627-1 . @conference{ComputationalIntelligence, title = {Image Factorization and Feature Fusion for Enhancing Robot Vision in Human Face Recognition}, author = {Hui Yu and Zhaojie Ju and Honghai Liu}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6889675}, doi = {10.1109/IJCNN.2014.6889675}, isbn = {978-1-4799-6627-1 }, year = {2014}, date = {2014-07-06}, booktitle = {Proceedings of IEEE World Congress on Computational Intelligence}, pages = {981 - 986}, publisher = {IEEE }, address = { Beijing, China}, abstract = {Illumination variation has been a challenging problem for face recognition in robot vision. To reduce the effect caused by illumination variation, a lot of studies have been explored. The Total Variation (TV) method is particular used to factorize images into a low frequency component and a high frequency one. However, the low frequency component still contains significant intrinsic features resulting in failure in face recognition in some cases. In this paper, we propose to further extract illumination invariant features from face images under uncontrolled varying lighting conditions. The Nonsampled Contourlet Transform (NSCT) method is employed to enhance the extraction of intrinsic feature. The combined factorization model is very effective in the experiment on the Yale database. }, keywords = {}, pubstate = {published}, tppubtype = {conference} } Illumination variation has been a challenging problem for face recognition in robot vision. To reduce the effect caused by illumination variation, a lot of studies have been explored. The Total Variation (TV) method is particular used to factorize images into a low frequency component and a high frequency one. However, the low frequency component still contains significant intrinsic features resulting in failure in face recognition in some cases. In this paper, we propose to further extract illumination invariant features from face images under uncontrolled varying lighting conditions. The Nonsampled Contourlet Transform (NSCT) method is employed to enhance the extraction of intrinsic feature. The combined factorization model is very effective in the experiment on the Yale database. |
Yu, Hui; Liu, Honghai Regression-Based Facial Expression Optimization Journal Article In: IEEE Transactions on Human-Machine Systems, 44 (3), pp. 386-394, 2014, ISSN: 2168-2291 . @article{IEEETransactionsonHuman-MachineSystems, title = { Regression-Based Facial Expression Optimization}, author = {Hui Yu and Honghai Liu }, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6798681}, doi = {10.1109/THMS.2014.2313912}, issn = {2168-2291 }, year = {2014}, date = {2014-04-15}, journal = { IEEE Transactions on Human-Machine Systems}, volume = {44}, number = {3}, pages = {386-394}, abstract = {This paper presents an approach for reproducing optimal 3-D facial expressions based on blendshape regression. It aims to improve fidelity of facial expressions but maintain the efficiency of the blendshape method, which is necessary for applications such as human-machine interaction and avatars. The method intends to optimize the given facial expression using action units (AUs) based on the facial action coding system recorded from human faces. To help capture facial movements for the target face, an intermediate model space is generated, where both the target and source AUs have the same mesh topology and vertex number. The optimization is conducted interactively in the intermediate model space through adjusting the regulating parameter. The optimized facial expression model is transferred back to the target facial model to produce the final facial expression. We demonstrate that given a sketched facial expression with rough vertex positions indicating the intended facial expression, the proposed method approaches the sketched facial expression through automatically selecting blendshapes with corresponding weights. The sketched expression model is finally approximated through AUs representing true muscle movements, which improves the fidelity of facial expressions. }, keywords = {}, pubstate = {published}, tppubtype = {article} } This paper presents an approach for reproducing optimal 3-D facial expressions based on blendshape regression. It aims to improve fidelity of facial expressions but maintain the efficiency of the blendshape method, which is necessary for applications such as human-machine interaction and avatars. The method intends to optimize the given facial expression using action units (AUs) based on the facial action coding system recorded from human faces. To help capture facial movements for the target face, an intermediate model space is generated, where both the target and source AUs have the same mesh topology and vertex number. The optimization is conducted interactively in the intermediate model space through adjusting the regulating parameter. The optimized facial expression model is transferred back to the target facial model to produce the final facial expression. We demonstrate that given a sketched facial expression with rough vertex positions indicating the intended facial expression, the proposed method approaches the sketched facial expression through automatically selecting blendshapes with corresponding weights. The sketched expression model is finally approximated through AUs representing true muscle movements, which improves the fidelity of facial expressions. |
Pop, Cristina A; Pintea, Sebastian; Vanderborght, Bram; David, Daniel Enhancing play skills, engagement and social skills in a play task in ASD children by using robot-based interventions. Journal Article In: Interaction Studies, 15 (2), pp. 71 - 91, 2014, ISSN: 1572-0373. @article{Pop2014, title = {Enhancing play skills, engagement and social skills in a play task in ASD children by using robot-based interventions.}, author = {Cristina A. Pop and Sebastian Pintea and Bram Vanderborght and Daniel David}, url = {http://dx.doi.org/10.1075/is.15.2.14pop}, issn = {1572-0373}, year = {2014}, date = {2014-01-01}, journal = {Interaction Studies}, volume = {15}, number = {2}, pages = {71 - 91}, abstract = {One of the most important deficits of children with autism spectrum disorder (ASD) is considered to be their social communication impairment (Weiss & Harris 2001). Prevalence of ASD is estimated to be 1 in every 88 children in the US (CDC, 2011). Individuals with ASD have impairments in social interaction, deficits in communication and restricted and repetitive behaviors and/or interests (DSM IV TR, 2000). Although early intensive behavioral intervention approaches are proven to be effective in reducing ASD symptoms (Peters-Scheffer, Didden, Korzilius & Sturmey 2011), there is more to be done in this field. More efficient early behavioral interventions focusing on children are needed to facilitate prog- ress at a later stage, ultimately allowing individuals with ASD to manage with com- plex social situations and to lead almost autonomous lives.}, keywords = {}, pubstate = {published}, tppubtype = {article} } One of the most important deficits of children with autism spectrum disorder (ASD) is considered to be their social communication impairment (Weiss & Harris 2001). Prevalence of ASD is estimated to be 1 in every 88 children in the US (CDC, 2011). Individuals with ASD have impairments in social interaction, deficits in communication and restricted and repetitive behaviors and/or interests (DSM IV TR, 2000). Although early intensive behavioral intervention approaches are proven to be effective in reducing ASD symptoms (Peters-Scheffer, Didden, Korzilius & Sturmey 2011), there is more to be done in this field. More efficient early behavioral interventions focusing on children are needed to facilitate prog- ress at a later stage, ultimately allowing individuals with ASD to manage with com- plex social situations and to lead almost autonomous lives. |
Costescu, Cristina A; Vanderborght, Bram; David, Daniel The effects of robot-enhanced psychotherapy. A meta-analysis Journal Article In: Review of General Psychology , 18 (2), pp. 127 - 136, 2014, ISSN: 1089-2680. @article{Costescu2014, title = {The effects of robot-enhanced psychotherapy. A meta-analysis}, author = {Cristina A. Costescu and Bram Vanderborght and Daniel David}, url = {https://asknao.aldebaran.com/sites/default/files/publications/review_of_general_psychology_costescu_1.pdf}, issn = {1089-2680}, year = {2014}, date = {2014-01-01}, journal = {Review of General Psychology }, volume = {18}, number = {2}, pages = {127 - 136}, abstract = {Through this meta-analysis we aimed to provide an estimation of the overall effect of robot-enhanced therapy on psychological outcome for different populations, to provide average effect sizes on different outcomes, such as cognitive, behavioral and subjective, and to test possible moderators of effect size. From a total of 861 considered studies for this meta-analysis, only 12 were included because of the lack of studies that have reported quantitative data in this area and because of their primary focus on describing the process of robotic development rather than measuring psychological outcomes. We calculated Cohen’s d effect sizes for every outcome measure for which sufficient data were reported. The results show that robot-enhanced therapy yielded a medium effect size overall and, specifically on the behavioral level, indicating that 69% of patients in the control groups did worse than the average number of participants in the intervention group. More studies are needed with regard to specific outcomes to prove the efficacy of robot-enhanced therapy, but the overall results clearly support the use of robot- enhanced therapy for different populations.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Through this meta-analysis we aimed to provide an estimation of the overall effect of robot-enhanced therapy on psychological outcome for different populations, to provide average effect sizes on different outcomes, such as cognitive, behavioral and subjective, and to test possible moderators of effect size. From a total of 861 considered studies for this meta-analysis, only 12 were included because of the lack of studies that have reported quantitative data in this area and because of their primary focus on describing the process of robotic development rather than measuring psychological outcomes. We calculated Cohen’s d effect sizes for every outcome measure for which sufficient data were reported. The results show that robot-enhanced therapy yielded a medium effect size overall and, specifically on the behavioral level, indicating that 69% of patients in the control groups did worse than the average number of participants in the intervention group. More studies are needed with regard to specific outcomes to prove the efficacy of robot-enhanced therapy, but the overall results clearly support the use of robot- enhanced therapy for different populations. |
David, Daniel; Matu, Silviu; David, Oana Robot-Based Psychotherapy: Concepts Development, State of the Art, and New Directions Journal Article In: International Journal of Cognitive Therapy, 7 (2), pp. 192 - 210, 2014, ISSN: 1937-1209. @article{David2014, title = {Robot-Based Psychotherapy: Concepts Development, State of the Art, and New Directions}, author = {Daniel David and Silviu Matu and Oana David}, url = {http://guilfordjournals.com/doi/abs/10.1521/ijct.2014.7.2.192}, doi = {10.1521/ijct.2014.7.2.192}, issn = {1937-1209}, year = {2014}, date = {2014-01-01}, journal = {International Journal of Cognitive Therapy}, volume = {7}, number = {2}, pages = {192 - 210}, abstract = {In this article we propose new concepts and a framework of robot-based psychotherapy (or robotics-based psychotherapy; i.e., Robo-Therapist, Robo-Mediator, Robo-Assistant). We also review the state of the art of the field based on this framework and delineate possible but also needed future developments and integrations in the field. Implications for future research, theory development, and practice are discussed in the context of evidence-based psychotherapy (i.e., cognitive-behavioral therapy/CBT).}, keywords = {}, pubstate = {published}, tppubtype = {article} } In this article we propose new concepts and a framework of robot-based psychotherapy (or robotics-based psychotherapy; i.e., Robo-Therapist, Robo-Mediator, Robo-Assistant). We also review the state of the art of the field based on this framework and delineate possible but also needed future developments and integrations in the field. Implications for future research, theory development, and practice are discussed in the context of evidence-based psychotherapy (i.e., cognitive-behavioral therapy/CBT). |
Cao, Hoang-Long; Van de Perre, Greet ; Simut Vanderborght, Ramona ; Pop, Cristina A; Peca, Andreea; Lefeber, Dirk; Vanderborght, Bram Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK, 2014. @conference{Hoang-LongCao2014, title = {Enhancing My Keepon Robot: A Simple and Low-Cost Solution for Robot Platform in Human-Robot Interaction Studies}, author = {Hoang-Long Cao and Greet { Van de Perre } and Ramona { Simut Vanderborght } and Cristina A. Pop and Andreea Peca and Dirk Lefeber and Bram Vanderborght}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6926311}, year = {2014}, date = {2014-01-01}, booktitle = {Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication}, pages = {555-560}, address = {Edinburgh, UK}, abstract = {Many robots capable of performing social behaviors have recently been developed for Human-Robot Interaction (HRI) studies. These social robots are applied in various domains such as education, entertainment, medicine, and collaboration. Besides the undisputed advantages, a major difficulty in HRI studies with social robots is that the robot platforms are typically expensive and/or not open-source. It burdens researchers to broaden experiments to a larger scale or apply study results in practice. This paper describes a method to modify My Keepon, a toy version of Keepon robot, to be a programmable platform for HRI studies, especially for robot-assisted therapies. With an Arduino microcontroller board and an open-source Microsoft Visual C# software, users are able to fully control the sounds and motions of My Keepon, and configure the robot to the needs of their research. Peripherals can be added for advanced studies (e.g., mouse, keyboard, buttons, PlayStation2 console, Emotiv neuroheadset, Kinect). Our psychological experiment results show that My Keepon modification is a useful and low-cost platform for several HRI studies. }, keywords = {}, pubstate = {published}, tppubtype = {conference} } Many robots capable of performing social behaviors have recently been developed for Human-Robot Interaction (HRI) studies. These social robots are applied in various domains such as education, entertainment, medicine, and collaboration. Besides the undisputed advantages, a major difficulty in HRI studies with social robots is that the robot platforms are typically expensive and/or not open-source. It burdens researchers to broaden experiments to a larger scale or apply study results in practice. This paper describes a method to modify My Keepon, a toy version of Keepon robot, to be a programmable platform for HRI studies, especially for robot-assisted therapies. With an Arduino microcontroller board and an open-source Microsoft Visual C# software, users are able to fully control the sounds and motions of My Keepon, and configure the robot to the needs of their research. Peripherals can be added for advanced studies (e.g., mouse, keyboard, buttons, PlayStation2 console, Emotiv neuroheadset, Kinect). Our psychological experiment results show that My Keepon modification is a useful and low-cost platform for several HRI studies. |
Cao, Hoang-Long; Gomez Esteban, Pablo ; De Beir, Albert ; Simut Vanderborght, Ramona ; Van de Perre, Greet ; Lefeber, Dirk; Vanderborght, Bram Proceedings of the 2014 IEEE International Conference on Robotics and Biomimetics, IEEE, Bali, Indonesia, 2014. @conference{Hoang-LongCao2014a, title = {ROBEE: A homeostatic-based social behavior controller for robots in Human-Robot Interaction experiments}, author = {Hoang-Long Cao and Pablo { Gomez Esteban } and Albert { De Beir } and Ramona { Simut Vanderborght } and Greet { Van de Perre } and Dirk Lefeber and Bram Vanderborght}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7090383}, year = {2014}, date = {2014-01-01}, booktitle = {Proceedings of the 2014 IEEE International Conference on Robotics and Biomimetics}, pages = {516-521}, publisher = {IEEE}, address = {Bali, Indonesia}, abstract = {This paper presents the development and implementation of Robee, a novel social behavior controller for robots with a focus on Human-Robot Interaction (HRI) studies. Using the homeostatic drive theory, Robee selects the behaviors in order to maintain the needs, mainly psychological and social, within an acceptable range. We propose a hybrid concept for the decision making process, which combines the hierarchical approach and Parallel-rooted, Ordered, Slip-stack Hierarchical (POSH) architecture. Emotions are mapped in a two-dimensional space consisting of valence and arousal. A joint attention HRI experiment with children and NAO robot has been conducted showing the usage of the controller. Robee is expected to be implemented in more robotic platforms. }, keywords = {}, pubstate = {published}, tppubtype = {conference} } This paper presents the development and implementation of Robee, a novel social behavior controller for robots with a focus on Human-Robot Interaction (HRI) studies. Using the homeostatic drive theory, Robee selects the behaviors in order to maintain the needs, mainly psychological and social, within an acceptable range. We propose a hybrid concept for the decision making process, which combines the hierarchical approach and Parallel-rooted, Ordered, Slip-stack Hierarchical (POSH) architecture. Emotions are mapped in a two-dimensional space consisting of valence and arousal. A joint attention HRI experiment with children and NAO robot has been conducted showing the usage of the controller. Robee is expected to be implemented in more robotic platforms. |
Pop, Cristina A; Simut Vanderborght, Ramona ; Pintea, Sebastian; Saldien, Jelle; Rusu, Alina S; David, Daniel; Vanderfaeillie, Johan; Lefeber, Dirk; Vanderborght, Bram Can the Social Robot Probo help Children with Autism to Identify Situation-based Emotions? Journal Article In: International Journal of Humanoid Robotics, 10 (3), 2013, ISSN: 0219-8436. @article{Pop2013, title = {Can the Social Robot Probo help Children with Autism to Identify Situation-based Emotions?}, author = {Cristina A. Pop and Ramona { Simut Vanderborght } and Sebastian Pintea and Jelle Saldien and Alina S. Rusu and Daniel David and Johan Vanderfaeillie and Dirk Lefeber and Bram Vanderborght}, url = {http://www.worldscientific.com/doi/abs/10.1142/S0219843613500254}, doi = {10.1142/S0219843613500254}, issn = {0219-8436}, year = {2013}, date = {2013-01-01}, journal = {International Journal of Humanoid Robotics}, volume = {10}, number = {3}, abstract = {Children with autism spectrum disorders have difficulties in identifying situation-based emotions, which is a fundamental ability for mind reading. Social robots received increased attention as assisting tools for improving the social and emotional skills of children with autism. This study investigates whether the social robot Probo can help children with autism spectrum disorders to enhance their performance in identifying situation-based emotions. Three participants (age between 5 and 6) diagnozed with autism spectrum disorders were included in a single case AB experimental design, with intersubjects replications. The results show that children's performance improved with moderate to large effect sizes in identifying both sadness and happiness. Based on these results, we intend to perform more extensive investigations regarding the effectiveness of robot assisted therapy in improving social-emotional abilities for children with autism spectrum disorders.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Children with autism spectrum disorders have difficulties in identifying situation-based emotions, which is a fundamental ability for mind reading. Social robots received increased attention as assisting tools for improving the social and emotional skills of children with autism. This study investigates whether the social robot Probo can help children with autism spectrum disorders to enhance their performance in identifying situation-based emotions. Three participants (age between 5 and 6) diagnozed with autism spectrum disorders were included in a single case AB experimental design, with intersubjects replications. The results show that children's performance improved with moderate to large effect sizes in identifying both sadness and happiness. Based on these results, we intend to perform more extensive investigations regarding the effectiveness of robot assisted therapy in improving social-emotional abilities for children with autism spectrum disorders. |
Pop, Cristina A; Simut Vanderborght, Ramona ; Pintea, Sebastian; Saldien, Jelle; Rusu, Alina S; Vanderfaeillie, Johan; David, Daniel; Lefeber, Dirk; Vanderborght, Bram Social Robots vs. Computer Display: Does the Way Social Stories are Delivered Make a Difference for Their Effectiveness on ASD Children? Journal Article In: Journal of Educational Computing Research, 49 (3), pp. 381 - 401, 2013, ISSN: 0735-6331. @article{Pop2013, title = {Social Robots vs. Computer Display: Does the Way Social Stories are Delivered Make a Difference for Their Effectiveness on ASD Children? }, author = {Cristina A. Pop and Ramona { Simut Vanderborght } and Sebastian Pintea and Jelle Saldien and Alina S. Rusu and Johan Vanderfaeillie and Daniel David and Dirk Lefeber and Bram Vanderborght}, url = {http://jec.sagepub.com/content/49/3/381}, doi = {10.2190/EC.49.3.f}, issn = {0735-6331}, year = {2013}, date = {2013-01-01}, journal = {Journal of Educational Computing Research}, volume = {49}, number = {3}, pages = {381 - 401}, abstract = {Background and Objectives: The aim of this exploratory study is to test whether social stories presented by a social robot have a greater effect than ones presented on a computer display in increasing the independency in expressing social abilities of children with autism spectrum disorders (ASD). Although much progress has been made in developing interventions to improve social skills of children with ASD, a number of unresolved problems still remain. Social robots received increased attention as assisting tools for improving social skills on children with ASD. Methods: Twenty children with ASD (age between 4 and 9 years old) were randomly allocated to three groups: control group (n = 7), computer-presented social stories (n = 6), and robot assisted therapy (n = 7). Results: Overall, our data indicate that using the social robot to implement social story intervention was more effective for improving the independency of expressing social abilities for the participants, than the computer screen. Limitations: Future studies should include a bigger sample size, more intervention sessions, and a follow-up session in order to see if the effect persists in time. Conclusions: The preliminary outcomes of this exploratory research provide empirical bases for further investigations regarding the effectiveness of robot assisted therapy in improving social skills for children with autism through future randomized clinical trials. }, keywords = {}, pubstate = {published}, tppubtype = {article} } Background and Objectives: The aim of this exploratory study is to test whether social stories presented by a social robot have a greater effect than ones presented on a computer display in increasing the independency in expressing social abilities of children with autism spectrum disorders (ASD). Although much progress has been made in developing interventions to improve social skills of children with ASD, a number of unresolved problems still remain. Social robots received increased attention as assisting tools for improving social skills on children with ASD. Methods: Twenty children with ASD (age between 4 and 9 years old) were randomly allocated to three groups: control group (n = 7), computer-presented social stories (n = 6), and robot assisted therapy (n = 7). Results: Overall, our data indicate that using the social robot to implement social story intervention was more effective for improving the independency of expressing social abilities for the participants, than the computer screen. Limitations: Future studies should include a bigger sample size, more intervention sessions, and a follow-up session in order to see if the effect persists in time. Conclusions: The preliminary outcomes of this exploratory research provide empirical bases for further investigations regarding the effectiveness of robot assisted therapy in improving social skills for children with autism through future randomized clinical trials. |
Thill, Serge; Pop, Cristina A; Belpaeme, Tony; Ziemke, Tom; Vanderborght, Bram Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook Journal Article In: Paladyn, 3 (4), pp. 209-217, 2012, ISSN: 2080-9778. @article{, title = {Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook}, author = {Serge Thill and Cristina A. Pop and Tony Belpaeme and Tom Ziemke and Bram Vanderborght}, url = {http://dx.doi.org/10.2478/s13230-013-0107-7}, issn = {2080-9778}, year = {2012}, date = {2012-01-01}, journal = {Paladyn}, volume = {3}, number = {4}, pages = {209-217}, publisher = {SP Versita}, abstract = {Robot-assisted therapy (RAT) is an emerging field that has already seen some success and is likely to develop in the future. One particular application area is within therapies for autism spectrum disorders, in which the viability of the approach has been demonstrated. The present paper is a vision paper with the aim of identifying research directions in the near future of RAT. Specifically, we argue that the next step in such therapeutic scenarios is the development of more substantial levels of autonomy which would allow the robot to adapt to the individual needs of children over longer periods of time (while remaining under the ultimate supervision of a therapist). We argue that this requires new advances on the level of robot controllers as well as the ability to infer and classify intentions, goals and emotional states of the robot’s interactants. We show that the state of the art in a number of relevant disciplines is now at the point at which such an endeavour can be approached in earnest.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Robot-assisted therapy (RAT) is an emerging field that has already seen some success and is likely to develop in the future. One particular application area is within therapies for autism spectrum disorders, in which the viability of the approach has been demonstrated. The present paper is a vision paper with the aim of identifying research directions in the near future of RAT. Specifically, we argue that the next step in such therapeutic scenarios is the development of more substantial levels of autonomy which would allow the robot to adapt to the individual needs of children over longer periods of time (while remaining under the ultimate supervision of a therapist). We argue that this requires new advances on the level of robot controllers as well as the ability to infer and classify intentions, goals and emotional states of the robot’s interactants. We show that the state of the art in a number of relevant disciplines is now at the point at which such an endeavour can be approached in earnest. |
Vanderborght, Bram; Simut Vanderborght, Ramona ; Saldien, Jelle; Pop, Cristina A; Rusu, Alina S; Pintea, Sebastian; Lefeber, Dirk; David, Daniel Using the social robot probo as a social story telling agent for children with ASD Journal Article In: Interaction Studies, 13 (3), pp. 348 - 372, 2012. @article{vanderborght2012, title = {Using the social robot probo as a social story telling agent for children with ASD}, author = {Bram Vanderborght and Ramona { Simut Vanderborght } and Jelle Saldien and Cristina A. Pop and Alina S. Rusu and Sebastian Pintea and Dirk Lefeber and Daniel David}, url = {http://dx.doi.org/10.1075/is.13.3.02van}, year = {2012}, date = {2012-01-01}, journal = {Interaction Studies}, volume = {13}, number = {3}, pages = {348 - 372}, abstract = {This paper aims to study the role of the social robot Probo in providing assistance to a therapist for robot assisted therapy (RAT) with autistic children. Children with autism have difficulties with social interaction and several studies indicate that they show preference toward interaction with objects, such as computers and robots, rather than with humans. In 1991, Carol Gray developed Social Stories, an intervention tool aimed to increase children’s social skills. Social stories are short scenarios written or tailored for autistic individuals to help them understand and behave appropriately in social situations. This study shows that, in specific situations, the social performance of autistic children improves when using the robot Probo, as a medium for social story telling, than when a human reader tells the stories. The robot tells Social Stories to teach ASD children how to react in situations like saying “hello”, saying “thank you” and “sharing toys”. The robot has the capability of expressing emotions and attention via its facial expressions and its gaze. The paper discusses the use of Probo as an added-value therapeutic tool for social story telling and presents the first experimental results. Keywords: social robot; ASD children; social story; robot assisted therapy.}, keywords = {}, pubstate = {published}, tppubtype = {article} } This paper aims to study the role of the social robot Probo in providing assistance to a therapist for robot assisted therapy (RAT) with autistic children. Children with autism have difficulties with social interaction and several studies indicate that they show preference toward interaction with objects, such as computers and robots, rather than with humans. In 1991, Carol Gray developed Social Stories, an intervention tool aimed to increase children’s social skills. Social stories are short scenarios written or tailored for autistic individuals to help them understand and behave appropriately in social situations. This study shows that, in specific situations, the social performance of autistic children improves when using the robot Probo, as a medium for social story telling, than when a human reader tells the stories. The robot tells Social Stories to teach ASD children how to react in situations like saying “hello”, saying “thank you” and “sharing toys”. The robot has the capability of expressing emotions and attention via its facial expressions and its gaze. The paper discusses the use of Probo as an added-value therapeutic tool for social story telling and presents the first experimental results. Keywords: social robot; ASD children; social story; robot assisted therapy. |
Tapus, Adriana; Peca, Andreea; Aly, Amir; Pop, Cristina A; Jisa, Lavinia; Pintea, Sebastian; Rusu, Alina S; David, Daniel Children with autism social engagement in interaction with Nao, an imitative robot A series of single case experiments Journal Article In: Interaction Studies, 13 (3), pp. 315-347, 2012, ISSN: 1572-0373 . @article{Tapus2012, title = {Children with autism social engagement in interaction with Nao, an imitative robot A series of single case experiments}, author = {Adriana Tapus and Andreea Peca and Amir Aly and Cristina A. Pop and Lavinia Jisa and Sebastian Pintea and Alina S. Rusu and Daniel David }, url = {http://dx.doi.org/10.1075/is.13.3.01tap}, issn = {1572-0373 }, year = {2012}, date = {2012-01-01}, journal = {Interaction Studies}, volume = {13}, number = {3}, pages = {315-347}, abstract = {This paper presents a series of 4 single subject experiments aimed to investigate whether children with autism show more social engagement when interacting with the Nao robot, compared to a human partner in a motor imitation task. The Nao robot imitates gross arm movements of the child in real-time. Different behavioral criteria (i.e. eye gaze, gaze shifting, free initiations and prompted initiations of arm movements, and smile/laughter) were analyzed based on the video data of the interaction. The results are mixed and suggest a high variability in reactions to the Nao robot. The results are as follows: For Child2 and Child3, the results indicate no effect of the Nao robot in any of the target variables. Child1 and Child4 showed more eye gaze and smile/laughter in the interaction with the Nao robot compared to the human partner and Child1 showed a higher frequency of motor initiations in the interaction with the Nao robot compared to the baselines, but not with respect to the human-interaction. The robot proved to be a better facilitator of shared attention only for Child1. Keywords: human-robot interaction; assistive robotics; autism}, keywords = {}, pubstate = {published}, tppubtype = {article} } This paper presents a series of 4 single subject experiments aimed to investigate whether children with autism show more social engagement when interacting with the Nao robot, compared to a human partner in a motor imitation task. The Nao robot imitates gross arm movements of the child in real-time. Different behavioral criteria (i.e. eye gaze, gaze shifting, free initiations and prompted initiations of arm movements, and smile/laughter) were analyzed based on the video data of the interaction. The results are mixed and suggest a high variability in reactions to the Nao robot. The results are as follows: For Child2 and Child3, the results indicate no effect of the Nao robot in any of the target variables. Child1 and Child4 showed more eye gaze and smile/laughter in the interaction with the Nao robot compared to the human partner and Child1 showed a higher frequency of motor initiations in the interaction with the Nao robot compared to the baselines, but not with respect to the human-interaction. The robot proved to be a better facilitator of shared attention only for Child1. Keywords: human-robot interaction; assistive robotics; autism |
Liao, Yajie; Sun, Ying; Li, Gongfa; Kong, Jianyi; Jiang, Guozhang; Jiang, Du; Cai, Haibin; Ju, Zhaojie; Yu, Hui; Liu, Honghai Simultaneous Calibration: A Joint Optimization Approach for Multiple Kinect and External Cameras Journal Article In: Sensors, 17 (7), 0000, ISSN: 1424-8220. @article{Liao2017b, title = {Simultaneous Calibration: A Joint Optimization Approach for Multiple Kinect and External Cameras}, author = {Yajie Liao and Ying Sun and Gongfa Li and Jianyi Kong and Guozhang Jiang and Du Jiang and Haibin Cai and Zhaojie Ju and Hui Yu and Honghai Liu}, url = {https://www.mdpi.com/1424-8220/17/7/1491 http://www.dream2020.eu/wp-content/uploads/2019/05/sensors-17-01491.pdf}, doi = {10.3390/s17071491}, issn = {1424-8220}, journal = {Sensors}, volume = {17}, number = {7}, abstract = {Camera calibration is a crucial problem in many applications, such as 3D reconstruction, structure from motion, object tracking and face alignment. Numerous methods have been proposed to solve the above problem with good performance in the last few decades. However, few methods are targeted at joint calibration of multi-sensors (more than four devices), which normally is a practical issue in the real-time systems. In this paper, we propose a novel method and a corresponding workflow framework to simultaneously calibrate relative poses of a Kinect and three external cameras. By optimizing the final cost function and adding corresponding weights to the external cameras in different locations, an effective joint calibration of multiple devices is constructed. Furthermore, the method is tested in a practical platform, and experiment results show that the proposed joint calibration method can achieve a satisfactory performance in a project real-time system and its accuracy is higher than the manufacturer’s calibration.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Camera calibration is a crucial problem in many applications, such as 3D reconstruction, structure from motion, object tracking and face alignment. Numerous methods have been proposed to solve the above problem with good performance in the last few decades. However, few methods are targeted at joint calibration of multi-sensors (more than four devices), which normally is a practical issue in the real-time systems. In this paper, we propose a novel method and a corresponding workflow framework to simultaneously calibrate relative poses of a Kinect and three external cameras. By optimizing the final cost function and adding corresponding weights to the external cameras in different locations, an effective joint calibration of multiple devices is constructed. Furthermore, the method is tested in a practical platform, and experiment results show that the proposed joint calibration method can achieve a satisfactory performance in a project real-time system and its accuracy is higher than the manufacturer’s calibration. |
Chen, Disi; Li, Gongfa; Sun, Ying; Kong, Jianyi; Jiang, Guozhang; Tang, Heng; Ju, Zhaojie; Yu, Hui; Liu, Honghai An Interactive Image Segmentation Method in Hand Gesture Recognition Journal Article In: Sensors, 17 , pp. 253, 0000. @article{Chen2017, title = {An Interactive Image Segmentation Method in Hand Gesture Recognition}, author = {Disi Chen and Gongfa Li and Ying Sun and Jianyi Kong and Guozhang Jiang and Heng Tang and Zhaojie Ju and Hui Yu and Honghai Liu}, url = {https://researchportal.port.ac.uk/portal/en/publications/an-interactive-image-segmentation-method-in-hand-gesture-recognition(04d2ecd6-0a1f-4a27-87eb-e0946196ef48).html http://www.dream2020.eu/wp-content/uploads/2019/05/sensors_17_00253_1_.pdf}, doi = {10.3390/s17020253}, journal = {Sensors}, volume = {17}, pages = {253}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Phiri, C C; Ju, Z; Kubota, N; Liu, H Enhanced robot learning using fuzzy Q-Learning context-aware middleware Inproceedings In: 2016 International Symposium on Micro-NanoMechatronics and Human Science (MHS), pp. 1-8, 0000, ISSN: 2474-3771. @inproceedings{Phiri2016, title = {Enhanced robot learning using fuzzy Q-Learning context-aware middleware}, author = {C C Phiri and Z Ju and N Kubota and H Liu}, url = {https://ieeexplore.ieee.org/abstract/document/7824246}, doi = {10.1109/MHS.2016.7824246}, issn = {2474-3771}, booktitle = {2016 International Symposium on Micro-NanoMechatronics and Human Science (MHS)}, pages = {1-8}, abstract = {In this paper we continue with previous work by the authors implementing context-aware middleware to accelerate robot learning from demonstration, LfD. Specifically, we apply Fuzzy Q-Learning, FQL, reinforcement learning strategy to enhance the learning experience of the robot. Typically, fuzzy techniques allow the robot to make decisions without the need for an exhaustive map of the world. FQL, approximates the observable configuration space allowing the robot to overcome the high dimension challenge of feature decomposition and navigation in a stochastic environment.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In this paper we continue with previous work by the authors implementing context-aware middleware to accelerate robot learning from demonstration, LfD. Specifically, we apply Fuzzy Q-Learning, FQL, reinforcement learning strategy to enhance the learning experience of the robot. Typically, fuzzy techniques allow the robot to make decisions without the need for an exhaustive map of the world. FQL, approximates the observable configuration space allowing the robot to overcome the high dimension challenge of feature decomposition and navigation in a stochastic environment. |
Li, Gongfa; Liu, Jia; Jiang, Guozhang; Liu, Honghai Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material Journal Article In: Advances in Mechanical Engineering, 7 , 0000. @article{Li2015, title = {Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material}, author = {Gongfa Li and Jia Liu and Guozhang Jiang and Honghai Liu}, url = {https://www.researchgate.net/publication/276170083_Numerical_simulation_of_temperature_field_and_thermal_stress_field_in_the_new_type_of_ladle_with_the_nanometer_adiabatic_material http://www.dream2020.eu/wp-content/uploads/2019/05/AdvancesinMechanicalEngineering-2015-Li-.pdf}, doi = {10.1177/1687814015575988}, journal = {Advances in Mechanical Engineering}, volume = {7}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Yu, H; Liu, H Facial pose estimation via dense and sparse representation Inproceedings In: 2014 IEEE Symposium on Robotic Intelligence in Informationally Structured Space (RiiSS), pp. 1-6, 0000. @inproceedings{Yu2014, title = {Facial pose estimation via dense and sparse representation}, author = {H Yu and H Liu}, url = {https://ieeexplore.ieee.org/document/7009177 http://www.dream2020.eu/wp-content/uploads/2019/05/Pose_Analysis_via_Dense_and_Sparse_Representation.pdf}, doi = {10.1109/RIISS.2014.7009177}, booktitle = {2014 IEEE Symposium on Robotic Intelligence in Informationally Structured Space (RiiSS)}, pages = {1-6}, abstract = {Facial pose estimation is an important part for facial analysis such as face and facial expression recognition. In most existing methods, facial features are essential for facial pose estimation. However, occluded key features and uncontrolled illumination of face images make the facial feature detection vulnerable. In this paper, we propose methods for facial pose estimation via dense reconstruction and sparse representation but avoid localizing facial features. The Sparse Representation Classifier (SRC) method has achieved successful results in face recognition. In this paper, we explore SRC in pose estimation. Sparse representation learns a dictionary of base functions, so each input pose can be approximated by a linear combination of just a sparse subset of the bases. The experiment conducted on the CMU Multiple face database has shown the effectiveness of the proposed method.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Facial pose estimation is an important part for facial analysis such as face and facial expression recognition. In most existing methods, facial features are essential for facial pose estimation. However, occluded key features and uncontrolled illumination of face images make the facial feature detection vulnerable. In this paper, we propose methods for facial pose estimation via dense reconstruction and sparse representation but avoid localizing facial features. The Sparse Representation Classifier (SRC) method has achieved successful results in face recognition. In this paper, we explore SRC in pose estimation. Sparse representation learns a dictionary of base functions, so each input pose can be approximated by a linear combination of just a sparse subset of the bases. The experiment conducted on the CMU Multiple face database has shown the effectiveness of the proposed method. |
Gao, Qing; Liu, Jinguo; Ju, Zhaojie Robust real-time hand detection and localization for space human-robot interaction based on deep learning Journal Article In: Neurocomputing, 0000, ISSN: 0925-2312, (12 MONTH EMBARGO). @article{Gao2019, title = {Robust real-time hand detection and localization for space human-robot interaction based on deep learning}, author = {Qing Gao and Jinguo Liu and Zhaojie Ju}, url = {https://researchportal.port.ac.uk/portal/en/publications/robust-realtime-hand-detection-and-localization-for-space-humanrobot-interaction-based-on-deep-learning(dc48920d-9094-4d9f-a5be-0fe6300850ca).html}, issn = {0925-2312}, journal = {Neurocomputing}, publisher = {Elsevier}, abstract = {Hand gestures are quite suitable for space human-robot interaction (SHRI) because of their natural and convenient features. While the detection and localization of hands are the premise and foundation for SHRI based on hand gestures. But hand gestures are very complicated and hand sizes are very small in some images. These problems make the robust real-time hand detection and local- ization very difficult. In this paper, a feature-map-fused single shot multibox detector (FF-SSD) which is a deep learning network is designed to deal with the problems of hand detection and localization in SHRI. First, the background of the method is introduced in this paper, including an astronaut assistant robot platform, the difficulties of hand detection and localization, and introduction of the state-of-the-art deep learning networks for object detection and localiza- tion. Then, the FF-SSD is proposed for detecting and localizing hands especially pony-size hands. This network magentatakes into consideration both accuracy and speed with balanced performance. And in the experiment part, the FF-SSD is trained and tested on hand databases which include a homemade database and two public databases. At last, the superiority of the proposed method is demonstrated compared with the state-of-the-art methods.}, note = {12 MONTH EMBARGO}, keywords = {}, pubstate = {published}, tppubtype = {article} } Hand gestures are quite suitable for space human-robot interaction (SHRI) because of their natural and convenient features. While the detection and localization of hands are the premise and foundation for SHRI based on hand gestures. But hand gestures are very complicated and hand sizes are very small in some images. These problems make the robust real-time hand detection and local- ization very difficult. In this paper, a feature-map-fused single shot multibox detector (FF-SSD) which is a deep learning network is designed to deal with the problems of hand detection and localization in SHRI. First, the background of the method is introduced in this paper, including an astronaut assistant robot platform, the difficulties of hand detection and localization, and introduction of the state-of-the-art deep learning networks for object detection and localiza- tion. Then, the FF-SSD is proposed for detecting and localizing hands especially pony-size hands. This network magentatakes into consideration both accuracy and speed with balanced performance. And in the experiment part, the FF-SSD is trained and tested on hand databases which include a homemade database and two public databases. At last, the superiority of the proposed method is demonstrated compared with the state-of-the-art methods. |
Dai, Lihong; Liu, Jinguo; Ju, Zhaojie Visual Tracking and Positioning for an Astronaut Assistant Robot Inproceedings In: pp. 435-440, 0000. @inproceedings{Dai2018, title = {Visual Tracking and Positioning for an Astronaut Assistant Robot}, author = {Lihong Dai and Jinguo Liu and Zhaojie Ju}, url = {https://www.researchgate.net/publication/328987726_Visual_Tracking_and_Positioning_for_an_Astronaut_Assistant_Robot}, doi = {10.1109/IISR.2018.8535853}, pages = {435-440}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
