Sign language and drum based interaction game schemes were developed to perform in-depth analysis of different education based modalities with users from different demographics.īy performing usability studies and evaluations in a gamification, scheme using turn-taking games, the effectiveness of the described three methods were analyzed.
#SIMON SAYS ACTIONS ANDROID#
For this thesis, a signing avatar was developed to represent a portion of the social sciences course for Turkish primary education curriculum.įinally, a web and android based experimental framework was developed to perform customized human-machine interaction experiments. However, compared to videos, animated avatars offer several advantages like easy reproducibility of gesture sequences, control of point of view, adjusting the speed of the sign and being smaller in storage and bandwidth then videos. Like videos of signers, a signing avatar for TİD would allow communicating information in the form of visual gestures, thus becoming usable when the use of text is infeasible. One other educational tool we have developed is a signing avatar. The most powerful aspect of the robot is its embodiment that causes children to interact with the robot socially, rather than treating it as a simple, book like education tool. In this study, we developed a method, where two humanoid robots namely R3 and NAO are used to perform sign language gestures to communicate with children using sign languages with creating human-like hand, arm and body gestures. Children prefer to interact by touching during a training session.
Robots are often used in children's education by making use of their inherent personified nature compared to other non-human teaching mediums. One such technological medium that can be used to connect with such children are robots. To allow the use of such methods, three applications aimed at educating deaf children using the robot, the web, and avatar-based technologies was designed. While they are not a direct substitute for human teachers, technological support materials in education have been shown to motivate children and aid instructors. Hearing impaired students cannot use text and speech based technological, educational material that is becoming a crucial tool for modern education. The aim of this thesis is to present usable educational tools of students with special needs. The results showed that the physical robot tutor, the feedback and training mechanisms had a positive effect on the subjects’ performance and, as expected, although the physical medium is preferred, the virtual medium for drumming caused less failure. We verified the applicability and effectiveness of the proposed system in a drum-playing game with adult human test subjects by evaluating it both objectively and subjectively. As a part of those scenarios, a system that enables drum strokes to be automatically detected and recognized using auditory cues was implemented and incorporated into the experimental framework. Three interactive scenarios with different experimental setups for humanoid robots and mobile devices were developed and tested. For that purpose, we created an interactive drumming game relying on turn-taking and imitation principles, in which a human user is able to play drums with a humanoid robot (Robotic Drum Mate). Several parameters, including the effect of the embodiment of the tutor/interface and the presence of feedback and training mechanisms, were the focus of interest. It specifically focuses on evaluating robot/virtual avatar tutors, tangible interaction devices, and mobile multimedia devices within a simple drumming-based interactive music tutoring game scenario. This study is part of a project to design robot and avatar assistants for education and therapy, especially for children with special needs. The interactive multimodal drumming game is used in conjunction with humanoid robots to establish an audiovisual interactive interface. This paper presents a study of an audiovisual interface-based drumming system for multimodal human–robot interaction.
0 kommentar(er)
