UNAIR NEWS – The Faculty of Humanities UNAIR students won second prize in the national-level Education for All (EFA) essay competition. The competition was held by the State University of Surabaya. In their essay, Iqbal Rohim Al Farisi and his team wrote the idea of SignMotion app for the deaf and hard-of-hearing people.

Iqbal’s team consisted of two students, Nada Alifia Susandi and Marvelia Penta Surya from the Japanese Studies department. They managed to surpass 130 teams from different regions. They attempt to help deaf people to communicate with other individuals.

To UNAIR NEWS, Iqbal narrated the SignMotion application. It detects hand movements integrated with gesture recognition for the deaf. The gesture recognition allows computers to identify human body movements, making communication easier for the deaf.

Two best features of SignMotion

The application offers six features, including motion sensor and sound sensor, the two best features.

The motion sensor is integrated into smartphone technology, similar to the image sensor in a camera. This feature is supported by an Inertial Measurement Unit (IMU), which consists of a combination of an accelerometer and a gyroscope. It provides more accurate information about the detected movements. “This way, it can maximize the accuracy of translation from sign language, and vice versa,” Iqbal said.

The sound sensor uses an AI tool. Iqbal used an algorithmic model to translate text or speech into sign language movements. Thus, the detected speech will be converted by AI into graphic movements in sign language. These graphics involve animated models or gesture cues, mapping each word or phrase to corresponding sign language movements.

“It’s easier to access sign language when communicating with deaf people, and vice versa,” he said.

Author: Syifa Rahmadina

Editor: Feri Fenoria