Tweaks on Smartwatches Could Enable Hand Motion Recognition and New Apps

Smartwatches are devices that have digital functionalities compared to standard watches / Photo by Aleksey Boldin via 123RF


Wearable items like smartwatches are so common in tech shops nowadays. These gadgets have become very popular due to their capabilities to sense what is happening in the body of their wearers. But what about the events occurring inside their hands? A group of researchers investigated this and found a number of things that may lead to new apps.

The hand-sensing of smartwatches has been investigated on by researchers at Carnegie Mellon University. Their findings showed that maximizing the accelerometer in smartwatches could unlock new apps in the future. Results would be presented in the Association for Computing Machinery's Conference on Human Factors in Computing Systems.


The Functions of Smartwatches

According to, smartwatches are devices that have digital functionalities compared to standard watches. These features can allow the wearer to check for messages and weather information. But the prime capabilities of these devices are linked to the Internet of Things (IoT), as well as in sensing the wearer's body.

For example, a smartwatch can play music files using Bluetooth headphones. The user can now do activities, such as jogging or working out, without relying on their smartphones.

Another example is business wherein the user can check out their email accounts or set up appointments without using their phones. At the same time, the smartwatch can track down the user's diet, health status, and even stress levels, depending on the model.

Overall, IoT has allowed smartwatches to become a personal assistant and personal trainer of wearers. Their size and weight have made them a preferred choice for people who do not want to constantly carry their heavier smartphones.

At Carnegie, researchers acknowledged the well-known capabilities of smartwatches. However, they noticed that models in the market, including the high-end ones, do not support sensing of the user's hands. They discovered that the devices were not initially created for that function, and if tweaks were made, smartwatches could detect the hand activities of wearers.

"We envision smartwatches as a unique beachhead on the body for capturing rich, everyday activities. A wide variety of apps could be made smarter and more context-sensitive if our devices knew the activity of our bodies and hands," said Chris Harrison, a researcher of the project and director of the Future Interfaces Group.


Enabling Hand-sensing on Smartwatches

The intention of the project is to make smartwatches sense the hands of their owners to improve their intuitiveness in particular situations. When a user's hands are occupied, the devices should know that the person is unlikely to flick their wrist and look at the notifications.

For instance, an individual, who owns a smartphone and a smartwatch, is driving on the road. When either of the devices sends out a notification, in a scenario where the Do Not Disturb mode is disabled, the drivers should never turn their attention away from the road to avoid accidents. If the smartwatch has been tweaked, the device will recognize that the hands are busy and must not interrupt the user until their hands are free.

The intention of the project is to make smartwatches sense the hands of their owners to improve their intuitiveness in particular situations / Photo by Tsung-Lin Wu via 123RF


Additionally, the proper tweaks on the smartwatches would allow the devices to sense hand activities associated with health. These activities include brushing teeth, smoking cigarettes, and washing hands. Smartwatches could use hand gestures as factors to monitor the health habits of wearers.

The researchers applied tweaks on the operating system of a watch. The changes enabled the built-in accelerometer on the device to detect hand motions. In certain cases, the device could identify bio-acoustic sounds related to 25 various hand activities. They said that the tweaks could detect more than that.


Initial Results of Hand-sensing Watches

In this study, the team recruited 50 individuals to wear customized smartwatches, which they tweaked. They instructed participants to wear the watches for almost 1,000 hours and never to remove them while performing their usual daily tasks.

The smartwatches constantly recorded hand motion, hand orientation, and bio-acoustic information. Then, the devices would ask the wearers on what they were doing. Participants would describe their activities, such as clapping, shaving, scratching, putting on lipstick, among others. The descriptions provided by the participants were used to label over 80 hand activities, which resulted in a distinct dataset.

But the experiment conducted in the participants was done on their active arm, as opposed to where people typically wear their watches: on their passive arm. Future experiments would explore the hand motions in the passive arm.

Results from the active arm experiment on tweaked smartwatches showed:

- About 25 hand activities were recognized by the watches with 95 percent accuracy.

- The unique dataset provided information about potential new apps for smartwatches.

- Hand motions recorded by the custom devices were able to detect useful signatures in monitoring health conditions.

Researchers said that smartwatches with hand-motion recognition could offer a new range of feedback for wearers who are learning a new skill, recovering from an arm or hand injury, undergoing physical rehabilitation, expressing signs of adverse habits that may lead to strains, and showing possible symptoms of neurological disorders.

The team is expecting to focus on the different classes of hand activities in the future. Activities like smoking cessation, typing, and elder care are going to be explored in upcoming studies.