Optimization of Hand Gesture Classification Using the CNN Method with the Implementation of a Complementary Color-Based Landmark Strategy

Abstract

The growth of hand gesture recognition technology has positively impacted various sectors. However, classification errors often occur due to the similarity of gesture shapes, which are challenging for models to differentiate. This study aims to develop a classification method based on Convolutional Neural Network (CNN) using a landmark modification approach with complementary colors. This approach applies significant color contrast to enhance the model’s ability to extract unique features from similar hand gestures. The dataset used includes gestures with color modifications on landmarks using an HSV-based color wheel to create maximum contrast. The data is then processed through bounding box creation, resizing, and transfer learning using the Teachable Machine architecture. The study results show a significant improvement in classification accuracy for models with landmark modifications compared to those without. Metrics analysis, including precision, recall, and F1-score, confirms that this approach effectively reduces classification errors caused by similar hand gestures.