Machine Learning with Synthetic Data – a New Way to Learn and Classify the Pictorial Augmented Reality Markers in Real-Time

Abstract

The idea of Augmented Reality (AR) appeared in the early 60s, which recently received a large amount of public attention. AR allows us to work, learn, play, and connect with the world around us both virtually and physically in real-time. However, picking the AR marker to match the users’ needs is one of the most challenging tasks due to different marker encryption/decryption methods and essential requirements. Barcode AR cards are fast and efficient, but they do not contain much visual information; pictorial coloured AR card, on the other hand, is slow and not reliable. This paper proposes a solution to obtain detectable arbitrary pictorial/colour AR cards in real-time by applying the benefit of machine learning and the power of synthetic data generation techniques. This technique solves the issue of labour-intensive tasks of manual annotations when building a massive training dataset of deep-learning. Thus, with a small number of input of the AR-enhanced target figures (as few as one for each coloured card), the synthetic data generated process will produce a deep-learning trainable dataset using computer-graphic rendering techniques (ten of thousands from just one image). Second, the generated dataset is then trained with a chosen object recognition convolutional neural network, acting as the AR marker tracking functionality. Our proposed idea works effectively well without modifying the original contents (of the chosen AR card). The benefits of using synthetic data generated techniques help us to improve the AR marker recognition accuracy and reduce the marker registration time. The trained model is capable of processing video sequences at approximately 25 frames per second without GPU Acceleration, which is suitable for AR experience on the mobile/web platform. We believed that it could be a promising low-cost AR approach in many areas, such as education and gaming.