VCX: An industry initiative to create an objective camera module evaluation for mobile devices

Abstract

Due to the fast evolving technologies and the increasing importance of Social Media, the camera is one of the most important components of today’s mobile phones. Nowadays, smartphones are taking over a big share of the compact camera market. A simple reason for this might be revealed by the famous quote: “The best camera is the one that’s with you”. But with the vast choice of devices and great promises of manufacturers, there is a demand to characterize image quality and performance in very simple terms in order to provide information that helps choosing the best-suited device. The current existing evaluation systems are either not entirely objective or are under development and haven't reached a useful level yet. Therefore the industry itself has gotten together and created a new objective quality evaluation system named Valued Camera eXperience (VCX). It is designed to reflect the user experience regarding the image quality and the performance of a camera in a mobile device. Members of the initiative so fare are: Apple, Huawei, Image Engineering, LG, Mediatec, Nomicam, Oppo, TCL, Vivo, and Vodafone. Introduction Why another mobile camera evaluation standard? In fact the basis for VCX existed way before CPIQ or DxOMark. In the early 2000 Vodafone as one of the main carriers in Europe looked into the quality of cellphones, which they bundled with their contracts. One of the important parts of these phones were and still are the cameras. So Vodafone decided to define KPIs (key performance indicators) based on ISO standards to assess the quality of cell phone camera modules. To define the KPIs Vodafone needed to get a feeling about the camera performance and consulted Image Engineering to get some guidance and to help with tests. In 2013 Vodafone decided to take the KPIs to the next level. Cameras in cell phones had outgrown the former KPIs and a lot of new technologies had been implemented. Therefore an update was needed and Vodafone asked Image Engineering to update the physical measurements in order to get a complete picture of the camera performance. In the background Vodafone worked on converting the physical measurements into an objective quality rating system. At that time the system was called Vodafone Camera eXperience. In 2015 the system was updated according to the latest ISO standards and in 2016 Vodafone and Image Engineering decided that due to a lack of resources within Vodafone that Image Engineering should make the system public and move it forward under the neutral name Valued Camera eXperience. This was done at Photokina in Cologne in September 2016. The feedback and interest from the industry was so good that in late 2016 the idea was born to make this an open industry standard managed by the industry. So in March 2017 a conference was held in Duesseldorf and the decision was made to found a non profit organization named VCX-Forum e.V. Today VCX-Forum e.V. has X members that decide on the path forward in the future. Figure 1: The VCX roadmap. VCX is based on 5 tenets which guarantee results that can be mapped to real life experience 1. VCX measurements shall ensure out-of-the-box experience 2. VCX shall remain 100% objective 3. VCX shall be open and transparent 4. VCX shall employ/use an independent imaging lab for testing 5. VCX shall seek continuous improvement Tenet 1. (VCX measurements shall ensure out-of-the-box experience): This tenet dictates that the device under test shall be obtained from an unbiased/untainted source i.e., a random sample/s from a store that sells the device under test. This ensures that neither special samples from suppliers nor custom hardware/software are accepted. The results are obtained from a device that is launched onto the market. The device is tested using the default camera application and setting (except for flash test cases). If a need arises due to market forces and/or user demand that a device result be published at launch under the VCX umbrella, the result is clearly marked as “provisional”. Tenet 2. (VCX shall remain 100% objective): The complete process on how the score is created from measurements is based on objective analysis of the device under test, followed by a fixed and 2 | VCX | 1st conference Valued Camera Experience