Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-269
Basel Salahieh, Yi Wu, Oscar Nestares
{"title":"Light Field Perception Enhancement for Integral Displays","authors":"Basel Salahieh, Yi Wu, Oscar Nestares","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-269","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-269","url":null,"abstract":"","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"13 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126275391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-172
Dietmar Wueller, Uwe Artmann, V. Rao, G. Reif, J. Kramer, Fabian Knauf
Due to the fast evolving technologies and the increasing importance of Social Media, the camera is one of the most important components of today’s mobile phones. Nowadays, smartphones are taking over a big share of the compact camera market. A simple reason for this might be revealed by the famous quote: “The best camera is the one that’s with you”. But with the vast choice of devices and great promises of manufacturers, there is a demand to characterize image quality and performance in very simple terms in order to provide information that helps choosing the best-suited device. The current existing evaluation systems are either not entirely objective or are under development and haven't reached a useful level yet. Therefore the industry itself has gotten together and created a new objective quality evaluation system named Valued Camera eXperience (VCX). It is designed to reflect the user experience regarding the image quality and the performance of a camera in a mobile device. Members of the initiative so fare are: Apple, Huawei, Image Engineering, LG, Mediatec, Nomicam, Oppo, TCL, Vivo, and Vodafone. Introduction Why another mobile camera evaluation standard? In fact the basis for VCX existed way before CPIQ or DxOMark. In the early 2000 Vodafone as one of the main carriers in Europe looked into the quality of cellphones, which they bundled with their contracts. One of the important parts of these phones were and still are the cameras. So Vodafone decided to define KPIs (key performance indicators) based on ISO standards to assess the quality of cell phone camera modules. To define the KPIs Vodafone needed to get a feeling about the camera performance and consulted Image Engineering to get some guidance and to help with tests. In 2013 Vodafone decided to take the KPIs to the next level. Cameras in cell phones had outgrown the former KPIs and a lot of new technologies had been implemented. Therefore an update was needed and Vodafone asked Image Engineering to update the physical measurements in order to get a complete picture of the camera performance. In the background Vodafone worked on converting the physical measurements into an objective quality rating system. At that time the system was called Vodafone Camera eXperience. In 2015 the system was updated according to the latest ISO standards and in 2016 Vodafone and Image Engineering decided that due to a lack of resources within Vodafone that Image Engineering should make the system public and move it forward under the neutral name Valued Camera eXperience. This was done at Photokina in Cologne in September 2016. The feedback and interest from the industry was so good that in late 2016 the idea was born to make this an open industry standard managed by the industry. So in March 2017 a conference was held in Duesseldorf and the decision was made to found a non profit organization named VCX-Forum e.V. Today VCX-Forum e.V. has X members that decide on the path forward in the future. Figure 1: T
{"title":"VCX: An industry initiative to create an objective camera module evaluation for mobile devices","authors":"Dietmar Wueller, Uwe Artmann, V. Rao, G. Reif, J. Kramer, Fabian Knauf","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-172","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-172","url":null,"abstract":"Due to the fast evolving technologies and the increasing importance of Social Media, the camera is one of the most important components of today’s mobile phones. Nowadays, smartphones are taking over a big share of the compact camera market. A simple reason for this might be revealed by the famous quote: “The best camera is the one that’s with you”. But with the vast choice of devices and great promises of manufacturers, there is a demand to characterize image quality and performance in very simple terms in order to provide information that helps choosing the best-suited device. The current existing evaluation systems are either not entirely objective or are under development and haven't reached a useful level yet. Therefore the industry itself has gotten together and created a new objective quality evaluation system named Valued Camera eXperience (VCX). It is designed to reflect the user experience regarding the image quality and the performance of a camera in a mobile device. Members of the initiative so fare are: Apple, Huawei, Image Engineering, LG, Mediatec, Nomicam, Oppo, TCL, Vivo, and Vodafone. Introduction Why another mobile camera evaluation standard? In fact the basis for VCX existed way before CPIQ or DxOMark. In the early 2000 Vodafone as one of the main carriers in Europe looked into the quality of cellphones, which they bundled with their contracts. One of the important parts of these phones were and still are the cameras. So Vodafone decided to define KPIs (key performance indicators) based on ISO standards to assess the quality of cell phone camera modules. To define the KPIs Vodafone needed to get a feeling about the camera performance and consulted Image Engineering to get some guidance and to help with tests. In 2013 Vodafone decided to take the KPIs to the next level. Cameras in cell phones had outgrown the former KPIs and a lot of new technologies had been implemented. Therefore an update was needed and Vodafone asked Image Engineering to update the physical measurements in order to get a complete picture of the camera performance. In the background Vodafone worked on converting the physical measurements into an objective quality rating system. At that time the system was called Vodafone Camera eXperience. In 2015 the system was updated according to the latest ISO standards and in 2016 Vodafone and Image Engineering decided that due to a lack of resources within Vodafone that Image Engineering should make the system public and move it forward under the neutral name Valued Camera eXperience. This was done at Photokina in Cologne in September 2016. The feedback and interest from the industry was so good that in late 2016 the idea was born to make this an open industry standard managed by the industry. So in March 2017 a conference was held in Duesseldorf and the decision was made to found a non profit organization named VCX-Forum e.V. Today VCX-Forum e.V. has X members that decide on the path forward in the future. Figure 1: T","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"115 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124016282","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/issn.2470-1173.2018.05.pmii-183
Nora Pfund, Nitin Sampat, J. Viggiano
High quality, 360 capture for Cinematic VR is a relatively new and rapidly evolving technology. The field demands very high quality, distortion- free 360 capture which is not possible with cameras that depend on fish- eye lenses for capturing a 360 field of view. The Facebook Surround 360 Camera, one of the few “players” in this space, is an open-source license design that Facebook has released for anyone that chooses to build it from off-the-shelf components and generate 8K stereo output using open-source licensed rendering software. However, the components are expensive and the system itself is extremely demanding in terms of computer hardware and software. Because of this, there have been very few implementations of this design and virtually no real deployment in the field. We have implemented the system, based on Facebook’s design, and have been testing and deploying it in various situations; even generating short video clips. We have discovered in our recent experience that high quality, 360 capture comes with its own set of new challenges. As an example, even the most fundamental tools of photography like “exposure” become difficult because one is always faced with ultra-high dynamic range scenes (one camera is pointing directly at the sun and the others may be pointing to a dark shadow). The conventional imaging pipeline is further complicated by the fact that the stitching software has different effects on various as- pects of the calibration or pipeline optimization. Most of our focus to date has been on optimizing the imaging pipeline and improving the qual- ity of the output for viewing in an Oculus Rift headset. We designed a controlled experiment to study 5 key parameters in the rendering pipeline– black level, neutral balance, color correction matrix (CCM), geometric calibration and vignetting. By varying all of these parameters in a combinatorial manner, we were able to assess the relative impact of these parameters on the perceived image quality of the output. Our results thus far indicate that the output image quality is greatly influenced by the black level of the individual cameras (the Facebook cam- era comprised of 17 cameras whose output need to be stitched to obtain a 360 view). Neutral balance is least sensitive. We are most confused about the results we obtain from accurately calculating and applying the CCM for each individual camera. We obtained improved results by using the average of the matrices for all cameras. Future work includes evaluating the effects of geometric calibration and vignetting on quality.
{"title":"Relative Impact of Key Rendering Parameters on Perceived Quality of VR Imagery Captured by the Facebook Surround 360 Camera","authors":"Nora Pfund, Nitin Sampat, J. Viggiano","doi":"10.2352/issn.2470-1173.2018.05.pmii-183","DOIUrl":"https://doi.org/10.2352/issn.2470-1173.2018.05.pmii-183","url":null,"abstract":"High quality, 360 capture for Cinematic VR is a relatively new and rapidly evolving technology. The field demands very high quality, distortion- free 360 capture which is not possible with cameras that depend on fish- eye lenses for capturing a 360 field of view. The Facebook Surround 360 Camera, one of the few “players” in this space, is an open-source license design that Facebook has released for anyone that chooses to build it from off-the-shelf components and generate 8K stereo output using open-source licensed rendering software. However, the components are expensive and the system itself is extremely demanding in terms of computer hardware and software. Because of this, there have been very few implementations of this design and virtually no real deployment in the field. We have implemented the system, based on Facebook’s design, and have been testing and deploying it in various situations; even generating short video clips. We have discovered in our recent experience that high quality, 360 capture comes with its own set of new challenges. As an example, even the most fundamental tools of photography like “exposure” become difficult because one is always faced with ultra-high dynamic range scenes (one camera is pointing directly at the sun and the others may be pointing to a dark shadow). The conventional imaging pipeline is further complicated by the fact that the stitching software has different effects on various as- pects of the calibration or pipeline optimization. Most of our focus to date has been on optimizing the imaging pipeline and improving the qual- ity of the output for viewing in an Oculus Rift headset. We designed a controlled experiment to study 5 key parameters in the rendering pipeline– black level, neutral balance, color correction matrix (CCM), geometric calibration and vignetting. By varying all of these parameters in a combinatorial manner, we were able to assess the relative impact of these parameters on the perceived image quality of the output. Our results thus far indicate that the output image quality is greatly influenced by the black level of the individual cameras (the Facebook cam- era comprised of 17 cameras whose output need to be stitched to obtain a 360 view). Neutral balance is least sensitive. We are most confused about the results we obtain from accurately calculating and applying the CCM for each individual camera. We obtained improved results by using the average of the matrices for all cameras. Future work includes evaluating the effects of geometric calibration and vignetting on quality.","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124158457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/issn.2470-1173.2018.05.pmii-182
H. Dietz, C. Demaree, P. Eberhart, Chelsea Kuball, J. Wu
A multicamera, array camera, cluster camera, or “supercamera” incorporates two or more component cameras in a single system that functions as a camera with superior performance or special capabilities. Many camera arrays have been built by many organizations, yet creating an effective multicamera has not become significantly easier. This paper attempts to provide some useful insights toward simplifying the design, construction, and use of multicameras. Nine multicameras our group built for diverse purposes between 1999 and 2017 are described in some detail, including four built during Summer 2017 using some of the proposed simplifications.
{"title":"Lessons from design, construction, and use of various multicameras","authors":"H. Dietz, C. Demaree, P. Eberhart, Chelsea Kuball, J. Wu","doi":"10.2352/issn.2470-1173.2018.05.pmii-182","DOIUrl":"https://doi.org/10.2352/issn.2470-1173.2018.05.pmii-182","url":null,"abstract":"A multicamera, array camera, cluster camera, or “supercamera” incorporates two or more component cameras in a single system that functions as a camera with superior performance or special capabilities. Many camera arrays have been built by many organizations, yet creating an effective multicamera has not become significantly easier. This paper attempts to provide some useful insights toward simplifying the design, construction, and use of multicameras. Nine multicameras our group built for diverse purposes between 1999 and 2017 are described in some detail, including four built during Summer 2017 using some of the proposed simplifications.","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122719358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-442
Weijuan Xi, Huanzhao Zeng, Jonathan B. Phillips
{"title":"An Automatic Tuning Method for Camera Denoising and Sharpening based on a Perception Model","authors":"Weijuan Xi, Huanzhao Zeng, Jonathan B. Phillips","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-442","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-442","url":null,"abstract":"","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127159448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-161
H. Blasinski, J. Farrell, Trisha Lian, Zhenyi Liu, B. Wandell
Task requirements for image acquisition systems vary substantially between applications: requirements for consumer photography may be irrelevant or may even interfere with requirements for automotive, medical and other applications. The remarkable capabilities of the imaging industry to create lens and sensor designs for specific applications has been demonstrated in the mobile computing market. We might expect that the industry can further innovate if we specify the requirements for other markets. This paper explains an approach to developing image system designs that meet the task requirements for autonomous vehicle applications. It is impractical to build a large number of image acquisition systems and evaluate each of them with real driving data; therefore, we assembled a simulation environment to provide guidance at an early stage. The open-source and freely available software (isetcam, iset3d, and isetauto) uses ray tracing to compute quantitatively how scene radiance propagates through a multi-element lens to form the sensor irradiance. The software then transforms the irradiance into the sensor pixel responses, accounting for a large number of sensor parameters. This enables the user to apply different types of image processing pipelines to generate images that are used to train and test convolutional networks used in autonomous driving. We use the simulation environment to assess performance for different cameras and networks. Introduction The market for image sensors in autonomous vehicles can be divided into two segments. Some image sensor data is used as images to the passengers, such as rendering views from behind the car as the driver backs up. Other image sensor data is used by computational algorithms that guide the vehicle; the output from these sensors is never rendered for the human eye. It is reasonable to expect that the optical design, sensor parameters, and image processing pipeline for these two systems will differ. Mobile imaging applications for consumer photography dominate the market, driving the industry towards sensors with very small pixels (1 micron), a large number of pixels, a Bayer color filter array, and an infrared cutoff filter. There is a nascent market for image sensors for autonomous vehicle decision-system applications, and the most desirable features for such applications are not yet settled. The current offerings include sensors with larger pixels, a color filter array that comprises one quarter red filters and three quarters clear filters, and no infrared cutoff filter (e.g. ON Semiconductor; Omnivision). The requirements for optical properties, such as depth of field effects, may also differ between consumer photography and autonomous vehicles. Consumer photography values narrow depth of field images (bokeh), while autonomous driving value large depth of field to support Lens
{"title":"Optimizing Image Acquisition Systems for Autonomous Driving","authors":"H. Blasinski, J. Farrell, Trisha Lian, Zhenyi Liu, B. Wandell","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-161","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-161","url":null,"abstract":"Task requirements for image acquisition systems vary substantially between applications: requirements for consumer photography may be irrelevant or may even interfere with requirements for automotive, medical and other applications. The remarkable capabilities of the imaging industry to create lens and sensor designs for specific applications has been demonstrated in the mobile computing market. We might expect that the industry can further innovate if we specify the requirements for other markets. This paper explains an approach to developing image system designs that meet the task requirements for autonomous vehicle applications. It is impractical to build a large number of image acquisition systems and evaluate each of them with real driving data; therefore, we assembled a simulation environment to provide guidance at an early stage. The open-source and freely available software (isetcam, iset3d, and isetauto) uses ray tracing to compute quantitatively how scene radiance propagates through a multi-element lens to form the sensor irradiance. The software then transforms the irradiance into the sensor pixel responses, accounting for a large number of sensor parameters. This enables the user to apply different types of image processing pipelines to generate images that are used to train and test convolutional networks used in autonomous driving. We use the simulation environment to assess performance for different cameras and networks. Introduction The market for image sensors in autonomous vehicles can be divided into two segments. Some image sensor data is used as images to the passengers, such as rendering views from behind the car as the driver backs up. Other image sensor data is used by computational algorithms that guide the vehicle; the output from these sensors is never rendered for the human eye. It is reasonable to expect that the optical design, sensor parameters, and image processing pipeline for these two systems will differ. Mobile imaging applications for consumer photography dominate the market, driving the industry towards sensors with very small pixels (1 micron), a large number of pixels, a Bayer color filter array, and an infrared cutoff filter. There is a nascent market for image sensors for autonomous vehicle decision-system applications, and the most desirable features for such applications are not yet settled. The current offerings include sensors with larger pixels, a color filter array that comprises one quarter red filters and three quarters clear filters, and no infrared cutoff filter (e.g. ON Semiconductor; Omnivision). The requirements for optical properties, such as depth of field effects, may also differ between consumer photography and autonomous vehicles. Consumer photography values narrow depth of field images (bokeh), while autonomous driving value large depth of field to support Lens","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117172110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-311
P. Sen
Modern digital cameras have very limited dynamic range, which makes them unable to capture the full range of illumination in natural scenes. Since this prevents them from accurately photographing visible detail, researchers have spent the last two decades developing algorithms for high-dynamic range (HDR) imaging which can capture a wider range of illumination and therefore allow us to reconstruct richer images of natural scenes. The most practical of these methods are stack-based approaches which take a set of images at different exposure levels and then merge them together to form the final HDR result. However, these algorithms produce ghost-like artifacts when the scene has motion or the camera is not perfectly static. In this paper, we present an overview of state-of-the-art deghosting algorithms for stackbased HDR imaging and discuss some of the tradeoffs of each.
{"title":"Overview of State-of-the-Art Algorithms for Stack-Based High-Dynamic Range (HDR) Imaging","authors":"P. Sen","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-311","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-311","url":null,"abstract":"Modern digital cameras have very limited dynamic range, which makes them unable to capture the full range of illumination in natural scenes. Since this prevents them from accurately photographing visible detail, researchers have spent the last two decades developing algorithms for high-dynamic range (HDR) imaging which can capture a wider range of illumination and therefore allow us to reconstruct richer images of natural scenes. The most practical of these methods are stack-based approaches which take a set of images at different exposure levels and then merge them together to form the final HDR result. However, these algorithms produce ghost-like artifacts when the scene has motion or the camera is not perfectly static. In this paper, we present an overview of state-of-the-art deghosting algorithms for stackbased HDR imaging and discuss some of the tradeoffs of each.","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114509014","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-352
Jayant Thatte, B. Girod
Allowing viewers to explore virtual reality in a headmounted display with six degrees of freedom (6-DoF) greatly enhances the associated immersion and comfort. It makes the experience more compelling compared to a fixed-viewpoint 2-DoF rendering produced by conventional algorithms using data from a stationary camera rig. In this work, we use subjective testing to study the relative importance of, and the interaction between, motion parallax and binocular disparity as depth cues that shape the perception of 3D environments by human viewers. Additionally, we use the recorded head trajectories to estimate the distribution of the head movements of a sedentary viewer exploring a virtual environment with 6-DoF. Finally, we demonstrate a real-time virtual reality rendering system that uses a Stacked OmniStereo intermediary representation to provide a 6-DoF viewing experience by utilizing data from a stationary camera rig. We outline the challenges involved in developing such a system and discuss the limitations of our approach. Introduction Cinematic virtual reality is a subfield of virtual reality (VR) that deals with live-action or natural environments captured using a camera system, in contrast to computer generated scenes rendered from synthetic 3D models. With the advent of modern camera rigs, ever-faster compute capability, and a new generation of head-mounted displays (HMDs), cinematic VR is well-poised to enter the mainstream market. However, the lack of an underlying 3D scene model makes it significantly more challenging to render accurate motion parallax in natural VR scenes. As a result, all the live-action VR content available today is rendered from a fixed vantage point disregarding any positional information from the HMD. The resulting mismatch in the perceived motion between the visual and the vestibular systems gives rise to significant discomfort including nausea, headache, and disorientation [1] [2]. Additionally, motion parallax is an important depth cue [3] and rendering VR content without motion parallax also makes the experience less immersive. Furthermore, since the axis of head rotation does not pass through the eyes, head rotation even from a fixed position leads to a small translation of the eyes and therefore cannot be accurately modelled using pure rotation. The following are the key contributions of our work. 1. We present a subjective study aimed at understanding the contributions of motion parallax and binocular stereopsis to perceptual quality of experience in VR 2. We use the recorded head trajectories of the study participants to estimate the distribution of the head movements of a sedentary viewer immersed in a 6-DoF virtual environment 3. We demonstrate a real-time VR rendering system that provides a 6-DoF viewing experience The rest of the paper is organized as follows. The following section gives an overview of the related work. The next three sections detail the three contributions of our work: the results of the s
{"title":"Towards Perceptual Evaluation of Six Degrees of Freedom Virtual Reality Rendering from Stacked OmniStereo Representation","authors":"Jayant Thatte, B. Girod","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-352","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-352","url":null,"abstract":"Allowing viewers to explore virtual reality in a headmounted display with six degrees of freedom (6-DoF) greatly enhances the associated immersion and comfort. It makes the experience more compelling compared to a fixed-viewpoint 2-DoF rendering produced by conventional algorithms using data from a stationary camera rig. In this work, we use subjective testing to study the relative importance of, and the interaction between, motion parallax and binocular disparity as depth cues that shape the perception of 3D environments by human viewers. Additionally, we use the recorded head trajectories to estimate the distribution of the head movements of a sedentary viewer exploring a virtual environment with 6-DoF. Finally, we demonstrate a real-time virtual reality rendering system that uses a Stacked OmniStereo intermediary representation to provide a 6-DoF viewing experience by utilizing data from a stationary camera rig. We outline the challenges involved in developing such a system and discuss the limitations of our approach. Introduction Cinematic virtual reality is a subfield of virtual reality (VR) that deals with live-action or natural environments captured using a camera system, in contrast to computer generated scenes rendered from synthetic 3D models. With the advent of modern camera rigs, ever-faster compute capability, and a new generation of head-mounted displays (HMDs), cinematic VR is well-poised to enter the mainstream market. However, the lack of an underlying 3D scene model makes it significantly more challenging to render accurate motion parallax in natural VR scenes. As a result, all the live-action VR content available today is rendered from a fixed vantage point disregarding any positional information from the HMD. The resulting mismatch in the perceived motion between the visual and the vestibular systems gives rise to significant discomfort including nausea, headache, and disorientation [1] [2]. Additionally, motion parallax is an important depth cue [3] and rendering VR content without motion parallax also makes the experience less immersive. Furthermore, since the axis of head rotation does not pass through the eyes, head rotation even from a fixed position leads to a small translation of the eyes and therefore cannot be accurately modelled using pure rotation. The following are the key contributions of our work. 1. We present a subjective study aimed at understanding the contributions of motion parallax and binocular stereopsis to perceptual quality of experience in VR 2. We use the recorded head trajectories of the study participants to estimate the distribution of the head movements of a sedentary viewer immersed in a 6-DoF virtual environment 3. We demonstrate a real-time VR rendering system that provides a 6-DoF viewing experience The rest of the paper is organized as follows. The following section gives an overview of the related work. The next three sections detail the three contributions of our work: the results of the s","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123888070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-353
Trisha Lian, J. Farrell, B. Wandell
Camera arrays are used to acquire the 360◦ surround video data presented on 3D immersive displays. The design of these arrays involves a large number of decisions ranging from the placement and orientation of the cameras to the choice of lenses and sensors. We implemented an open-source software environment (iset360) to support engineers designing and evaluating camera arrays for virtual and augmented reality applications. The software uses physically based ray tracing to simulate a 3D virtual spectral scene and traces these rays through multi-element spherical lenses to calculate the irradiance at the imaging sensor. The software then simulates imaging sensors to predict the captured images. The sensor data can be processed to produce the stereo and monoscopic 360◦ panoramas commonly used in virtual reality applications. By simulating the entire capture pipeline, we can visualize how changes in the system components influence the system performance. We demonstrate the use of the software by simulating a variety of different camera rigs, including the Facebook Surround360, the GoPro Odyssey, the GoPro Omni, and the Samsung Gear 360. Introduction Head mounted visual displays can provide a compelling and immersive experience of a three-dimensional scene. Because the experience can be very impactful, there is a great deal of interest in developing applications ranging from clinical medicine, behavioral change, entertainment, education and experience-sharing [1] [2]. In some applications, computer graphics is used to generate content, providing a realistic, but not real, experience (e.g., video games). In other applications, the content is acquired from a real event (e.g., sports, concerts, news, or family gathering) using camera arrays (rigs) and subsequent extensive image processing that capture and render the environment (Figure 1). The design of these rigs involves many different engineering decisions, including the selection of lenses, sensors, and camera positions. In addition to the rig, there are many choices of how to store and process the acquired content. For example, data from multiple cameras are often transformed into a stereo pair of 360◦ panoramas [3] by stitching together images captured by multiple cameras. Based on the user’s head position and orientation, data are extracted from the panorama and rendered on a head mounted display. There is no single quality-limiting element of this system, and moreover, interactions between the hardware and software design choices limit how well metrics of individual components predict overall system quality. To create a good experience, we must be able to assess the combination of hardware and software components that comprise the entire system. Building and testing a complete rig is costly and slow; hence, it can be useful to obtain guidance about design choices by using Figure 1. Overview of the hardware and software components that combine in an camera rig for an immersive head-mounted display
{"title":"Image Systems Simulation for 360° Camera Rigs","authors":"Trisha Lian, J. Farrell, B. Wandell","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-353","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-353","url":null,"abstract":"Camera arrays are used to acquire the 360◦ surround video data presented on 3D immersive displays. The design of these arrays involves a large number of decisions ranging from the placement and orientation of the cameras to the choice of lenses and sensors. We implemented an open-source software environment (iset360) to support engineers designing and evaluating camera arrays for virtual and augmented reality applications. The software uses physically based ray tracing to simulate a 3D virtual spectral scene and traces these rays through multi-element spherical lenses to calculate the irradiance at the imaging sensor. The software then simulates imaging sensors to predict the captured images. The sensor data can be processed to produce the stereo and monoscopic 360◦ panoramas commonly used in virtual reality applications. By simulating the entire capture pipeline, we can visualize how changes in the system components influence the system performance. We demonstrate the use of the software by simulating a variety of different camera rigs, including the Facebook Surround360, the GoPro Odyssey, the GoPro Omni, and the Samsung Gear 360. Introduction Head mounted visual displays can provide a compelling and immersive experience of a three-dimensional scene. Because the experience can be very impactful, there is a great deal of interest in developing applications ranging from clinical medicine, behavioral change, entertainment, education and experience-sharing [1] [2]. In some applications, computer graphics is used to generate content, providing a realistic, but not real, experience (e.g., video games). In other applications, the content is acquired from a real event (e.g., sports, concerts, news, or family gathering) using camera arrays (rigs) and subsequent extensive image processing that capture and render the environment (Figure 1). The design of these rigs involves many different engineering decisions, including the selection of lenses, sensors, and camera positions. In addition to the rig, there are many choices of how to store and process the acquired content. For example, data from multiple cameras are often transformed into a stereo pair of 360◦ panoramas [3] by stitching together images captured by multiple cameras. Based on the user’s head position and orientation, data are extracted from the panorama and rendered on a head mounted display. There is no single quality-limiting element of this system, and moreover, interactions between the hardware and software design choices limit how well metrics of individual components predict overall system quality. To create a good experience, we must be able to assess the combination of hardware and software components that comprise the entire system. Building and testing a complete rig is costly and slow; hence, it can be useful to obtain guidance about design choices by using Figure 1. Overview of the hardware and software components that combine in an camera rig for an immersive head-mounted display","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114559157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-28DOI: 10.2352/ISSN.2470-1173.2018.05.PMII-409
H. Dietz, P. Eberhart, C. Demaree
{"title":"Multispectral, high dynamic range, time domain continuous imaging","authors":"H. Dietz, P. Eberhart, C. Demaree","doi":"10.2352/ISSN.2470-1173.2018.05.PMII-409","DOIUrl":"https://doi.org/10.2352/ISSN.2470-1173.2018.05.PMII-409","url":null,"abstract":"","PeriodicalId":309050,"journal":{"name":"Photography, Mobile, and Immersive Imaging","volume":"100 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124057278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：