Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.4.mwsf-376
Xiyang Luo, Michael Goebel, Elnaz Barshan, Feng Yang
In this work, we present an efficient multi-bit deep image watermarking method that is cover-agnostic yet also robust to geometric distortions such as translation and scaling as well as other distortions such as JPEG compression and noise. Our design consists of a light-weight watermark encoder jointly trained with a deep neural network based decoder. Such a design allows us to retain the efficiency of the encoder while fully utilizing the power of a deep neural network. Moreover, the watermark encoder is independent of the image content, allowing users to pre-generate the watermarks for further efficiency. To offer robustness towards geometric transformations, we introduced a learned model for predicting the scale and offset of the watermarked images. Moreover, our watermark encoder is independent of the image content, making the generated watermarks universally applicable to different cover images. Experiments show that our method outperforms comparably efficient watermarking methods by a large margin.
{"title":"LECA: A learned approach for efficient cover-agnostic watermarking","authors":"Xiyang Luo, Michael Goebel, Elnaz Barshan, Feng Yang","doi":"10.2352/ei.2023.35.4.mwsf-376","DOIUrl":"https://doi.org/10.2352/ei.2023.35.4.mwsf-376","url":null,"abstract":"In this work, we present an efficient multi-bit deep image watermarking method that is cover-agnostic yet also robust to geometric distortions such as translation and scaling as well as other distortions such as JPEG compression and noise. Our design consists of a light-weight watermark encoder jointly trained with a deep neural network based decoder. Such a design allows us to retain the efficiency of the encoder while fully utilizing the power of a deep neural network. Moreover, the watermark encoder is independent of the image content, allowing users to pre-generate the watermarks for further efficiency. To offer robustness towards geometric transformations, we introduced a learned model for predicting the scale and offset of the watermarked images. Moreover, our watermark encoder is independent of the image content, making the generated watermarks universally applicable to different cover images. Experiments show that our method outperforms comparably efficient watermarking methods by a large margin.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135694925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.8.iqsp-314
Shubham Ravindra Alai, Radhesh Bhat
An image signal processor (ISP) transforms a sensor's raw image into a RGB image for use in computer or human vision applications. ISP is composed of various functional blocks and each block contributes uniquely to make the image best suitable for the target application. Whereas, each block consists of several hyperparameters and each hyperparameter needs to be tuned (usually done manually by experts in an iterative manner) to achieve the target image quality. The tuning becomes challenging and increasingly iterative especially in low to very low light conditions where the amount of details preserved by the sensor is limited and ISP parameters have to be tuned to balance the amount of details recovered, noise, sharpness, contrast etc. To extract maximum information out of the image, usually it is required to increase the ISO gain which eventually impacts the noise and color accuracy. Also, the number of ISP parameters that need to be tuned are huge and it becomes impractical to consider all of them in such low light conditions to arrive at the best possible settings. To tackle challenges in manual tuning, especially for low light conditions we have implemented an automatic hyperparameter optimization model that can tune the low lux images so that they are perceptually equivalent to high-lux images. The experiments for IQ validation are carried out under challenging low light conditions and scenarios using Qualcomm’s Spectra ISP simulator with a 13MP OV sensor, and the performance of automatic tuned IQ is compared with manual tuned IQ for human vision use-cases. With experimental results, we have proved that with the help of evolutionary algorithms and local optimization it is possible to optimize the ISP parameters such that without using any of the KPI metrics still low-lux image/ image captured with different ISP (test image) can perceptually be improved that are equivalent to high-lux or well-tuned (reference) image.
{"title":"Optimization of ISP parameters for low light conditions using a non-linear reference based approach","authors":"Shubham Ravindra Alai, Radhesh Bhat","doi":"10.2352/ei.2023.35.8.iqsp-314","DOIUrl":"https://doi.org/10.2352/ei.2023.35.8.iqsp-314","url":null,"abstract":"An image signal processor (ISP) transforms a sensor's raw image into a RGB image for use in computer or human vision applications. ISP is composed of various functional blocks and each block contributes uniquely to make the image best suitable for the target application. Whereas, each block consists of several hyperparameters and each hyperparameter needs to be tuned (usually done manually by experts in an iterative manner) to achieve the target image quality. The tuning becomes challenging and increasingly iterative especially in low to very low light conditions where the amount of details preserved by the sensor is limited and ISP parameters have to be tuned to balance the amount of details recovered, noise, sharpness, contrast etc. To extract maximum information out of the image, usually it is required to increase the ISO gain which eventually impacts the noise and color accuracy. Also, the number of ISP parameters that need to be tuned are huge and it becomes impractical to consider all of them in such low light conditions to arrive at the best possible settings. To tackle challenges in manual tuning, especially for low light conditions we have implemented an automatic hyperparameter optimization model that can tune the low lux images so that they are perceptually equivalent to high-lux images. The experiments for IQ validation are carried out under challenging low light conditions and scenarios using Qualcomm’s Spectra ISP simulator with a 13MP OV sensor, and the performance of automatic tuned IQ is compared with manual tuned IQ for human vision use-cases. With experimental results, we have proved that with the help of evolutionary algorithms and local optimization it is possible to optimize the ISP parameters such that without using any of the KPI metrics still low-lux image/ image captured with different ISP (test image) can perceptually be improved that are equivalent to high-lux or well-tuned (reference) image.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135695030","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.11.hpci-a11
Abstract In recent years, the rapid development of imaging systems and the growth of compute-intensive imaging algorithms have led to a strong demand for High Performance Computing (HPC) for efficient image processing. However, the two communities, imaging and HPC, have largely remained separate, with little synergy. This conference focuses on research topics that converge HPC and imaging research with an emphasis on advanced HPC facilities and techniques for imaging systems/algorithms and applications. In addition, the conference provides a unique platform that brings imaging and HPC people together and discusses emerging research topics and techniques that benefit both the HPC and imaging community. Papers are solicited on all aspects of research, development, and application of high-performance computing or efficient computing algorithms and systems for imaging applications.
{"title":"High Performance Computing for Imaging 2023 Overview and Papers Program","authors":"","doi":"10.2352/ei.2023.35.11.hpci-a11","DOIUrl":"https://doi.org/10.2352/ei.2023.35.11.hpci-a11","url":null,"abstract":"Abstract In recent years, the rapid development of imaging systems and the growth of compute-intensive imaging algorithms have led to a strong demand for High Performance Computing (HPC) for efficient image processing. However, the two communities, imaging and HPC, have largely remained separate, with little synergy. This conference focuses on research topics that converge HPC and imaging research with an emphasis on advanced HPC facilities and techniques for imaging systems/algorithms and applications. In addition, the conference provides a unique platform that brings imaging and HPC people together and discusses emerging research topics and techniques that benefit both the HPC and imaging community. Papers are solicited on all aspects of research, development, and application of high-performance computing or efficient computing algorithms and systems for imaging applications.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135695206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.7.image-a07
Abstract Recent progress at the intersection of deep learning and imaging has created a new wave of interest in imaging and multimedia analytics topics, from social media sharing to augmented reality, from food and nutrition to health surveillance, from remote sensing and agriculture to wildlife and environment monitoring. Compared to many subjects in traditional imaging, these topics are more multi-disciplinary in nature. This conference will provide a forum for researchers and engineers from various related areas, both academic and industrial, to exchange ideas and share research results in this rapidly evolving field.
{"title":"Imaging and Multimedia Analytics at the Edge 2023 Conference Overview and Papers Program","authors":"","doi":"10.2352/ei.2023.35.7.image-a07","DOIUrl":"https://doi.org/10.2352/ei.2023.35.7.image-a07","url":null,"abstract":"Abstract Recent progress at the intersection of deep learning and imaging has created a new wave of interest in imaging and multimedia analytics topics, from social media sharing to augmented reality, from food and nutrition to health surveillance, from remote sensing and agriculture to wildlife and environment monitoring. Compared to many subjects in traditional imaging, these topics are more multi-disciplinary in nature. This conference will provide a forum for researchers and engineers from various related areas, both academic and industrial, to exchange ideas and share research results in this rapidly evolving field.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135695211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.6.iss-a06
Abstract Solid state optical sensors and solid state cameras have established themselves as the imaging systems of choice for many demanding professional applications such as automotive, space, medical, scientific and industrial applications. The advantages of low-power, low-noise, high-resolution, high-geometric fidelity, broad spectral sensitivity, and extremely high quantum efficiency have led to a number of revolutionary uses. ISS focuses on image sensing for consumer, industrial, medical, and scientific applications, as well as embedded image processing, and pipeline tuning for these camera systems. This conference will serve to bring together researchers, scientists, and engineers working in these fields, and provides the opportunity for quick publication of their work. Topics can include, but are not limited to, research and applications in image sensors and detectors, camera/sensor characterization, ISP pipelines and tuning, image artifact correction and removal, image reconstruction, color calibration, image enhancement, HDR imaging, light-field imaging, multi-frame processing, computational photography, 3D imaging, 360/cinematic VR cameras, camera image quality evaluation and metrics, novel imaging applications, imaging system design, and deep learning applications in imaging.
{"title":"Imaging Sensors and Systems 2023 Conference Overview and Papers Program","authors":"","doi":"10.2352/ei.2023.35.6.iss-a06","DOIUrl":"https://doi.org/10.2352/ei.2023.35.6.iss-a06","url":null,"abstract":"Abstract Solid state optical sensors and solid state cameras have established themselves as the imaging systems of choice for many demanding professional applications such as automotive, space, medical, scientific and industrial applications. The advantages of low-power, low-noise, high-resolution, high-geometric fidelity, broad spectral sensitivity, and extremely high quantum efficiency have led to a number of revolutionary uses. ISS focuses on image sensing for consumer, industrial, medical, and scientific applications, as well as embedded image processing, and pipeline tuning for these camera systems. This conference will serve to bring together researchers, scientists, and engineers working in these fields, and provides the opportunity for quick publication of their work. Topics can include, but are not limited to, research and applications in image sensors and detectors, camera/sensor characterization, ISP pipelines and tuning, image artifact correction and removal, image reconstruction, color calibration, image enhancement, HDR imaging, light-field imaging, multi-frame processing, computational photography, 3D imaging, 360/cinematic VR cameras, camera image quality evaluation and metrics, novel imaging applications, imaging system design, and deep learning applications in imaging.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135695212","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.3.mobmu-355
Artem Sklyar, Klaus Schwarz, Reiner Creutzburg
Open-source intelligence is gaining popularity due to the rapid development of social networks. There is more and more information in the public domain. One of the most popular social networks is Twitter. It was chosen to analyze the dependence of changes in the number of likes, reposts, quotes and retweets on the aggressiveness of the post text for a separate profile, as this information can be important not only for the owner of the channel in the social network, but also for other studies that in some way influence user accounts and their behavior in the social network. Furthermore, this work includes a detailed analysis and evaluation of the Tweety library capabilities and situations in which it can be effectively applied. Lastly, this work includes the creation and description of a compiled neural network whose purpose is to predict changes in the number of likes, reposts, quotes, and retweets from the aggressiveness of the post text for a separate profile.
{"title":"Practical OSINT investigation in Twitter utilizing AI-based aggressiveness analysis","authors":"Artem Sklyar, Klaus Schwarz, Reiner Creutzburg","doi":"10.2352/ei.2023.35.3.mobmu-355","DOIUrl":"https://doi.org/10.2352/ei.2023.35.3.mobmu-355","url":null,"abstract":"Open-source intelligence is gaining popularity due to the rapid development of social networks. There is more and more information in the public domain. One of the most popular social networks is Twitter. It was chosen to analyze the dependence of changes in the number of likes, reposts, quotes and retweets on the aggressiveness of the post text for a separate profile, as this information can be important not only for the owner of the channel in the social network, but also for other studies that in some way influence user accounts and their behavior in the social network. Furthermore, this work includes a detailed analysis and evaluation of the Tweety library capabilities and situations in which it can be effectively applied. Lastly, this work includes the creation and description of a compiled neural network whose purpose is to predict changes in the number of likes, reposts, quotes, and retweets from the aggressiveness of the post text for a separate profile.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135695031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.17.3dia-a17
Abstract Scientific and technological advances during the last decade in the fields of image acquisition, data processing, telecommunications, and computer graphics have contributed to the emergence of new multimedia, especially 3D digital data. Modern 3D imaging technologies allow for the acquisition of 3D and 4D (3D video) data at higher speeds, resolutions, and accuracies. With the ability to capture increasingly complex 3D/4D information, advancements have also been made in the areas of 3D data processing (e.g., filtering, reconstruction, compression). As such, 3D/4D technologies are now being used in a large variety of applications, such as medicine, forensic science, cultural heritage, manufacturing, autonomous vehicles, security, and bioinformatics. Further, with mixed reality (AR, VR, XR), 3D/4D technologies may also change the ways we work, play, and communicate with each other every day.
{"title":"3D Imaging and Applications 2023 Conference Overview and Papers Program","authors":"","doi":"10.2352/ei.2023.35.17.3dia-a17","DOIUrl":"https://doi.org/10.2352/ei.2023.35.17.3dia-a17","url":null,"abstract":"Abstract Scientific and technological advances during the last decade in the fields of image acquisition, data processing, telecommunications, and computer graphics have contributed to the emergence of new multimedia, especially 3D digital data. Modern 3D imaging technologies allow for the acquisition of 3D and 4D (3D video) data at higher speeds, resolutions, and accuracies. With the ability to capture increasingly complex 3D/4D information, advancements have also been made in the areas of 3D data processing (e.g., filtering, reconstruction, compression). As such, 3D/4D technologies are now being used in a large variety of applications, such as medicine, forensic science, cultural heritage, manufacturing, autonomous vehicles, security, and bioinformatics. Further, with mixed reality (AR, VR, XR), 3D/4D technologies may also change the ways we work, play, and communicate with each other every day.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135695213","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.13.cvaa-a13
Abstract This conference on computer image analysis in the study of art presents leading research in the application of image analysis, computer vision, and pattern recognition to problems of interest to art historians, curators and conservators. A number of recent questions and controversies have highlighted the value of rigorous image analysis in the service of the analysis of art, particularly painting. Consider these examples: the fractal image analysis for the authentication of drip paintings possibly by Jackson Pollock; sophisticated perspective, shading and form analysis to address claims that early Renaissance masters such as Jan van Eyck or Baroque masters such as Georges de la Tour traced optically projected images; automatic multi-scale analysis of brushstrokes for the attribution of portraits within a painting by Perugino; and multi-spectral, x-ray and infra-red scanning and image analysis of the Mona Lisa to reveal the painting techniques of Leonardo. The value of image analysis to these and other questions strongly suggests that current and future computer methods will play an ever larger role in the scholarship of visual arts.
{"title":"Computer Vision and Image Analysis of Art 2023 Conference Overview and Papers Program","authors":"","doi":"10.2352/ei.2023.35.13.cvaa-a13","DOIUrl":"https://doi.org/10.2352/ei.2023.35.13.cvaa-a13","url":null,"abstract":"Abstract This conference on computer image analysis in the study of art presents leading research in the application of image analysis, computer vision, and pattern recognition to problems of interest to art historians, curators and conservators. A number of recent questions and controversies have highlighted the value of rigorous image analysis in the service of the analysis of art, particularly painting. Consider these examples: the fractal image analysis for the authentication of drip paintings possibly by Jackson Pollock; sophisticated perspective, shading and form analysis to address claims that early Renaissance masters such as Jan van Eyck or Baroque masters such as Georges de la Tour traced optically projected images; automatic multi-scale analysis of brushstrokes for the attribution of portraits within a painting by Perugino; and multi-spectral, x-ray and infra-red scanning and image analysis of the Mona Lisa to reveal the painting techniques of Leonardo. The value of image analysis to these and other questions strongly suggests that current and future computer methods will play an ever larger role in the scholarship of visual arts.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135695216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The design and evaluation of complex systems can benefit from a software simulation - sometimes called a digital twin. The simulation can be used to characterize system performance or to test its performance under conditions that are difficult to measure (e.g., nighttime for automotive perception systems). We describe the image system simulation software tools that we use to evaluate the performance of image systems for object (automobile) detection. We describe experiments with 13 different cameras with a variety of optics and pixel sizes. To measure the impact of camera spatial resolution, we designed a collection of driving scenes that had cars at many different distances. We quantified system performance by measuring average precision and we report a trend relating system resolution and object detection performance. We also quantified the large performance degradation under nighttime conditions, compared to daytime, for all cameras and a COCO pre-trained network.
{"title":"Using simulation to quantify the performance of automotive perception systems","authors":"Zhenyi Liu, Devesh Shah, Alireza Rahimpour, Devesh Upadhyay, Joyce Farrell, Brian Wandell","doi":"10.2352/ei.2023.35.16.avm-118","DOIUrl":"https://doi.org/10.2352/ei.2023.35.16.avm-118","url":null,"abstract":"The design and evaluation of complex systems can benefit from a software simulation - sometimes called a digital twin. The simulation can be used to characterize system performance or to test its performance under conditions that are difficult to measure (e.g., nighttime for automotive perception systems). We describe the image system simulation software tools that we use to evaluate the performance of image systems for object (automobile) detection. We describe experiments with 13 different cameras with a variety of optics and pixel sizes. To measure the impact of camera spatial resolution, we designed a collection of driving scenes that had cars at many different distances. We quantified system performance by measuring average precision and we report a trend relating system resolution and object detection performance. We also quantified the large performance degradation under nighttime conditions, compared to daytime, for all cameras and a COCO pre-trained network.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135693975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-16DOI: 10.2352/ei.2023.35.14.coimg-153
Zhong Zhuang, David Yang, Felix Hofmann, David Barmherzig, Ju Sun
Phase retrieval (PR) consists of recovering complex-valued objects from their oversampled Fourier magnitudes and takes a central place in scientific imaging. A critical issue around PR is the typical nonconvexity in natural formulations and the associated bad local minimizers. The issue is exacerbated when the support of the object is not precisely known and hence must be overspecified in practice. Practical methods for PR hence involve convolved algorithms, e.g., multiple cycles of hybrid input-output (HIO) + error reduction (ER), to avoid the bad local minimizers and attain reasonable speed, and heuristics to refine the support of the object, e.g., the famous shrinkwrap trick. Overall, the convolved algorithms and the support-refinement heuristics induce multiple algorithm hyperparameters, to which the recovery quality is often sensitive. In this work, we propose a novel PR method by parameterizing the object as the output of a learnable neural network, i.e., deep image prior (DIP). For complex-valued objects in PR, we can flexibly parametrize the magnitude and phase, or the real and imaginary parts separately by two DIPs. We show that this simple idea, free from multi-hyperparameter tuning and support-refinement heuristics, can obtain superior performance than gold-standard PR methods. For the session: Computational Imaging using Fourier Ptychography and Phase Retrieval.
{"title":"Practical phase retrieval using double deep image priors","authors":"Zhong Zhuang, David Yang, Felix Hofmann, David Barmherzig, Ju Sun","doi":"10.2352/ei.2023.35.14.coimg-153","DOIUrl":"https://doi.org/10.2352/ei.2023.35.14.coimg-153","url":null,"abstract":"Phase retrieval (PR) consists of recovering complex-valued objects from their oversampled Fourier magnitudes and takes a central place in scientific imaging. A critical issue around PR is the typical nonconvexity in natural formulations and the associated bad local minimizers. The issue is exacerbated when the support of the object is not precisely known and hence must be overspecified in practice. Practical methods for PR hence involve convolved algorithms, e.g., multiple cycles of hybrid input-output (HIO) + error reduction (ER), to avoid the bad local minimizers and attain reasonable speed, and heuristics to refine the support of the object, e.g., the famous shrinkwrap trick. Overall, the convolved algorithms and the support-refinement heuristics induce multiple algorithm hyperparameters, to which the recovery quality is often sensitive. In this work, we propose a novel PR method by parameterizing the object as the output of a learnable neural network, i.e., deep image prior (DIP). For complex-valued objects in PR, we can flexibly parametrize the magnitude and phase, or the real and imaginary parts separately by two DIPs. We show that this simple idea, free from multi-hyperparameter tuning and support-refinement heuristics, can obtain superior performance than gold-standard PR methods. For the session: Computational Imaging using Fourier Ptychography and Phase Retrieval.","PeriodicalId":73514,"journal":{"name":"IS&T International Symposium on Electronic Imaging","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135694169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：