首页 > 最新文献

2020 IEEE International Conference on Computational Photography (ICCP)最新文献

英文 中文
High Resolution Light Field Recovery with Fourier Disparity Layer Completion, Demosaicing, and Super-Resolution 高分辨率光场恢复与傅里叶视差层补全,去马赛克,和超分辨率
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105172
Mikael Le Pendu, A. Smolic
In this paper, we present a novel approach for recovering high resolution light fields from input data with many types of degradation and challenges typically found in lenslet based plenoptic cameras. Those include the low spatial resolution, but also the irregular spatio-angular sampling and color sampling, the depth-dependent blur, and even axial chromatic aberrations. Our approach, based on the recent Fourier Disparity Layer representation of the light field, allows the construction of high resolution layers directly from the low resolution input views. High resolution light field views are then simply reconstructed by shifting and summing the layers. We show that when the spatial sampling is regular, the layer construction can be decomposed into linear optimization problems formulated in the Fourier domain for small groups of frequency components. We additionally propose a new preconditioning approach ensuring spatial consistency, and a color regularization term to simultaneously perform color demosaicing. For the general case of light field completion from an irregular sampling, we define a simple iterative version of the algorithm. Both approaches are then combined for an efficient super-resolution of the irregularly sampled data of plenoptic cameras. Finally, the Fourier Disparity Layer model naturally extends to take into account a depth-dependent blur and axial chromatic aberrations without requiring an estimation of depth or disparity maps.
在本文中,我们提出了一种从输入数据中恢复高分辨率光场的新方法,这些数据具有许多类型的退化和挑战,通常在基于透镜的全光学相机中发现。这些问题包括低空间分辨率,但也有不规则的空间角采样和颜色采样,深度相关的模糊,甚至轴向色差。我们的方法,基于最近的傅里叶视差层表示的光场,允许直接从低分辨率输入视图构建高分辨率层。高分辨率的光场视图,然后简单地通过移动和叠加层重建。我们表明,当空间采样是规则的,层的结构可以分解成线性优化问题,在傅里叶域为小群的频率成分。我们还提出了一种新的预处理方法来确保空间一致性,并提出了一个颜色正则化项来同时执行颜色去马赛克。对于不规则采样的光场补全的一般情况,我们定义了该算法的一个简单迭代版本。然后将这两种方法结合起来,对全光学相机的不规则采样数据进行有效的超分辨率处理。最后,傅里叶视差层模型自然扩展到考虑到深度相关的模糊和轴向色差,而不需要估计深度或视差图。
{"title":"High Resolution Light Field Recovery with Fourier Disparity Layer Completion, Demosaicing, and Super-Resolution","authors":"Mikael Le Pendu, A. Smolic","doi":"10.1109/ICCP48838.2020.9105172","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105172","url":null,"abstract":"In this paper, we present a novel approach for recovering high resolution light fields from input data with many types of degradation and challenges typically found in lenslet based plenoptic cameras. Those include the low spatial resolution, but also the irregular spatio-angular sampling and color sampling, the depth-dependent blur, and even axial chromatic aberrations. Our approach, based on the recent Fourier Disparity Layer representation of the light field, allows the construction of high resolution layers directly from the low resolution input views. High resolution light field views are then simply reconstructed by shifting and summing the layers. We show that when the spatial sampling is regular, the layer construction can be decomposed into linear optimization problems formulated in the Fourier domain for small groups of frequency components. We additionally propose a new preconditioning approach ensuring spatial consistency, and a color regularization term to simultaneously perform color demosaicing. For the general case of light field completion from an irregular sampling, we define a simple iterative version of the algorithm. Both approaches are then combined for an efficient super-resolution of the irregularly sampled data of plenoptic cameras. Finally, the Fourier Disparity Layer model naturally extends to take into account a depth-dependent blur and axial chromatic aberrations without requiring an estimation of depth or disparity maps.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129307503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 9
WISHED: Wavefront imaging sensor with high resolution and depth ranging wish:高分辨率深度测距波前成像传感器
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105280
Yicheng Wu, Fengqiang Li, F. Willomitzer, A. Veeraraghavan, O. Cossairt
Phase-retrieval based wavefront sensors have been shown to reconstruct the complex field from an object with a high spatial resolution. Although the reconstructed complex field encodes the depth information of the object, it is impractical to be used as a depth sensor for macroscopic objects, since the unambiguous depth imaging range is limited by the optical wavelength. To improve the depth range of imaging and handle depth discontinuities, we propose a novel three-dimensional sensor by leveraging wavelength diversity and wavefront sensing. Complex fields at two optical wavelengths are recorded, and a synthetic wavelength can be generated by correlating those wavefronts. The proposed system achieves high lateral and depth resolutions. Our experimental prototype shows an unambiguous range of more than 1,000 x larger compared with the optical wavelengths, while the depth precision is up to 9µm for smooth objects and up to 69µm for rough objects. We experimentally demonstrate 3D reconstructions for transparent, translucent, and opaque objects with smooth and rough surfaces.
基于相位恢复的波前传感器已被证明能够以高空间分辨率重建物体的复杂场。虽然重建的复场编码了物体的深度信息,但由于其清晰的深度成像范围受光波长的限制,无法用于宏观物体的深度传感器。为了提高成像深度范围和处理深度不连续,我们提出了一种利用波长分集和波前传感的新型三维传感器。记录两个光学波长的复杂场,并通过将这些波前关联产生合成波长。该系统具有较高的横向分辨率和深度分辨率。我们的实验原型显示,与光学波长相比,其精确范围超过1000倍,而对于光滑物体的深度精度高达9 μ m,对于粗糙物体的深度精度高达69 μ m。我们通过实验演示了具有光滑和粗糙表面的透明,半透明和不透明物体的3D重建。
{"title":"WISHED: Wavefront imaging sensor with high resolution and depth ranging","authors":"Yicheng Wu, Fengqiang Li, F. Willomitzer, A. Veeraraghavan, O. Cossairt","doi":"10.1109/ICCP48838.2020.9105280","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105280","url":null,"abstract":"Phase-retrieval based wavefront sensors have been shown to reconstruct the complex field from an object with a high spatial resolution. Although the reconstructed complex field encodes the depth information of the object, it is impractical to be used as a depth sensor for macroscopic objects, since the unambiguous depth imaging range is limited by the optical wavelength. To improve the depth range of imaging and handle depth discontinuities, we propose a novel three-dimensional sensor by leveraging wavelength diversity and wavefront sensing. Complex fields at two optical wavelengths are recorded, and a synthetic wavelength can be generated by correlating those wavefronts. The proposed system achieves high lateral and depth resolutions. Our experimental prototype shows an unambiguous range of more than 1,000 x larger compared with the optical wavelengths, while the depth precision is up to 9µm for smooth objects and up to 69µm for rough objects. We experimentally demonstrate 3D reconstructions for transparent, translucent, and opaque objects with smooth and rough surfaces.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132310772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Simulating Anisoplanatic Turbulence by Sampling Correlated Zernike Coefficients 用采样相关泽尼克系数模拟各向异性湍流
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105270
Nicholas Chimitt, Stanley H. Chan
Simulating atmospheric turbulence is an essential task for evaluating turbulence mitigation algorithms and training learning-based methods. Advanced numerical simulators for atmospheric turbulence are available, but they require sophisticated wave propagations which are computationally very expensive. In this paper, we present a propagation-free method for simulating imaging through anisoplanatic atmospheric turbulence. The key innovation that enables this work is a new method to draw spatially correlated tilts and high-order abberations in the Zernike space. By establishing the equivalence between the angle-of-arrival correlation by Basu, McCrae and Fiorino (2015) and the multi-aperture correlation by Chanan (1992), we show that the Zernike coefficients can be drawn according to a covariance matrix defining the spatial correlations. We propose fast and scalable sampling strategies to draw these samples. The new method allows us to compress the wave propagation problem into a sampling problem, hence making the new simulator significantly faster than existing ones. Experimental results show that the simulator has an excellent match with the theory and real turbulence data.
模拟大气湍流是评估湍流缓解算法和训练基于学习的方法的重要任务。先进的大气湍流数值模拟器是可用的,但它们需要复杂的波传播,这在计算上非常昂贵。在本文中,我们提出了一种无传播的方法来模拟通过各向异性大气湍流成像。实现这项工作的关键创新是在Zernike空间中绘制空间相关倾斜和高阶像差的新方法。通过建立Basu、McCrae和Fiorino(2015)的到达角相关性与Chanan(1992)的多孔径相关性之间的等价性,我们发现Zernike系数可以根据定义空间相关性的协方差矩阵来绘制。我们提出了快速和可扩展的采样策略来绘制这些样本。新方法允许我们将波传播问题压缩为采样问题,从而使新的模拟器比现有的模拟器快得多。实验结果表明,该模拟器与理论和实际湍流数据吻合良好。
{"title":"Simulating Anisoplanatic Turbulence by Sampling Correlated Zernike Coefficients","authors":"Nicholas Chimitt, Stanley H. Chan","doi":"10.1109/ICCP48838.2020.9105270","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105270","url":null,"abstract":"Simulating atmospheric turbulence is an essential task for evaluating turbulence mitigation algorithms and training learning-based methods. Advanced numerical simulators for atmospheric turbulence are available, but they require sophisticated wave propagations which are computationally very expensive. In this paper, we present a propagation-free method for simulating imaging through anisoplanatic atmospheric turbulence. The key innovation that enables this work is a new method to draw spatially correlated tilts and high-order abberations in the Zernike space. By establishing the equivalence between the angle-of-arrival correlation by Basu, McCrae and Fiorino (2015) and the multi-aperture correlation by Chanan (1992), we show that the Zernike coefficients can be drawn according to a covariance matrix defining the spatial correlations. We propose fast and scalable sampling strategies to draw these samples. The new method allows us to compress the wave propagation problem into a sampling problem, hence making the new simulator significantly faster than existing ones. Experimental results show that the simulator has an excellent match with the theory and real turbulence data.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131080163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 9
Deep Adaptive LiDAR: End-to-end Optimization of Sampling and Depth Completion at Low Sampling Rates 深度自适应激光雷达:低采样率下采样和深度完成的端到端优化
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105252
Alexander W. Bergman, David B. Lindell, Gordon Wetzstein
Current LiDAR systems are limited in their ability to capture dense 3D point clouds. To overcome this challenge, deep learning-based depth completion algorithms have been developed to inpaint missing depth guided by an RGB image. However, these methods fail for low sampling rates. Here, we propose an adaptive sampling scheme for LiDAR systems that demonstrates state-of-the-art performance for depth completion at low sampling rates. Our system is fully differentiable, allowing the sparse depth sampling and the depth inpainting components to be trained end-to-end with an upstream task.
目前的激光雷达系统在捕捉密集的3D点云方面能力有限。为了克服这一挑战,基于深度学习的深度补全算法已经被开发出来,以RGB图像为指导来补绘缺失的深度。然而,这些方法在低采样率下失败。在这里,我们提出了一种用于激光雷达系统的自适应采样方案,该方案在低采样率下展示了最先进的深度完井性能。我们的系统是完全可微的,允许稀疏深度采样和深度绘制组件通过上游任务进行端到端训练。
{"title":"Deep Adaptive LiDAR: End-to-end Optimization of Sampling and Depth Completion at Low Sampling Rates","authors":"Alexander W. Bergman, David B. Lindell, Gordon Wetzstein","doi":"10.1109/ICCP48838.2020.9105252","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105252","url":null,"abstract":"Current LiDAR systems are limited in their ability to capture dense 3D point clouds. To overcome this challenge, deep learning-based depth completion algorithms have been developed to inpaint missing depth guided by an RGB image. However, these methods fail for low sampling rates. Here, we propose an adaptive sampling scheme for LiDAR systems that demonstrates state-of-the-art performance for depth completion at low sampling rates. Our system is fully differentiable, allowing the sparse depth sampling and the depth inpainting components to be trained end-to-end with an upstream task.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125388358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 32
Comparing Vision-based to Sonar-based 3D Reconstruction 比较基于视觉和基于声纳的3D重建
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105273
Netanel Frank, Lior Wolf, D. Olshansky, A. Boonman, Y. Yovel
Our understanding of sonar based sensing is very limited in comparison to light based imaging. In this work, we synthesize a ShapeNet variant in which echolocation replaces the role of vision. A new hypernetwork method is presented for 3D reconstruction from a single echolocation view. The success of the method demonstrates the ability to reconstruct a 3D shape from bat-like sonar, and not just obtain the relative position of the bat with respect to obstacles. In addition, it is shown that integrating information from multiple orientations around the same view point helps performance. The sonar-based method we develop is analog to the state-of-the-art single image reconstruction method, which allows us to directly compare the two imaging modalities. Based on this analysis, we learn that while 3D can be reliably reconstructed form sonar, as far as the current technology shows, the accuracy is lower than the one obtained based on vision, that the performance in sonar and in vision are highly correlated, that both modalities favor shapes that are not round, and that while the current vision method is able to better reconstruct the 3D shape, its advantage with respect to estimating the normal's direction is much lower.
与基于光的成像相比,我们对基于声纳的传感的理解非常有限。在这项工作中,我们合成了一个ShapeNet变体,其中回声定位取代了视觉的作用。提出了一种基于单一回波定位视图的超网络三维重建方法。该方法的成功证明了利用类似蝙蝠的声纳重建三维形状的能力,而不仅仅是获得蝙蝠相对于障碍物的相对位置。此外,在同一视点周围集成来自多个方向的信息有助于提高性能。我们开发的基于声纳的方法类似于最先进的单图像重建方法,这使我们能够直接比较两种成像方式。基于此分析,我们了解到,虽然可以可靠地从声纳中重建3D,但就目前的技术而言,精度低于基于视觉的3D,声纳和视觉的性能高度相关,两种模式都倾向于非圆形的形状,并且虽然目前的视觉方法能够更好地重建3D形状,但其在估计法线方向方面的优势要低得多。
{"title":"Comparing Vision-based to Sonar-based 3D Reconstruction","authors":"Netanel Frank, Lior Wolf, D. Olshansky, A. Boonman, Y. Yovel","doi":"10.1109/ICCP48838.2020.9105273","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105273","url":null,"abstract":"Our understanding of sonar based sensing is very limited in comparison to light based imaging. In this work, we synthesize a ShapeNet variant in which echolocation replaces the role of vision. A new hypernetwork method is presented for 3D reconstruction from a single echolocation view. The success of the method demonstrates the ability to reconstruct a 3D shape from bat-like sonar, and not just obtain the relative position of the bat with respect to obstacles. In addition, it is shown that integrating information from multiple orientations around the same view point helps performance. The sonar-based method we develop is analog to the state-of-the-art single image reconstruction method, which allows us to directly compare the two imaging modalities. Based on this analysis, we learn that while 3D can be reliably reconstructed form sonar, as far as the current technology shows, the accuracy is lower than the one obtained based on vision, that the performance in sonar and in vision are highly correlated, that both modalities favor shapes that are not round, and that while the current vision method is able to better reconstruct the 3D shape, its advantage with respect to estimating the normal's direction is much lower.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123303239","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Per-Image Super-Resolution for Material BTFs 材料btf的逐图超分辨率
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105256
D. D. Brok, S. Merzbach, Michael Weinmann, R. Klein
Image-based appearance measurements are fundamentally limited in spatial resolution by the acquisition hardware. Due to the ever-increasing resolution of displaying hardware, high-resolution representations of digital material appearance are desireable for authentic renderings. In the present paper, we demonstrate that high-resolution bidirectional texture functions (BTFs) for materials can be obtained from low-resolution measurements using single-image convolutional neural network (CNN) architectures for image super-resolution. In particular, we show that this approach works for high-dynamic-range data and produces consistent BTFs, even though it operates on an image-by-image basis. Moreover, the CNN can be trained on down-sampled measured data, therefore no high-resolution ground-truth data, which would be difficult to obtain, is necessary. We train and test our method's performance on a large-scale BTF database and evaluate against the current state-of-the-art in BTF super-resolution, finding superior performance.
基于图像的外观测量在空间分辨率上受到采集硬件的限制。由于显示硬件的分辨率不断提高,数字材料外观的高分辨率表示是真实渲染所需要的。在本文中,我们证明了材料的高分辨率双向纹理函数(btf)可以通过使用图像超分辨率的单图像卷积神经网络(CNN)架构从低分辨率测量中获得。特别是,我们表明这种方法适用于高动态范围的数据,并产生一致的btf,即使它是在逐幅图像的基础上运行的。此外,CNN可以在下采样的测量数据上进行训练,因此不需要高分辨率的地面真值数据,这将难以获得。我们在一个大规模的BTF数据库上训练和测试了我们的方法的性能,并对当前最先进的BTF超分辨率进行了评估,发现了更好的性能。
{"title":"Per-Image Super-Resolution for Material BTFs","authors":"D. D. Brok, S. Merzbach, Michael Weinmann, R. Klein","doi":"10.1109/ICCP48838.2020.9105256","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105256","url":null,"abstract":"Image-based appearance measurements are fundamentally limited in spatial resolution by the acquisition hardware. Due to the ever-increasing resolution of displaying hardware, high-resolution representations of digital material appearance are desireable for authentic renderings. In the present paper, we demonstrate that high-resolution bidirectional texture functions (BTFs) for materials can be obtained from low-resolution measurements using single-image convolutional neural network (CNN) architectures for image super-resolution. In particular, we show that this approach works for high-dynamic-range data and produces consistent BTFs, even though it operates on an image-by-image basis. Moreover, the CNN can be trained on down-sampled measured data, therefore no high-resolution ground-truth data, which would be difficult to obtain, is necessary. We train and test our method's performance on a large-scale BTF database and evaluate against the current state-of-the-art in BTF super-resolution, finding superior performance.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124791407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Fast confocal microscopy imaging based on deep learning 基于深度学习的快速共聚焦显微镜成像
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105215
Xiu Li, J. Dong, Bowen Li, Yi Zhang, Yongbing Zhang, A. Veeraraghavan, Xiangyang Ji
Confocal microscopy is the de-facto standard technique in bio-imaging for acquiring 3D images in the presence of tissue scattering. However, the point-scanning mechanism inherent in confocal microscopy implies that the capture speed is much too slow for imaging dynamic objects at sufficient spatial resolution and signal to noise ratio(SNR). In this paper, we propose an algorithm for super-resolution confocal microscopy that allows us to capture high-resolution, high SNR confocal images at an order of magnitude faster acquisition speed. The proposed Back-Projection Generative Adversarial Network (BPGAN) consists of a feature extraction step followed by a back-projection feedback module (BPFM) and an associated reconstruction network, these together allow for super-resolution of low-resolution confocal scans. We validate our method using real confocal captures of multiple biological specimens and the results demonstrate that our proposed BPGAN is able to achieve similar quality to high-resolution confocal scans while the imaging speed can be up to 64 times faster.
共聚焦显微镜是事实上的标准技术,在生物成像获得三维图像的存在组织散射。然而,共聚焦显微镜固有的点扫描机制意味着,在足够的空间分辨率和信噪比(SNR)下,对动态物体成像的捕获速度太慢。在本文中,我们提出了一种超分辨率共聚焦显微镜算法,使我们能够以更快的采集速度捕获高分辨率,高信噪比的共聚焦图像。提出的反投影生成对抗网络(BPGAN)包括一个特征提取步骤,然后是一个反投影反馈模块(BPFM)和一个相关的重建网络,这些共同允许低分辨率共聚焦扫描的超分辨率。我们使用多个生物标本的真实共聚焦捕获验证了我们的方法,结果表明,我们提出的BPGAN能够达到与高分辨率共聚焦扫描相似的质量,而成像速度可以快64倍。
{"title":"Fast confocal microscopy imaging based on deep learning","authors":"Xiu Li, J. Dong, Bowen Li, Yi Zhang, Yongbing Zhang, A. Veeraraghavan, Xiangyang Ji","doi":"10.1109/ICCP48838.2020.9105215","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105215","url":null,"abstract":"Confocal microscopy is the de-facto standard technique in bio-imaging for acquiring 3D images in the presence of tissue scattering. However, the point-scanning mechanism inherent in confocal microscopy implies that the capture speed is much too slow for imaging dynamic objects at sufficient spatial resolution and signal to noise ratio(SNR). In this paper, we propose an algorithm for super-resolution confocal microscopy that allows us to capture high-resolution, high SNR confocal images at an order of magnitude faster acquisition speed. The proposed Back-Projection Generative Adversarial Network (BPGAN) consists of a feature extraction step followed by a back-projection feedback module (BPFM) and an associated reconstruction network, these together allow for super-resolution of low-resolution confocal scans. We validate our method using real confocal captures of multiple biological specimens and the results demonstrate that our proposed BPGAN is able to achieve similar quality to high-resolution confocal scans while the imaging speed can be up to 64 times faster.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117125048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
FoveaCam: A MEMS Mirror-Enabled Foveating Camera FoveaCam:一种MEMS反射镜支持的FoveaCam相机
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105183
Brevin Tilmon, Eakta Jain, S. Ferrari, S. Koppal
Most cameras today photograph their entire visual field. In contrast, decades of active vision research have proposed foveating camera designs, which allow for selective scene viewing. However, active vision's impact is limited by slow options for mechanical camera movement. We propose a new design, called FoveaCam, and which works by capturing reflections off a tiny, fast moving mirror. FoveaCams can obtain high resolution imagery on multiple regions of interest, even if these are at different depths and viewing directions. We first discuss our prototype and optical calibration strategies. We then outline a control algorithm for the mirror to track target pairs. Finally, we demonstrate a practical application of the full system to enable eye tracking at a distance for frontal faces.
现在大多数照相机都能拍摄整个视野。相比之下,几十年的主动视觉研究已经提出了聚焦相机的设计,它允许选择性地观看场景。然而,主动视觉的影响受到机械相机运动缓慢选项的限制。我们提出了一种新的设计,叫做FoveaCam,它通过捕捉一个微小的、快速移动的镜子的反射来工作。FoveaCams可以在多个感兴趣的区域获得高分辨率图像,即使这些区域处于不同的深度和观看方向。我们首先讨论了我们的原型和光学校准策略。然后,我们概述了镜子跟踪目标对的控制算法。最后,我们演示了整个系统的实际应用,以实现对正面面部的远距离眼动追踪。
{"title":"FoveaCam: A MEMS Mirror-Enabled Foveating Camera","authors":"Brevin Tilmon, Eakta Jain, S. Ferrari, S. Koppal","doi":"10.1109/ICCP48838.2020.9105183","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105183","url":null,"abstract":"Most cameras today photograph their entire visual field. In contrast, decades of active vision research have proposed foveating camera designs, which allow for selective scene viewing. However, active vision's impact is limited by slow options for mechanical camera movement. We propose a new design, called FoveaCam, and which works by capturing reflections off a tiny, fast moving mirror. FoveaCams can obtain high resolution imagery on multiple regions of interest, even if these are at different depths and viewing directions. We first discuss our prototype and optical calibration strategies. We then outline a control algorithm for the mirror to track target pairs. Finally, we demonstrate a practical application of the full system to enable eye tracking at a distance for frontal faces.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129286468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
[Copyright notice] (版权)
Pub Date : 2020-04-01 DOI: 10.1109/iccp48838.2020.9105200
{"title":"[Copyright notice]","authors":"","doi":"10.1109/iccp48838.2020.9105200","DOIUrl":"https://doi.org/10.1109/iccp48838.2020.9105200","url":null,"abstract":"","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124582320","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Action Recognition from a Single Coded Image 基于单个编码图像的动作识别
Pub Date : 2020-04-01 DOI: 10.1109/ICCP48838.2020.9105176
Tadashi Okawara, Michitaka Yoshida, Hajime Nagahara, Yasushi Yagi
Cameras are prevalent in society at the present time, for example, surveillance cameras, and smartphones equipped with cameras and smart speakers. There is an increasing demand to analyze human actions from these cameras to detect unusual behavior or within a man-machine interface for Internet of Things (IoT) devices. For a camera, there is a trade-off between spatial resolution and frame rate. A feasible approach to overcome this trade-off is compressive video sensing. Compressive video sensing uses random coded exposure and reconstructs higher than read out of sensor frame rate video from a single coded image. It is possible to recognize an action in a scene from a single coded image because the image contains multiple temporal information for reconstructing a video. In this paper, we propose reconstruction-free action recognition from a single coded exposure image. We also proposed deep sensing framework which models camera sensing and classification models into convolutional neural network (CNN) and jointly optimize the coded exposure and classification model simultaneously. We demonstrated that the proposed method can recognize human actions from only a single coded image. We also compared it with competitive inputs, such as low-resolution video with a high frame rate and high-resolution video with a single frame in simulation and real experiments.
摄像头在当今社会非常普遍,例如监控摄像头,以及配备摄像头和智能扬声器的智能手机。越来越多的人需要分析这些摄像头的人类行为,以检测异常行为或物联网(IoT)设备的人机界面。对于相机来说,在空间分辨率和帧率之间存在权衡。一个可行的方法来克服这种权衡是压缩视频传感。压缩视频感知使用随机编码曝光,并从单个编码图像重建高于传感器读出帧率的视频。从单个编码图像中识别场景中的动作是可能的,因为图像包含用于重建视频的多个时间信息。本文提出了一种基于单幅编码曝光图像的无重构动作识别方法。我们还提出了一种深度感知框架,该框架将相机感知和分类模型建模到卷积神经网络(CNN)中,同时对编码曝光和分类模型进行联合优化。我们证明了该方法可以仅从单个编码图像中识别人类行为。我们还将其与竞争性输入进行了比较,例如模拟和真实实验中的高帧率的低分辨率视频和单帧的高分辨率视频。
{"title":"Action Recognition from a Single Coded Image","authors":"Tadashi Okawara, Michitaka Yoshida, Hajime Nagahara, Yasushi Yagi","doi":"10.1109/ICCP48838.2020.9105176","DOIUrl":"https://doi.org/10.1109/ICCP48838.2020.9105176","url":null,"abstract":"Cameras are prevalent in society at the present time, for example, surveillance cameras, and smartphones equipped with cameras and smart speakers. There is an increasing demand to analyze human actions from these cameras to detect unusual behavior or within a man-machine interface for Internet of Things (IoT) devices. For a camera, there is a trade-off between spatial resolution and frame rate. A feasible approach to overcome this trade-off is compressive video sensing. Compressive video sensing uses random coded exposure and reconstructs higher than read out of sensor frame rate video from a single coded image. It is possible to recognize an action in a scene from a single coded image because the image contains multiple temporal information for reconstructing a video. In this paper, we propose reconstruction-free action recognition from a single coded exposure image. We also proposed deep sensing framework which models camera sensing and classification models into convolutional neural network (CNN) and jointly optimize the coded exposure and classification model simultaneously. We demonstrated that the proposed method can recognize human actions from only a single coded image. We also compared it with competitive inputs, such as low-resolution video with a high frame rate and high-resolution video with a single frame in simulation and real experiments.","PeriodicalId":406823,"journal":{"name":"2020 IEEE International Conference on Computational Photography (ICCP)","volume":"2006 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117043235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
期刊
2020 IEEE International Conference on Computational Photography (ICCP)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1