Department of Intelligent Media, ISIR, Osaka Univ.

Sitemap Japanese

Computational Imaging/Photography

Pioneering novel optical sensing systems exploiting computation

What is computational imaging?

Imaging is the foundation of all science and technology, including medical, biotechnology, and robotics. Imaging technology has been improved over a long history, mainly in lens design, but in recent years, the technological innovation at the principle level is emerging against the backdrop of advances in information science. In Yagi Laboratory, we are studying the next generation imaging technology called Computational Imaging, which is based on the cooperative design of cutting-edge optics and information science. Our research themes are encoded measurement methods using optical information processing, image reconstruction processing methods based on mathematical optimization, and pioneering their co-design.

Wide-field lens-less camera with a sparse image sensor and sparse restoration

In this study, we propose a wide-field lens-less camera that combines image capture by a “sparse image sensor” containing many randomly arranged micro-apertures and image restoration by compressed sensing (sparse restoration). The optical system with two sparse image sensors with their photosensitive surfaces facing each other enables us to measure sparse lens-less encoded images on the front and back sides at a time. After the measurement, a dense object image can be recovered from each encoded image by applying a compressed sensing-type image reconstruction process. This enables snapshot wide-angle imaging with an ultra-thin optical system consisting only of an image sensor. We have so far succeeded in a proof-of-principle demonstration using simulated optical experiments, and are currently developing a prototype sparse image sensor.

Award
  1. IWISS2018 ITE Open Poster Session Award 1st place (2018.11).
Publications
  1. T. Nakamura, K. Kagawa, S. Torashima, and M. Yamaguchi, “Super Field-of-View Lensless Camera by Coded Image Sensors,” Sensors, Vol 19, No. 6, 1329 (2019). [PDF]
  2. T. Nakamura, K. Kagawa, S. Torashima, and M. Yamaguchi, “Lensless imaging by coded image sensors,” 4th International Workshop on Image Sensors and Imaging Systems (IWISS2018), 16 (Tokyo, demo, 2018.11).

High-speed three-dimensional laser scanning microscopy using depth-spatial coordinate linear transformation optics

Laser scanning microscopy is a microscopy technique that realizes high spatial resolution and deep measurement depth and is widely used for biological sample observation. In this study, we propose a new three-dimensional laser scanning microscopy method that can be implemented with two-dimensional scanning by combining a wide-depth batch illumination system using optical needles and a measurement system via a depth-to-spatial coordinate linear transformation optical system using computer-synthesized holographic optical elements. Optical experiments demonstrate the realization of depth-spatial coordinate linear transformation imaging and the resulting high-speed three-dimensional microscopic imaging of the mouse brain.

Publication
  1. Y. Kozawa, T. Nakamura, Y. Uesugi, and S. Sato, “Wavefront engineered light needle microscopy for axially resolved rapid volumetric imaging,” Biomedical Optics Express, Vol. 13, No. 3, pp. 1702-1717, (2022). [PDF]
  2. Y. Kozawa, T. Nakamura, and S. Sato, “Volumetric Imaging Utilizing Linear-Shift Point-Spread Function Based on Multiplexed Computer-Generated Hologram,” Focus on Microscopy (FOM2021), SU-PAR1-E (online, oral, 2021.03).
  3. T. Nakamura, S. Igarashi, Y. Kozawa, and M. Yamaguchi, “Non-diffracting linear-shift point-spread function by focus-multiplexed computer-generated hologram,” Optics Letters, Vol. 43, No. 24, pp. 5949-5952 (2018). [PDF]

All-in-focus lensless camera with radial encoding mask

Camera out-of-focus is often preferred in art photography to making the subject stand out beautifully, but in measurement applications, it is desirable to eliminate it as much as possible because it lacks information content. Recently studied lens-less cameras, which are based on post-measurement reconstruction processing, can realize an all-in-focus camera with no out-of-focus images in principle because the optical design without a lens is allowed. In this study, a radial encoding mask is used as an encoding optical element of the lens-less camera to realize snapshot all-in-focus imaging by physically implementing an optical system response function that is independent of the object’s distance. We have confirmed the numerical demonstration of the proposed all-in-focus lensless imaging principle and the experimental demonstration in a simplified setup.

Publication
  1. T. Nakamura, S. Igarashi, S. Torashima, and M. Yamaguchi, “Extended depth-of-field lensless camera using a radial amplitude mask,” Computational Optical Sensing and Imaging (COSI2020), CW3B.2 (online, oral, 2020.6). [PDF]

Improvement of resolution of lensless camera by using wavelength dependence of optical transfer function

Fresnel zone aperture lens-less cameras have advantages in terms of high-speed image reconstruction processing and analytical solution of the optical transfer function, but the spatial resolution is limited by the zero-crossing of the optical transfer function (OTF) caused by the aperture structure and light diffraction. In this study, we propose an image processing technique to recover high-resolution image information by compensating the cutoff frequency information of a specific color channel from the information of other color channels, taking advantage of the strong wavelength dependence of this zero-crossing property. Optical experiments based on the prototype construction quantitatively demonstrate the effect of the proposed method on the high resolution.

Publication
  1. T. Nakamura, T. Watanabe, S. Igarashi, X. Chen, K. Tajima, K. Yamaguchi, T. Shimano, and M. Yamaguchi, “Superresolved image reconstruction in FZA lensless camera by color-channel synthesis,” Optics Express, Vol. 28, No. 26, pp. 39137-39155 (2020). [PDF]

Non-line-of-sight imaging using holographic unequal angle reflectors

A holographic optical element (HOE) is a type of diffractive optical element that modulates light waves based on the principle of holography and can record and reproduce any light wave input/output response that can be implemented by optical diffraction. The HOE can record and reproduce arbitrary light wave input/output responses that can be implemented by optical diffraction. This enables us to implement a “see-through unequal angle reflector”, which cannot be realized by refractive optical elements, and to configure an imaging system through it, it is possible to take a frontal view of an object from a non-line-of-sight direction through a transparent orthorhombic element. In this study, we developed a large homogeneous mounting method of the unequal-angle reflector HOE, proposed a method of optical compensation of wavelength dispersion, full colorization, removal of unwanted ambient light, and demonstrated its application to image communication systems. The effectiveness of each research item was demonstrated based on optical experiments and prototype construction.

Award
  1. IDW’17 Best Paper Award (2018.1)
Publications
  1. S. Kimura, Y. Aburakawa, F. Watanabe, S. Torashima, S. Igarashi, T. Nakamura, and M. Yamaguchi, “Holographic Video Communication System Realizing Virtual Image Projection and Frontal Image Capture,” ITE Transactions on Media Technology and Applications, Vol. 9, No. 1, pp. 105-112 (2021). [PDF]
  2. F. Matsui, F. Watanabe, T. Nakamura, and M. Yamaguchi, “Unmixing of the background components in an off-axis holographic-mirror-based imaging system using spectral image processing,” Optics Express, Vol. 28, No. 26, pp. 39998-40012 (2020). [PDF]
  3. F. Watanabe, T. Nakamura, S. Torashima, S. Igarashi, S. Kimura, Y. Aburakawa, and M.Yamaguchi, “Dispersion compensation for full-color virtual-imaging systems with a holographic off-axis mirror,” SPIE Photonics West, 11306-3 (San Francisco, oral, 2020.2). [LINK]
  4. T.Nakamura, S. Kimura, K. Takahashi, Y. Aburakawa, S. Takahashi, S. Igarashi, S. Torashima, and M. Yamaguchi, “Off-axis virtual-image display and camera by holographic mirror and blur compensation,” Optics Express, Vol. 26, No. 19, pp. 24864-24880 (2018). [PDF]

Projected see-through 4D light field display and aerial video touch detection

A 4D light field display is a type of stereoscopic display based on the light ray reproduction method using lens arrays and can reproduce optical real images in the air. In the projection-type configuration where the projector and lens array are placed far from each other, it is important to align the optical elements with the projected image on the lens array surface. In the case of the holographic optical element (HOE), the lens array surface can be made see-through and the environment-integrated application is possible, but the alignment is not easy due to the transparency of the HOE. In this study, we proposed a fast positioning method based on binary sinusoidal pattern projection and experimentally demonstrated 4D image reproduction based on it. In addition, we propose a fast and stable method for touch detection of 4D images by using scattered light detection and color information identification for 3D touch user interface applications and experimentally demonstrate the effectiveness of the proposed method.

Publications
  1. I. A. S. S. Chavarría, T. Nakamura, and M. Yamaguchi, “Interactive optical 3D-touch user interface using a holographic light-field display and color information,” Optics Express, Vol. 28, No. 24, pp. 36740-36755 (2020). [PDF]
  2. T. Nakamura and M. Yamaguchi, “Simple geometrical calibration procedure for a projection-type holographic light-field display,” Digital Holography & 3-D Imaging (DH2018), DTh3D.5 (Orlando, oral, 2018.6).
  3. T. Nakamura and M. Yamaguchi, “Rapid calibration of a projection-type holographic light-field display using hierarchically upconverted binary sinusoidal patterns,” Applied Optics, Vol. 56, No. 34, pp. 9520-9525 (2017). [PDF]

Gigapixel camera autofocus

The gigapixel camera, which combines a ball lens and a micro-camera array, can take snapshots of images that are two orders of magnitude higher in resolution than ordinary cameras. Because they combine a wide field of view with high resolution and can faithfully record every corner of a large physical space, they are expected to find applications in surveillance and other applications. On the other hand, the optical design of individual micro cameras is designed for high magnification, so the depth range of focus is narrow and a fast autofocus mechanism is required. In this study, a fast autofocus system using a hierarchical algorithm is proposed and implemented in a gigapixel camera. The effectiveness of the proposed method was demonstrated by an actual experiment using a prototype.

Publications
  1. T. Nakamura, D. S. Kittle, S. H. Youn, S. D. Feller, J. Tanida, and D. J. Brady, “Autofocus for a multiscale gigapixel camera,” Applied Optics, Vol. 52, No. 33, pp. 8146-8153 (2013). [PDF]
  2. S. H. Youn, H. S. Son, D. L. Marks, A. Pendleton, P. O. McLaughlin, T. Nakamura, D. J. Brady, and J. Kim, “Thru-focus Optical Analysis of Microcameras in a Gigapixel Camera,” US-Korea Conference on Science, Technology, and Entrepreneurship (UKC2013), K-1029 (New York, oral, 2013.8).

Small wide-angle imaging via multiple scatterers based on the transmission matrix method

Since light passing through multiple scatterers is strongly disturbed physically, it is naturally impossible to see object image information (object image) in the incident space from the transmitted light (scattered image). On the other hand, since linearity holds for optical phenomena in general, the object image can be mathematically reconstructed from the scattered image by matrix operations. In this study, we implemented an ultra-compact wide-angle lens-less imaging system by placing an artificial scatterer in front of the image sensor, which is optimally designed for wide-angle imaging. The imaging principle is demonstrated and its effectiveness is quantitatively verified.

Publication
  1. T. Nakamura, R.Horisaki, and J. Tanida, “Compact wide-field-of-view imager with a designed disordered medium,” Optical Review, Vol. 22, No. 1, pp. 19-24 (2015). [PDF]

Imaging with active use of compound eye optics

Compound eye optics in insects is widely known for its advantage of being thin, lightweight, and wide-angle compared with the monocular optics corresponding to human visual systems and cameras, but it is also valuable from the viewpoint of the feasibility of 3D measurement and improvement of freedom of optical design in engineering applications. In this study, we investigated 3D image reconstruction and its parallel processing in the configuration where individual eyes are separated (serial image eye). In addition, we proposed an imaging system with an extended field of view and depth of field inspired by the configuration in which individual eyes are combined (duplex eye) and showed that the system can be applied not only to sensing but also to projection. We have also shown that the system can be applied not only to sense but also to a projector. The results were demonstrated through experiments.

Publication
  1. T. Nakamura, R. Horisaki, and J. Tanida, “Computational phase modulation in light field imaging,” Optics Express, Vol. 21, No. 24, pp. 29523-29543 (2013). [PDF]
  2. T. Nakamura, R. Horisaki, and J. Tanida, “Computational superposition projector for extended depth of field and field of view,” Optics Letters, Vol. 38, No. 9, pp. 1560-1562 (2013). [PDF]
  3. T. Nakamura, R. Horisaki, and J. Tanida, “Computational superposition compound eye imaging for extended depth-of-field and field-of-view,” Optics Express, Vol. 20, No. 25, pp. 27482-27495 (2012). [PDF]