Image Processing

From image segmentation and super resolution

Temporal Super Resolution from a Single Quasi-periodic Image Sequence based on Phase Registration

This paper describes a method for temporal super-resolution from a single quasi-periodic image sequence. A so-called reconstruction-based method is applied to construct a one-period image sequence with a high frame rate based on phase registration data in sub-frame order among multiple periods of the image sequence. First, the periodic image sequence to be reconstructed is expressed as a manifold in the parametric eigenspace of the phase. Given an input image sequence, phase registration and manifold reconstruction are alternately executed iteratively within an energy minimization framework that considers data fitness and the smoothness of both the manifold and the phase evolution. The energy minimization problem is solved through three-step coarse-to-fine procedures to avoid local minima. The experiments using both simulated and real data confirm the realization of temporal super-resolution from a single image sequence.

Publications

1. Y. Makihara, A. Mori, and Y. Yagi, “Temporal Super Resolution from a Single Quasi-Periodic Image Sequence Based on Phase Registration,” Proc. the 10th Asian. Conf. on Computer Vision, pp. 107-120, Queenstown, New Zealand, Nov. 2010.[PDF]

Phase Registration of a Single Quasi-periodic Signal using Self Dynamic Time Warping

This paper proposes a method for phase registration of a single non-parametric quasi-periodic signal. After a short-term period has been detected for each sample by normalized autocorrelation, Self Dynamic Time Warping (Self DTW) between a quasi-periodic signal and that with multiple-period shifts is applied to obtain corresponding samples of the same phase. A phase sequence is finally estimated by the optimization framework including the data term derived from the correspondences, the regularization term derived from short-term periods, and a monotonic increasing constraint of the phase. Experiments on quasi-periodic signals from both simulated and real data show the effectiveness of the proposed method.

Publications

1. Y. Makihara, N.T. Trung, H. Nagahara, R. Sagawa, Y. Mukaigawa, and Y. Yagi, “Phase Registration of a Single Quasi-Periodic Signal Using Self Dynamic Time Warping,” Proc. the 10th Asian. Conf. on Computer Vision, pp. 1965-1975, Queenstown, New Zealand, Nov. 2010.[PDF]

Earth Mover’s Morphing: Topology-Free Shape Morphing Using Cluster-based EMD Flows

This paper describes a method for topology-free shape morphing based on region cluster-based Earth Mover’s Distance (EMD) flows since existing methods for closed curve/surface-based shape morphing are inapplicable to regions with different genera. First, the shape region is decomposed into a number of small clusters by Fuzzy C-Means clustering. Next, the EMD between the clusters of two key shapes is calculated and the resultant EMD flows are exploited as a weighted many-to-many correspondence among the clusters. Then, the fuzzy clusters are transported based on the EMD flows and a transition control parameter. Unlike the closed curve/surface-based methods, the morphs using cluster transportation are not guaranteed to be a binary image, and hence graph cut-based binary denoising is applied to a volumetric image of the two-dimensional position and the one-dimensional transition control parameter. The experiments demonstrate that the proposed method can perform morphing between shapes with different genera, such as walking silhouettes or alphabetical characters.

Publications

1. Y. Makihara and Y. Yagi, “Earth Mover’s Morphing: Topology-Free Shape Morphing Using Cluster-Based EMD Flows,” Proc. the 10th Asian. Conf. on Computer Vision, pp. 2302-2315, Queenstown, New Zealand, Nov. 2010.[PDF]

Foreground and Shadow Segmentation based on a Homography-Correspondence Pair

A static binocular camera system is widely used in many computer vision applications; and being able to segment foreground, shadow, and background is an important problem for them. In this paper, we propose a homography-correspondence pair-based segmentation framework. Existing segmentation approaches, based on homography constraints, often suffer from occlusion problems. In our approach, we treat a homography-correspondence pair symmetrically, to explicitly take the occlusion relationship into account, and we regard the segmentation problem as a multi-labeling problem for the homography-correspondence pair. We then formulate an energy function for this problem and get the pair-wise segmentation results by minimizing them via an alpha-beta swap algorithm. Experimental results show that accurate segmentation is obtained in the presence of the occlusion region in each side image.

Publications

1. H. Iwama, Y. Makihara, and Y. Yagi, “Foreground and Shadow Segmentation Based on a Homography-Correspondence Pair,” Proc. the 10th Asian. Conf. on Computer Vision, pp. 2790-2802, Queenstown, New Zealand, Nov. 2010.[PDF]

Deformable Registration for Generating Dissection Image of an Intestine from Annular Image Sequence

Examination inside an intestine by an endoscope is difficult and time-consuming because the whole image of the intestine cannot be taken at one time due to the limited field of view. Thus, it is necessary to generate a dissection image, which can be obtained by extending the image of an intestine. We acquire an annular image sequence with an omnidirectional or wide-angle camera, and then generate the dissection image by mosaicing the image sequence. Though usual mosaicing techniques transform an image by perspective or affine transformations, these are not suitable for our situation because the target object is a generalized cylinder and the camera motion is unknown a priori. Therefore, we propose a novel approach for image registration that deforms images by a twodimensional-polynomial function which parameters are estimated from optical flow. We evaluated our method by registering annular image sequences and we successfully generated dissection images, as presented in this paper.

Publications

1. Suchit Pongnumkul, Ryusuke Sagawa, Tomio Echigo, Yasushi Yagi, “Deformable Registration for Generating Dissection Image of an Intestine from Annular Image Sequence”, In Computer Vision for Biomedical Image Applications, pp.271–280, 2005.