1.

INTRODUCTION

MOUNT ZION COLLEGE OF ENGINEERING,

IMAGE STITCHING

Recently a specialised form of image mosaicing known as image stitching has become increasingly common, especially in the making of panoramic images. Stitched images are used in the applications such as interactive panoramic viewing of images, architectural walk-through, multi-node movies and other applications associated with modelling the 3D environment using images acquired from the real world. Panoramic image stitching is the process performed to generate one panoramic image from a series of smaller, overlapping images. The fig. below shows the process of producing a panoramic image.

Mount zion college of engineering

IMAGE STITCHING

The first step in the generation of a panoramic image is to select the position and acquisition of images. In this step, a decision needs to be taken on the type of resultant images. According to the required panoramic images, different image acquisition methods are used to acquire the series of images. After the images have been acquired, some processing might need to be applied to images before they can be stitched. But in this project we are not bothered about this procedure as we are emphasizing on only x-direction stitching. The process of image stitching has been divided into two steps, image registration or feature matching and image merging. During feature matching, portions of adjacent images are compared to find the translations which align the images. Once the overlapping images have been registered, they need to be merged together to form a single panoramic image. The process of merging is performed to make the transition between adjacent images visually undetectable. IMAGE ACQUISITION Image acquisition is not actually a part of our project, but we specify that the images taken for stitching must be taken from the same camera with the same focal length so that the calibration matrix of the camera remains the same. One of the most commonly faced problems in image acquisition is the intensity shift between adjacent images. In an ideal case, the same region or object should have the same intensity values in adjacent images. However, due to the variation in the lighting intensity, or the angle between the camera and the light source, the intensity values for the same region or object are different in adjacent images. And so we specify that the camera is placed on a tripod stand so that the images taken for stitching is of the same rotation axis. IMAGE REGISTRATION To form a larger image with a set of overlapping images, it is necessary to find the translations to align the images. The process of image registration aims to find the translations to align two or more overlapping images such that the projection from the view point through any position in the aligned images into the 3D world is unique. An image registration method usually consists of four main components. They are the feature set, similarity measure, search set and search strategy. These four components
Mount zion college of engineering

IMAGE STITCHING

respectively define what is used to compare the images, how to evaluate the similarity between the images, the range of possible transformations between the images and how to decide the next transformation for evaluation based on the current similarity measure. By varying the contents of these four components, different registration methods with different behaviours can be constructed. The feature set refers to the set of features to be used in the comparison of the images. The similarity measure is a function which returns a scalar value that provides an indication of the similarities between two features. The intensity values of the selected feature from the images are used to calculate the similarity measures. The values of the similarity measures are used to select transformations for aligning the images. The search set is a set of possible transformations for aligning the images. It contains transformations such as horizontal or vertical translation, rotations, or other more complex transformations obtained by a combination of translations and rotations. The transformations contained in the search set are evaluated by the similarity measures to decide the best transformation required to align the given images. The search strategy is the algorithm that decides how to select the next transformations from the search set. BASIC METHODS OF FEATURE MATCHING There are different methods of matching the features of the two images to find out the overlapping region. They may be: 1. Pixel Matching 2. Edge Matching 3. Vertices Matching 4. RANSAC Methods 5. SIFT Features Here in this project the basic method, pixel matching is used. The disadvantage of this method is that there may be many pixels of second image matching the same pixel of the first image. So a window (a 9*9 window) is selected and the window is compared for a match thereby reducing the possibility of multiple matches. The most modern technology in feature matching is SIFT features matching in which there must be at least four points for comparison whereas the earlier RANSAC technique required minimum seven points. By selecting the first method and improving it we find the overlapping region and successfully complete the first step of image stitching, that is, feature matching.

Mount zion college of engineering

IMAGE STITCHING

IMAGE MERGING Image merging is the process of adjusting the values of pixels in two registered images, such that when the images are joined, the transition from one image to the next is invisible. At the same time, the merged images should preserve the quality of the input images as much as possible. In an ideal case, the overlapping region of adjacent images should be identical, so that the intensity values of Ik are equal to intensity values of the corresponding position in Ik+1 for any point (a, b) within the overlapping region. Removing part of the overlapping regions in adjacent images and concatenating the trimmed images often produce images with distinctive seams. A seam is the artificial edge produced by the intensity differences of pixels immediately next to where the images are joined. One approach to remove the seam is to perform the intensity adjustment locally, within a defined neighbourhood of the seam, so that only the intensity values in the neighbourhood are affected by the adjustment. Another approach is to perform a global intensity adjustment on the images to be merged, so that apart from the intensity values within the overlapping regions, intensity values outside the overlapping regions may also need to be adjusted. One of our objectives is to merge the images so that the seam between images is visually undetectable .The second objective is to preserve the quality of the original images as much as possible so that the merged image is not seriously degraded by the intensity adjustment required to remove the seam. There are many methods for merging two images. Most of them are based on translation. The merging algorithms differ based on the applications where the stitched image may be used. The intensity adjustment may be done with respect to the median filtered regions or with respect to corresponding pixels in overlapping regions. But here we propose and use a selective technique to increase the simplicity of the work. This is done just by cropping the second image till the seam which contains the overlapping region calculated in the Feature Matching stage. The advantage of this technique is that we can reduce the complexity of the blending algorithm that has to be used to conquer reality to the stitched image.

CHALLENGES
Mount zion college of engineering

IMAGE STITCHING

The challenges of this project may lie in the fact that we calculate more than one point in the feature matching phase. If there are two points, say, one point may stitch the image to perfection but the other will definitely have error (this is because two images cant have more than one overlapping region). So to avoid this we collect user data to select which point to be used to stitch the image. ANALYSIS The project also analyses the perfection of the stitched image. This is done by calculating the percentage of variance of the stitched image from the original image. The process is as follows. We cut the original image into two with overlapping regions. Then we stitch the two images and compare the stitched image with the original image using PSNR (peak signal to noise ratio) or Euclidean distance. PSNR PSNR or peak signal to noise ratio is the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. Because many signals have a very wide dynamic range, PSNR is usually expressed in terms of the logarithmic decibel scale. The mean square error (MSE) which for two mn monochrome images I and K where one of the images is considered a noisy approximation of the other is defined as:

The PSNR is defined as:

Here, MAXi is the maximum possible pixel value of the image. For colour images with three RGB values per pixel, the definition of PSNR is the same except the MSE is the sum over all squared value differences divided by image size and by three.
Mount zion college of engineering

IMAGE STITCHING

Euclidean Distance Euclidean distance or Euclidean metric is the "ordinary" distance between two points that one would measure with a ruler, which can be proven by repeated application of the Pythagorean Theorem. For two N-D points, computed as and , the distance is

Mount zion college of engineering

IMAGE STITCHING

2. SOFTWARE REQUIREMENTS SPECIFICATION

Mount zion college of engineering

IMAGE STITCHING

2.1. Introduction 2.1.1. Purpose

The purpose of this document is to present a detailed description of Image Stitching. It explains the purpose and features of the system, what the system does, and the constraints under which it operates. This document is intended to the programmers and those who intend to implement the system. This document may be viewed and read with an overview of understanding the basic terms and requirements in designing the Image Stitching software. 2.1.2. Scope

Image stitching or photo stitching is the process of combining multiple photographic images with overlapping fields of view to produce a segmented panorama or high-resolution image. Commonly performed through the use of computer software, most approaches to image stitching require nearly exact overlaps between images and identical exposures to produce seamless results. It is also known as mosaicing. This project aims at developing a solution for the stitching of multiple images together (two at a time). Stitched images are intended to be used in applications such as interactive panoramic viewing of images, architectural walk-through, multi-node movies and other applications associated with modelling the 3D environment using images acquired from the real world. Stitching may also be applied in many clinical studies, including those of cancer, where it is required to acquire images of whole tumour section while retaining a microscopic resolution. 2.1.3. Definitions, Acronyms and Abbreviations.

Stitching:

Process of joining together two images with overlap regions.

Correlation points: Similar points/Points of similarity between images. Seams: UI: A line of junction formed while stitching images User Interface

Mount zion college of engineering

IMAGE STITCHING

10

2.1.4. References

1. 2.

IEEE Guide to Software Requirements Specifications (Std 830-1993) An Improved Algorithm on Image Stitching based on SIFT features Jing Xing, Zhenjiang Miao Institute of Information Science, Beijing Jiao Tong University, Beijing 100044, P.R. China 3. Seamless Image Stitching in the Gradient Domain Anat Levin School of Computer Science and Engineering, The Hebrew University of Jerusalem 4. Image Stitching -Comparisons and New Techniques Chia-Yen Chen Computer Science Department of the University of Auckland 5. An algorithm for image stitching and blending Vladan Rankov Advanced Technology Development Group, Gray Cancer Institute, Mount Vernon Hospital, Northwood, Middlesex, HA6 2JR, United Kingdom; 2.1.5. Overview This document deals with the Image Stitching System- the requirements of

the system, the design, functions, constraints and interfaces. This document is ment to give the developers and users of the system, a clear understanding of what the system does, what are the constraints involved etc.

Mount zion college of engineering

IMAGE STITCHING

11

2.2. The Overall Description 2.2.1. Product Perspective The project proposes an effective algorithm for achieving effective image stitching for the use in medical applications, 3D modelling etc. The main feature of this image stitching system is that it provides a cost effective method for acquiring panoramic images from panoramic camera. The system comprises of effective seamless stitching using automatic detection of correlation points. The user will also have an option to select the correlation points in case no correlation points were found automatically. 2.2.1.1. Memory Constraints

The system will be able to work with 256 Mb of primary memory. 2.2.1.2. Operations

The system accepts as input, two images at a time, determines the correlation points and stitches the images together seamlessly. The user will be given option to select the correlation points manually in case the system fails to find them automatically. 2.2.2. Product Functions 1. 2. 3. 4. 5. Browse through the system and load images. Find correlation points between the images. Accept user input of correlation points if needed. Stitch the images by transformations. Blend the images to eliminate visible seams. 2.2.3. User Characteristics The expected users of the system may be professionals who would wish to stitch images for applications such as medical applications, 3D modelling etc as well as non-professionals who stitch images for ordinary needs. The User Interface (UI) of the system will be designed to suit the needs of both classes of users. It will be user friendly and easy to understand and use.

Mount zion college of engineering

IMAGE STITCHING

12

2.2.4. Constraints The system needs the Matlab engine to function. The hardware should support the Matlab 7 specifications. 2.2.5. Operating Environment The system works with Windows or Linux machines with Matlab engine installed. 2.3. Specific Requirements 2.3.1. External Interfaces The system accepts as input, two images and stitches them together after determining the correlation points. The input images must have overlapping regions for the algorithm to produce desired results. 2.1.1. Interfaces The system provides a Graphical User Interface (GUI) for the users. The GUI will have options to load the images to be stitched, automatically or manually select the correlation points, stitch the images, view and save the stitched image etc. 2.1.2. Hardware Interfaces The system has no special hardware interfaces (other than external cameras which are to be used to capture the images to be stitched). 2.1.3. Software Interfaces The system needs Matlab engine for functioning. 2.1.4. Communications Interfaces There is no special communication interfaces required.

2.3.2. Functional Requirements On accepting the input, the system shall normalize the intensities of the images. To find the correlation points, the algorithm constructs equal sized windows over each image

Mount zion college of engineering

IMAGE STITCHING

13

and applies intensity or binary edge calculations on the window region to find out the correlation points. Once the correlation points are determined, the algorithm transforms the second image over the first to stitch the images. The third function is to blend the images together. The system uses the gradient blending algorithm, according to which, a lookup table which contains the weightage values is constructed and the overlapping region between the images is constructed according to the weight. 2.3.3. Performance Requirements The system shall be able to work with images taken from the same region with overlapping regions despite of the intensity differences between the images. The algorithms shall be executed in less than 5 seconds on two images. The stitch produced shall be seamless and realistic. 2.3.4. Logical Database Requirements No logical database requirements 2.3.5. Design Constraints In the project is an evaluation of the image stitching algorithm. system needs Matlab engine to function. 2.3.6. Software System Attributes The

algorithm shall be implemented in Matlab and hence portability constraints may arise. The

3.6.1. Reliability and Portability

Mount zion college of engineering

IMAGE STITCHING

14

3. DESIGN DIAGRAMS AND IMPLEMENTATION DETAILS

Mount zion college of engineering

IMAGE STITCHING

15

3.1. DATA FLOW DIAGRAM

LEVEL 0: CONTEXT-LEVEL DIAGRAM

INPUT IMAGES-IMAGE1, IMAGE2

USER IMAGE STITCHER

Mount zion college of engineering

IMAGE STITCHING

16

LEVEL 1

IMAGE STITCHER

IMAGE1, IMAGE2

IMAGE REGISTRAT -ION CORRELATION POINTS

USER

IMAGE TRANSFO R-MER

STITCHED IMAGE

OUTPUT IMAGE

BLENDER

Mount zion college of engineering

IMAGE STITCHING

17

LEVEL 2

INPUT IMAGES USER

OBTAIN INTENSITY INFORMATIO N

GAUSSIAN TRANSFORM SCALE SPACE EXTREMUM DETECTION

CORRELATION POINTS

KEY POINT LOCALIZATIO N LOCALIZED CORRELATION POINTS

IMAGE TRANSFFOR MER

STITCHED IMAGE

BLENDER

LOOK UP TABLE WEIGHTED INTENSITY

OUTPUT IMAGE

USER

Mount zion college of engineering

IMAGE STITCHING

18

4. RESULTS AND DISCUSSIONS

Mount zion college of engineering

IMAGE STITCHING

19

In this software, two images are read using a file browser using the interface provided and both the images are displayed. Then a button STITCH is provided in order that on pressing the button the two images are stitched together. The stitched image is displayed in a window provided in the interface. The user interface is created using Matlab itself. After stitching the stitched image is analyzed with the original image using PSNR technique. The PSNR gives the amount of variance from the original image. That is minimum the PSNR value higher will be the accuracy of stitching. But it is observed that the PSNR value varies from co-relation point to point and it goes sometimes high else approaches 1. Hence we get on to the discussion that there must be some improved algorithm for better stitching.

Mount zion college of engineering

IMAGE STITCHING

20

5. CONCLUSIONS AND FUTURE SCOPE

Mount zion college of engineering

IMAGE STITCHING

21

The project accomplishes an analysis part which takes us on to the conclusion that some other feature matching technique must be employed to get a well stitched image. There are so many of them. The SIFT features may be one of them. Ransac may be another. After calculating the PSNR for SIFT and RANSAC algorithms the value may show a great decrease. Hence this takes the project on the run. The future scope of this project is to extend the Image Stitching possibilities using improved algorithms (n-direction) so that the idea of robotic vision through panoramic images be accomplished. The scope of the project is so vast as robotics is the emerging science and any development in that will lead to revolution. The vastness of Image stitching can extend beyond our earth till satellites. This is when the satellites use this particular software to trace its path. The satellites take a series of images and stitch them together to form a panoramic image and thereby satisfying the need. Thus the project proves to be the limited version of the need of tomorrow and thus is worth and any work or study continued upon this one will be an achievement as there are only limited amount of thesis available on the particular topic.

Mount zion college of engineering

IMAGE STITCHING

22

6. REFERENCES

Mount zion college of engineering

IMAGE STITCHING

23

1. An algorithm for image stitching and blending Vladan Rankov , Rosalind J. Locke 2. An Improved Algorithm on Image Stitching based on SIFT features Jing Xing, Zhenjiang Miao 3. Image Stitching -Comparisons and New Techniques Chia-Yen Chen 4. Distinctive Image Features from Scale-Invariant Key points David G. Lowe 5. Panoramic Image Mosaics Heung-Yeung Shum and Richard Szeliski

Mount zion college of engineering