Panorama stitching

how do we stitch images from diff viewpoints?

• image homographies

we can calculate homographies when

• the scene is captured under rotation only

homography matrix

• can be written as $A_{i}h=0$??

• gives 2 rows per matching keypoint

  • homogeneous linear least squares problem???

• solve with singular value decomposition (SVD)

  • h params are the eigenvector in V associated w smallest eigenvalue in sigma
  • $h=v_i$

Create a panorama

1. find keypoints with SIFT - local detector or Harris Corner Detector
2. Find matches with local descriptor , matching keypoints
3. Sample N keypoints
   1. put samples in matrix form Ah = 0
   2. use singular value decomposition (SVD) to solve for homography matrix
   3. calculate inliners (reprojection error + threshold)
   4. repeat
4. Re-calculate h using inliers from best homography

we are doing projective transformations, not affine

challenges solved by step 3

• noisy keypoints give bad transformation
• solution: RANSAC

---

use cases

• google street view
• other stitching software

accomplished tasks

• detect and match keypoints from many images to build one panoramic image
  • Harris Corner Detector finds keypoints
  • Build local descriptor and compare two sets of descriptors from diff images to find matching keypoints
  • use least-squares to find affine transformation matrix that maps points in one image to another
  • use RANSAC to get better estimate of affine transformation matrix
    • transform second image and overlay on the first
  • also, try different method HOG