In this problem we will develop an algorithm for stitching a panorama from overlapping
photos (Figure 1), which amounts to estimating a transformation that aligns one image to
another. To do this, we will compute ORB features1 in both images and match them to obtain
correspondences. We will then estimate a homography from these correspondences, and we’ll
use it to stitch the two images together in a common coordinate system.

In order to get an accurate transformation, we will need many accurate feature matches.

Unfortunately, feature matching is a noisy process: even if two image patches (and their ORB
descriptors) look alike, they may not be an actual match.

To make our algorithm robust to matching errors, we will use RANSAC, a method for
estimating a parametric model from noisy observations. We will detect keypoints and represent
descriptors using ORB. We will then match features, using heuristics to remove bad matches.

We have provided you with two images (Figure 1) that you’ll use to create a panorama.

(a) You will start by computing features and image correspondences.

• (2 points) Implement get orb features(img) to compute orb features for both
of the given image.

Figure 1: Panorama produced using our implementation. The image pair shown on the left represents
the keypoints in the two source images and below them are the predicted feature correspondences. On
the right is the stitched panorama.

• (2 points) Implement match keypoints(desc1, desc2) to compute keypoint
correspondences between the two source images using the ratio test. Run the
plotting code to visualize the detected features and resulting correspondences.

(b) (2 points) Write a function find homography(pts1, pts2) that takes in two N × 2
matrices with the x and y coordinates of matching 2D points in the two images and
computes the 3 × 3 homography H that maps pts1 to pts2. You can implement this
function using nonlinear least squares (or, alternatively, the direct linear transform).

Hint: For nonlinear least squares, we reccomend using scipy library’s built-in nonlinear
least squares. To use that function, you need to:

• Define a cost function, f(h; pts1; pts2), that calculates the projection error (a
vector of length 2N) between pts1 and projected pts2 using homography H
as pts1 − cart(H ∗ homog(pts2)). Here homog(x) converts x into homogeneous
coordinates, cart(x) converts x to cartesian coordinates and h is the flattened
version of the homography H to be estimated.2

• Provide an initial guess for the homography h: a length 9 vector filled with ‘1’s
should be good enough.

(c) (2 points) Your homography-fitting function from (b) will only work well if there are no
mismatched features. To make it more robust, implement a function transform ransac(pts1,
pts2) that fits a homography using RANSAC. You can call find homography(pts1,
pts2) inside the inner loop of RANSAC. You will also be responsible for figuring out
the set of parameters to use to produce the best results.

(d) (2 points) Write a function panoramic stitching(img1, img2) that produces a
panorama from a pair of overlapping images using your functions from the previous
parts. Run the algorithm on the two images provided. You result should be similar to
that of Figure 1.