Project 1

Russian photographer Prokudin-Gorskii created colored photographs with RGB filters to create three color channel images. In this project, we'll try to put these three together to reproduce a fully colored image. Here is one such set of images:

Cathedral

Some of the challenges are to align the images automatically in order to produce a coherent photo. The metric that I used to align the images was Euclidean Distance. I computed the difference of the images' flattened vectors and its L2 norm, assigning a higher similarity score to lower values. For the smaller photos, I did an exhaustive search over a space of displacements, from (x, y) = [-15, -15] pixels to [15, 15] pixels. These are the results I achieved for the smaller images. I also trimmed out the borders from the images.

Cathedral, Offset={r: [5, 2], g: [12, 3]}

Monastery, Offset={r: [-3, 2], g: [3, 2]}

Tobolsk, Offset={r: [3, 3], g: [6, 3]}

However, for larger images, offsets may be way farther than 15 pixels, so the result will be of poor quality. It also becomes computationally infeasible to brute-force a search in a correspondingly larger space due to the increased size of the image. The way this can be solved is by doing an image pyramid.

Partial bells and whistles: If you just run the above algorithm with raw pixel values on Emir, you'll get this:

Bad Emir, Offset={r: [49, 24], g: [93, -635]}

A better metric is to use vertical edge detection with a convolution operator. I used the Sobel filter (Source: Ed) to emphasize vertical edges present in the image. Here's a better one below.

Emir, Offset={r: [49, 24], g: [105, 41]}	Church, Offset={r: [25, 3], g: [58, -5]}	Harvesters, Offset={r: [60, 16], g: [123, 13]}
Icon, Offset={r: [40, 16], g: [89, 23]}	Lady, Offset={r: [57, 8], g: [120, 12]}	Melons, Offset={r: [81, 10], g: [178, 13]}
Onion church, Offset={r: [52, 23], g: [108, 36]}	Sculpture, Offset={r: [33, -11], g: [140, -27]}	Self portrait, Offset={r: [78, 29], g: [176, 37]}
Three generations, Offset={r: [55, 12], g: [113, 11]}	Train, Offset={r: [41, -1], g: [85, 29]}

Some more examples from the Prokudin-Gorskii collection:

Extended example 1, Offset={r: [26, 18], g: [122, 34]}

Extended example 2, Offset={r: [36, -7], g: [122, -17]}

Extended example 3, Offset={r: [30, -7], g: [111, -14]}

Additional bells and whistles: Here's an automatic contrasting that was performed. I opted for the Gamma correction technique instead of a linear transformation, due to the fact that our perception of light is different from camera lenses. In particular, images appeared to de-emphasize the darker tones. I corrected this with a positive gamma value across all images. Here are some results:

Harvesters original	Harvesters contrast
Extended example 3 original	Extended example 3 contrast

As you can see, it noticeably improves the quality of certain images. Harvesters does not have the unsightly glare. Also, for the second image, you can observe more details on the surface of the stones and the wooden structure in the background.

One interesting one is Icon, revisited. I performed Gamma correction on each color channel, so the relationship between different colors must not have been preserved through this transformation. Take a look at the following.

Icon original

Icon contrast

You can see that the new image appears to have a blue filter over it. You can actually see this by printing the average pixel intensity in the blue channel:
0.35710825523766937
I deliberately tuned the algorithm to intensity low-intensity channels. Since this image has a lack of "blue" (most colors tend to be red and yellow), it will actually appear more "blue" due to the correction.