Images of the Russian Empire

Project Overview

In the early 1900s, Sergei Mikhailovich Prokudin-Gorskii was convinced that color photography would be the future. With that he traveled throughout the Russian empire taking photos using red, green, and blue filters. He envisioned that one day some sort of projector would be able to overlay them and produce a color image of the photos he took. The goal of this project is to "colorize" those photos.

Basic Approach with exhaustive search

The basic approach involves exhaustively trying alignments in the range of [-15, 15] and taking the alignment with the best score. The score can be calculated using L2 norm or Normalized Cross-Correlation (NCC). You can see from the below images that I found L2 norm to give better results. I additionally tried using a mutual information function to find alignment. This gave exceptional results but took quite a bit longer.

Tobolsk L2, green offset: (-2, -3), red offset: (-3, -6)

Tobolsk, green offset: (-2, -3), red offset: (-3, -6)

Monastery, green offset: (0, 6), red offset: (-1, -9)

Monastery L2, green offset: (0, 6), red offset: (-1, -9)

Monastery, green offset: (0, 6), red offset: (-1, -9)

Monastery NCC, green offset: (13, -15), red offset: (12, -15)

Monastery mutual info, green offset: (-2, 3), red offset: (-2, -3)

Monastery mutual info, green offset: (-2, 3), red offset: (-2, -3)

Approach with pyramid search

The basic approach works for smaller images, but for larger images such as the .tif images, exhaustive search will take too long. To resolve this, I made use of an image pyramid which represents an image at multiple scales and aligns at each scale going from coarse to fine and updating the estimate along the way.

Pyramid Search Illustration

Pyramid search visualization

Emir.tif

Emir.tif

Icon.tif

Icon.tif

Three_generations.tif

Three_generations.tif

Onion_church.tif

Onion_church.tif

Bells & Whistles

For Bells & Whistles I implemented autocropping and autocontrast. For autocropping I detected the edge/borader of the R, G, and B input images and removed rows with pizels close to 0 or 1. The cropping struggled to identify the dark boarders but was veyr good at removing the light boarders. For autocontrast, I used a simple rescale of pixel intesities making the darkest and lightest pixels in the image zero and one respectively.

Monastery without cropping or contrast.

Monastery without cropping or contrast.

Monastery with cropping and contrast.

Monastery with cropping and contrast.