Zds Project Log

This tutorial goes through how to create HDR panoramic images using Free and Open Source software and Photoshop.

First, lets go through what this means:

HDR image is an image with high dynamic range. “High” usually means “higher than regular cameras can capture and displays display”.

Panoramic image, in this tutorial, means image is constructed from several images taken from same point but with different angles, thus enabling us to simulate wider field of view than the camera lens would otherwise allow. It can also be used to create higher resolution images than your camera sensor can capture in a single frame.

Tonemapping means manipulating an image in a way that increases local contrast and decreases global contrast. This meaning is up to a debate, but in this tutorial tonemapping means different algorithms that take HDR image as input and output LDR (8-bit) image in purpose of making HDR image viewable on LDR monitor.

The tools I use:
-Canon PowerShot A620
-chdk running in my Powershot A620, enabling me to save raw sensor data, see sensor temperature and to vary exposure continuous shooting modes.
-dcraw, mogrify and exiftool and two BASH scripts to command them, to clean up the raw data and for a TIFF out of the result.
-Qtpfsgui 1.8.12 to combine the varied exposure shots in each HDR stack to single HDR image.
-If covering larger area than a single frame does, Hugin to combine number of HDR stacks or single source images to a single image, often called panorama.
-Qtpfsgui 1.9.2 to tonemap the resulting HDR or the HDR panorama created by Hugin.
-Photoshop to convert EXR to 16-bit (per channel) TIFFs and to post-process.

In the example I use in this tutorial I shot 14 images, 2 HDR frames each consisting of 7 shots. In this tutorial “HDR frame” stands for set of images taken from exactly the same location, usually with varying exposures and meant to be combined to a single high-dynamic frame.

I strongly recommend that once you start experimenting yourself, first shoot a simple and trivial case of source images. The process has rather many details that can cause frustration with incomplete source data, and it’s better to first see how things work and then go experimenting farther. A good starting point might be to take images from your bookshelf from about 2-3 meters away with as long focal length as your camera easily supports. This way you get lots of local details and less panoramic imperfections than when going close and personal. There’s not much high dynamic range to speak of there, but as said, start with something simple to get the feel of the software and the process.

I have found a good practice to split HDR panoramic images into logical directories: parts of the same final image into one directory and parts of each HDR frame to their own directories under that one. In my work flow there’s three files per each shot, so this kind of separation lessens unnecessary confusion. In the example case I have at this point 28 files, 2*7 (JPEG+raw) for both frames, located in hdr169/frame0 and hdr169/frame1.

I started post-processing by converting raw-images to TIFFs. For each image I have JPEG and CRW file and a BASH script that uses these and temperature I give to produce TIFF file. The script further calls dcraw, mogrify and exiftool. dcraw takes the raw file (CRW), dark frame and dead pixel list as input. Mogrify is used to compress the TIFF file and exiftool to copy the EXIF data from JPEG to the resulting TIFF. As the end result I have TIFFs with correct EXIF data, compressed data block and without deterministic noise of the sensor. I use TIFF here because it can be compressed somewhat and Qtpfsgui can read it, PNG it cannot.

Then I look through the source images, especially the histograms, to determine which of the 7 source images to use. I have found it a good habit to keep the covered EV range the same in all HDR frames of a panoramic image. Hugin can do some exposure correction to the images, but you ask for trouble by deliberately giving it in-consistently exposed images. So in the dark end I find the first frame that has no signs of overexposed pixels and leave out any darker frames. In this particular case IMG_8089 is the last one having no overexposed pixels and IMG_8087 is the darkest still having them. So anything darker than them, IMG_9091 in this case, is not used.

Here you see the histograms for the two darkest images I picked:

In the lighter end it gets more tricky. As I use compact camera, the optics and sensor bleed throught in bright areas, so I have to walk the line between noise on dark areas and washed out dark areas near brighter areas. It’s a judgement call in the end. Sometimes I even compose HDR frames with different source image set, run them through Hugin and tonemapping with 1:1 the same settings and then hand-pick with masks in Photoshop which parts of them to use. In this case I decided to drop just the brightest source image (IMG_8090 and IMG_8097 in this panoramic set) of each HDR frame.

After this I mark down which images I picked so that I can later delete the reproducable images and be able to re-compose them if I happen to want so. In HDR panorama imaging the size of the intermediate files soons becomes a real concern, so I want to delete all images I can easily reproduce. Some of my images have taken more than 6 gigabytes worth space with all the intermediate version counted and I really do not want to store all that.

After composing the HDR frames in Qtpfsgui 1.8.12 I now have files hdr169_frame0_0a.exr and hdr169_frame1_0a.exr in directory hdr169. I use two version of Qtpfsgui since because they have different bugs in them. 1.9.2 does better job at tonemapping, while 1.8.12 can save HDR stacks without crashing.

Next I open both HDR frames in Photoshop and convert them from EXR to 16-bits per pixel PNG. In this conversion I first find an exposure setting where no pixel is left overexposed and then tune up the gamma so that most parts of the image are clearly visible. The latter is necessary so that I can hand-tune control points in Hugin and still see what I am doing. If the result is not what you wanted, you can overwrite the now created TIFFs with other ones and just re-run Hugin stitching process with them once you are fine with the panoramic composition results. I use PNG here because it gives me the best compression ratio and Hugin can read it PNG.

I also write the exposure and gamma settings to a settings log file so that I can delete the tiffs later on. At this point my settings file is ready, and it looks like this:


Temp: 5C
EXR -> PNG: -12 0,8
XXXXX--


With this information I can reproduce the HDR-frame-PNGs I now have. The last row just marks which source images were used in each frame; first five were used, two last (the most over- and under-exposed ones) not. Up to this point it was mostly routine stuff and this files lists all the decisions I had to make myself.

Now we get into Hugin. If you have never used Hugin before, you first need to acquire lens data for your particular camera and lens. Open any JPEG or similar with EXIF data in place with Hugin go to “Camera and Lens” tab and pick “Save lens”. Now you can get back to the HDR frames. I pick new project and load the HDR frame PNGs to Hugin. I am asked about lens parameters, so I offer the previously saved file and possibly change the focal length to match the EXIF data. Yes, I could write a script that copies the parameters that make sense from some of the source images and writes it to the Hugin input files. No, I haven’t felt like writing it yet. Also in my case the RAW data contains a bit more pixels than JPEG, so I have wanted to hand-adjust the field of view to match that.

With the files now loaded to Hugin and camera and lens parameters set, I start by letting autopano-SIFT-something to automatically guess me control points. Press “Create control points” in “Images” tab to do this. Then, depending of the source images and your luck, you are either almost done or just beginning. With the example images I have here the “Optimiser” “Optimize: Everything” warns about possibly invalid results. Hand-adjusting, here I come.

Here’s my example image straight from autopano-SIFT-foo:

As you can see in the preview, the border where images overlap is mismatched.

If you happened to get good results right away, skip the next couple of paragraphs and go on with stitching. Good results mean the image looks sane in the preview and often mean error is <1 and biggest error is <5. Sometimes you just cannot get them due to inaccurasies in the capture, and the closer to the subject you were, the harder it is to get good results. Also sometimes some parts of the scene, like clouds, move during the capture, the tripod is not 100% stable etc etc. You’ll learn over time how to minimize these problems, but still you are almost always working with imperfect data.

Work with preview, control points and optimizer until the image looks good in the preview. You do not need perfect match, just close enough enblend can hide the small imperfections. In fact, you usually even cannot get perfect match. To get them, you’d need to rotate camera only around the focal point of the camera and lens, and to do that, you need a special tripod attachment. So run optimizer look through the control points autopano guessed for you and see which of them are good and which are not. Delete the points that do not look fine to you, add points to areas where you see mismatch in the preview. Remember you need to prioritize; the in-focus main subjects are most important to get to line up, the blurry background matters less. Keep adding control points until the preview looks fine. Keep eye on the per-points error you see in the Control points view after each optimization round.

Here’s my example image after I have removed and added control points:

It’s quite good, but there’s still one annoying mismatch, circled in red.

After concentrating a bit to the problematic area, here’s the end result:

Before stitching, save. Save early, save often. You can then try to optimize the exposure in Hugin. Sometimes it can guess vignetting nicely and not screw up the exposure values. Sometimes not. If it failed, load the previously saved version from disc. You cannot reset the exposure settings in the program. If you got bad results and saved, the .pto file is just text, open it and edit it to reset the settings. Before stitching, click and right-click in the preview to get the image at proper place and to rotate it properly. Then hit “Calculate Optimal Size” in the “Stitching” tab and “Stitch”. Change TIFF compression to LZW, if you want smaller file size.

Then if the result has problems, iterate control points, optimization and stitching. In the example case I cannot see any problems, so I am happy with it.

Here is the image as Hugin outputted it (except in 8 bits):

Now I crop the image to largest possible size I can need, ie. cut the black edges and borders of the image where it’s not possible ever to crop a meaningful image that would include them. I save that as TIFF and go to Qtpfsgui 1.9.2.

In Qtpfsgui, “Open Hdr” and pick the panoramic result file. Then “Tonemap the Hdr”. I create tonemapped versions with Mantiuk, Mantiuk+Contras Equalization and Reinhard05. For Reinhard you need to adjust the brightness; the default value gives too light results with 16-bit images.

Below are Mantiuk and Reinhard05 versions; in this particular image Mantiuk+CE was not useful so I dropped it.

Now it’s time to let the artist loose - open the tonemapped versions and the 16-bit versions to your image editor of choise and start combining the layers. Also: run pngcrush to the PNGs created by Qtpfsgui. Their level of compression is sub-par. Here is a BASH spell to compress all PNGs modified in past 2 days, in the working directory and below it:


for f in `find . -name "*.png" -ctime -2`; do if pngcrush $f ${f/.png/_.png}; then mv ${f/.png/_.png} $f; fi; done


Here is, approximately, what I did myself in Photoshop:
-Curves to the 16-bit version to brighten image without losing all contrast
-Mantiuk tonemapped version on top of it with filter “Luminosity”
-Reinhard05 tonemapped version on top of both with filter “Soft Light” and opacity of 27%.
-Copy merged, paste as topmost layer
-Auto Colors
-Shadows+Midtone Contrast, 50% transparent

A tip: When you start cropping the image, save the selection you used to do the crop and then continue editing in a new file based on that cropping. This way you can go back to layered and non-cropped image and alter things and still preserve the same cropping.

Here is the final image I got out of this image set:

Feedback can be sent to zds at domain iki.fi.

Appendix

decoderaw.sh, decodes raw images from my Powershot A620 using dcraw:


#!/bin/bash

# Decodes a single raw file given as parameter.

jpeg_name=${1/CRW/JPG};
iso=`exiftool -ISO -s -s -s $jpeg_name`;
exposure_time=`exiftool -s -s -s -ExposureTime $jpeg_name`;
exposure_time=${exposure_time/\//-}

if [ -z $2 ]
then
temperature=”";
else
temperature=”_”$2″C”;
fi

dark_file=”/share/zds/empty_frames/dark_iso”${iso}”_”${exposure_time}”s”${temperature}”.pgm”
echo Decocing $1 with temperature ${temperature}
dcraw -K $dark_file -4 -T -o 1 $1
mogrify -compress lzw ${1/CRW/tiff}
exiftool -q -tagsfromfile ${jpeg_name} -overwrite_original ${1/CRW/tiff}


How I call it (temperature set to +5C in this run):


for f in `find . -name "IMG_*.CRW"`; do ~/decoderaw_.sh $f 5; done


rename_dark_frames.sh, converts dark frames to pgm and renames based on their EXIF data.


zds@terra:~$ cat ~/rename_dark_frames_.sh
#!/bin/bash

# Decodes a single raw file given as parameter.

jpeg_name=${1/CRW/JPG};
iso=`exiftool -ISO -s -s -s $jpeg_name`;
exposure_time=`exiftool -s -s -s -ExposureTime $jpeg_name`;
exposure_time=${exposure_time/\//-}

if [ -z $2 ]
then
echo “No temperature specified, reverting to 25″;
temperature=”";
else
temperature=”_”$2″C”;
echo “Using temperature “$temperature;
fi

filename=”dark_iso”${iso}”_”${exposure_time}”s”${temperature}”.pgm”;
echo Creating $filename
dcraw -D -4 -j -t 0 $1
mv ${1/CRW/pgm} $filename