[chupathingie]

oooooohhhhhh….. me likey that last one especially!

[chupathingie]

You have discussed somewhat your post procedures and software but do you mind elaborating? I seem to remember that you use ImageMagick for some things.

Oh I suppose I could do that…this might get a wee bit wordy, just because I’m a big fan of the whys in addition to the hows; plus I’d like for anyone else who stumbles into this thread to have as much information in one place as I can pass on.
/me grabs a cup of coffee and Bailey’s

I’m sure everyone here knows what a long exposure (2mins+) looks like from a DSLR…very grainy, noisy and with scattered sprinkles of saturated red, green, blue and white pixels (“hot pixels”). It’s not pretty.

The basic premise here (and this is likely the only original thought I have on this, all the rest is information I’ve gathered reading forums and the writings of those more learned than I) is to create as good an image as you can of what you don’t want in your final image, and use that to clean up your data. “Data” in this sense is the subject of your composition, and expressed in basic math any image you take looks like this:

Image=data+noise

So the idea is to gather as much of the data in one pile, and noise in another. In-camera noise reduction does this in a simple fashion; you take your picture and the camera then takes an identical exposure with the shutter closed. Rearranging the above formula gives you:

data=image-noise

and this is exactly what in-camera noise reduction does. It subtracts the image taken with the shutter closed (a “darkframe”) from the normal image (a “lightframe”). This eliminates the hot pixels and cuts down on some of the grainy static in the final image.

It doesn’t eliminate the static, however. Some of the static is what’s called thermal noise, and it’s random, so the darkframe will only share some of that thermal noise with the lightframe. The truly random thermal noise gets dealt with almost accidentally, and I’ll explain that shortly.

The rest of the static in the image is not completely random, but due rather to imperfections in the camera sensor that makes some pixels more or less prone to thermal noise. The thermal noise is random, but the level of thermal noise at a given pixel may be higher or lower, on average, than its neighboring pixel. This is repeatable at any given temperature, so it’s something you can capture in a dark frame and use to your advantage.

OK, so here’s a quick run-down on gathering the raw data:

Shoot your lightframes, all at the same ISO, f-ratio (if applicable) and shutter speed. Shoot as many of these as you can get away with. The combined exposure time for them should be at least an hour, but two hours or more is better. Temperature is a factor, and it’s rarely stable…it will change during the duration of your shoot. In the middle of your shoot, stop and put the lens cap on (don’t forget to cover the viewfinder any time you’re shooting astrophotos, BTW) and shoot at least 20 darkframes at the same settings as your lightframes. When this is done, remove the lens cap and continue with the second half of your lightframes. Doing them this way will help ensure that your darkframes are taken at a temperature that falls somewhere in the middle of the temperature range of your lightframes. Shoot all of your images in RAW…you need the extra color precision with astrophotography, it’s simply a must.

There is more to noise than just thermal noise and hot pixels…any flaw in the data collection path is considered noise, and includes dust on the sensor, vignetting, sky glow, light pollution, sensor read noise, etc. Everything from sky conditions to the technological limitations of your hardware. If you can isolate it and get an image of it, it can be used to clean up your data. I’m only covering thermal noise here, because I feel like I know enough about it to be useful. I’m not much help with shooting and using “flats” yet. Flats are images taken out-of-focus or of a uniform, evenly-lit field of view and are used to counter some of the other noise sources.

Post processing:

The first thing to do here is to convert all of your files to a 48 bit color format (often called 16 bit color, which refers to the 16 bits in each color channel). I convert to TIFF, but any lossless format that can handle the extra color depth works equally well. There are plenty of software titles available to do all of the post processing, but I’m a bit of a minimalist (were I an artist, I’d be the guy mixing his own paints) and running an open-source home photo-lab so my choices may seem limited. I use ImageMagick for the major steps, but whatever you use should be capable of working in 48bit color space. “Normal” 24 bit color is simply inadequate for astrophotographic (as well as advanced “normal” photographic) processing.

I start by separating my lights and darks into different folders. I then open up a terminal window (or a CMD window in a Microsoft OS) and navigate to the folder where I’ve put my darks. Next, I run an ImageMagick command; convert -average *.tif darkframe.tif . This gives me a final image called darkframe.tif that is composed of a straight-up average of all of the color values for each pixel. What this does is filter out the truly random thermal noise leaving behind the repeatable noise… the hot pixels and those pixels prone to higher and lower levels of thermal noise.

Next I apply darkframe.tif to all of the lights in that other folder, again using an ImageMagick command; composite -compose minus xxxx.tif darkframe.tif dsxxxx.tif, where “xxxx” is the filename for each lightframe. This is very tedious, especially if you have, say, 75 lightframes (I’ve posted a script with notes here that does all of the post-processing described here with a single command). This creates a new set of lights with “ds” (for “dark subtracted”) added to the start of each filename that have the repeatable noise removed. It is crucial that this darkframe subtraction be done before any other processing is performed on the lightframes. If it is not the first step, you will actually increase the noise in the images.

Next I align the all the lightframes so that the images will lie precisely on top of each other for stacking. I use one of the commands included with Hugin’s Panorama Creator and it looks like this in my workflow: align_image_stack -a ais ds*.tif . It takes all of the .tif’s that begin with “ds” and shifts them as needed, creating a new set of files that now begin with “aisds”. If you’re using an auto-guider and are confident that all of your frames are aligned precisely with your target, this step is not needed.

Here’s where the truly random noise gets dealt with, by using the same command we used to average the darkframes: convert -average aisds*.tif final_lightframe.tif. As stated above, this filters out the truly random noise component, and is the main reason behind shooting a large number of lightframes. Remove “ais” from this command if you’ve skipped the previous step.

What you’re left with is an unadjusted lightframe that has had the vast majority of static and noise removed which you can then adjust levels on to bring out the wispy details. You can stretch levels and curves on this image much more aggressively before the remaining noise starts to show up.

A lot of this applies to regular photos. You won’t get the benefit of stacked lightframes, but images shot in low light or with dark shadows will benefit from simply putting the lens cap on and shooting 1/2 dozen darks right afterwards and subtracting their average from the original…shadows or dim areas will be quite a bit cleaner without the blurring that goes along with traditional noise reduction. I have not done a side-by-side comparison, but I suspect this might be a cleaner method than in-camera noise reduction due to the filtering of the random noise in the darks, and not introducing a second set of random noise data into the processing.

[chupathingie]

Yeah, rav‘s got the gist of it there.

The best way to avoid the CA is to shoot prime with a T-adapter thru a Newt…that eliminates CA entirely, but trades it for coma in fast systems. Stopping down as rav mentioned helps quite a bit, at the expense of longer exposures; you have to make a trade-off between the accuracy of your guiding and how much CA you can deal with. Avoid attaching filters to your lens, they will produce flares.

You’ve got me wondering just how low I can drop the R/G/B sensitivity with camera settings; that would eliminate the filter issue if it can be dropped to 0.

I know there is software to help deal with CA in post, but I’ve not tried any so I can’t make a recommendation.

As far as camera choice goes, any DSLR can be used… most any of them work surprisingly well. All the noise reduction is done in post and is determined more by technique than by the camera unless you’re shooting way up at the highest ISOs where things wind up being less repeatable/predictable.

[chupathingie]

yikes… how did they get it in there (not entirely sure I really wanna know…). My first thought about the TSA was that your doctors are gonna remove that after it sets, right? then I saw the pic…

[chupathingie]

heh… my wife wouldn’t let me within 6 feet of her with that bottle brush on my face! LOL

[chupathingie]

Jerry Lodriguss is one of the folks responsible for the popularity of digital astrophotography (I think I’ve linked to some of his stuff here before)… he’s written a huge volume of great, practical info on the topic and was one of the first people who’s writings got me moving in the right direction a few years ago when I was just getting started. Great stuff.

[chupathingie]

Gotta say this is a tough one for patient immobility… any upper body movement will stress that break.

I don’t envy you, regardless of the painkiller bliss.

[chupathingie]

That “otherwise occupied” feelling is a also good part of why I enjoy long road trips.

That’s a good way to put that. most people dread the idea of 12+ hours x-country. I do them solo, since my wife is one of the above dreading types. I’d much rather have a carful or a damn convoy LOL

[chupathingie]

Years ago, I had surgery on my throat. My doc prescribed a diet of slim-fast and sent me home with a 16oz bottle of liquid codeine solution. I supplemented that diet with beer. I’d dose for the pain, and after the first day I doubled the dose. I was still hurting. Then I tripled it. Eventually, I skipped dosing altogether and just turned the bottle up every few hours and swallowed a full mouth of it. I still hurt, but I didn’t care anymore. I stayed at that level until the bottle went dry, then had to explain to the doc why I’d gone through what was intended to last 3 weeks in about 5 days. I told him that no matter how much I took, I still hurt, and that the dosage that gave me relief was basically “drink until blotto”. I also told him that the high for the first 2-3 days was awesome, but that now I could hardly stand the mental fog anymore but it was better than the pain.

He wrote me 2 more refills, which I killed off at about the same pace as the first. I’m not cut out to be an opiate junkie, but I think I’m a minority. That stuff would drive me unpleasantly insane with extended use.

[chupathingie]

Here’s hoping it heals fast, and that the pain pills make it all bearable!

[chupathingie]

I really miss Fry’s. Since I’ve moved out to Rooster Poot I’ve had to order all of my build materials from Newegg. It’s cheaper, but I have to wait for shipping. I can’t wait to do another build, the workhorse is now about 2 years old, and 16G of ram ran me $500 back when it was built; and motherboards that newegg carried maxed out at 16. I could use 32 or more easily; panoramas pushing 1/2 gigapixel and up chew through unholy amounts of ram during processing.

I’m still going to keep plugging away with my dirt cheap lappy ($500) with it’s 4G of ram for as long as I can get away with it, but it’s just a matter of time before I need to process a big image on the road for local printing and then regret that decision.

[chupathingie]

This is, indeed, Neat Stuff.

/don’t google “wet plate iso” and fall into the well…

[chupathingie]

That’s cool… I’ve never tried to get shots of asteroids. Thought about composing the 2 nights’ shoots into an animated gif as sort of a blink/compare?

[chupathingie]

Oblig echo of the above sentiments (ouch!)

In the future, if you get ribbed here over being a gearhead pointedly, it’s only because you posted about saving your gear and only mentioned the painful broken freekin’ bone in the header. 8)

Beer, Spongebob and whatever painkillers you can wheedle to you…

Heal comfortably.

[chupathingie]

Isn’t that B&W? And could you look at it under a darkroom light if so?

edit:skip that, stupid question… red light’s only safe for prints…

[Author]

Posts