Problem with Flats

CarlightExpress wrote: Sure is an interesting subject, and I still don't know what my optimal ADU is for flats

Ah yes, flats, that's where we started!

I can't find anywhere that explains the reasoning behind the '1/3 - 1/2' of total ADU advice.

Flats get divided into the lights so the actual values shouldn't really matter (processing software will handle the details!). They should have high signal to noise so you'd think longer is better; too short (too low ADU) and you'll have noise, but too long and the flat will get saturated. So I'd have thought that ideal flats would be ones where the highest pixel value was near but within the upper linearity limit of the CCD.

That said I just use around 25k ADU Seems to work ok!

Albert

The subject came about because what the manufacturers say is "Full Well" is not necessarily the full well capacity of the CCD, so doing all this will determine where the full well is of my CCD

I was doing mine at 27k ADU

Simon

To get "optimal" flats exposure implies going to the highest exposure before hitting the full well capacity.
To be more accurate - it implies the highest exposure before we hit the non-linear region which before we hit full-well.
In the case of CCDs with anti-blooming gates (and I think your CCD, Simon, has ABG) electrons are siphoned off before they over-spill into neighbouring pixels on the same column. This inevitably means there is a range of exposures before FWC where the responsiveness is non-linear. - and no good for flats.
Also - not all pixels are equally responsive and some pixels can reach FWC before others.
To determine the optimum exposure for flats would be very difficult to determine.

I suspect (and I'd like to research this) that in the science community they would start off with the criteria of what signal-to-noise ratio is necessary for the data they wish to capture and then choose flats which are of no less an SNR - for choosing flats of poorer SNR would degrade the quality of the science data and defeat the purpose of using them.

I'd determine where linearity ends (which could be FWC) in a representative region of interest - or whole chip - and cut-back by some conservative safety margin.
But unless one is doing quantitative work where every achievable decimal place should be savoured I'd go something like 1/2 full scale - if you can no longer see a dust doughnut or vignetting after flat-fielding then job done. I'd like to see a purveyor of aesthetic astrophotography contribute any comment on that one.
I think at end of day flat-fielding shouldn't degrade your work regardless of purpose of the image. Cost / benefit sort of thing - more good should be done than harm.

Mark C.

Reading Janesick (re: PTC) - there's a couple of things that immediately strike me.

I've never been clear on whether it is necessary to do anything special about dark-current (ignoring DTC for now).

Janesick - without talking to me as if I were a 10 year-old (which I need right now) seems to suggest (after several readings) that the exposures taken from zero seconds to that which exposes full well capacity - are all left on the CCD for an equal amount of time (which he calls a "fixed integration period") before downloading - regardless of the actual exposure.

I have not seen a reference to the zero-second exposure being used as a dark-frame for dark-subtraction purposes - esp in the explanatory equations - but it is called a dark-frame in the text (it is zero-second exposure to light - but left on the CCD for the integration period).

Now I can't do that with my CCD as once exposure is complete it is automatically downloaded.

Now Janesick also suggests using a an LED as the light source - of which the switching on and off serves as the timing mechanism of the exposure which is especially better at the very short exposures for which a mechanical shutter (and I assume inter-line and frame transfer CCDs) cannot do as well.

While I've taken this on board - I can't say that I am in full understanding as yet.

Also pointed out, there is (or can be) a region before full well capacity is reached where differences in pixel-to-pixel responsiveness affects the curve and he proposes a technique (possibly reflected in the spreadsheet in earlier part of our discussions) whereby every exposure is duplicated (we have pairs of exposures) and a little bit of pixel arithmetic can be done on each image pair so that we eliminate this artefact that affects the linearity in that part of the PTC from which we estimate the gain (by extrapolating down to the x-axis). I'm still trying to digest the equation explaining this one.

I want to (if I can) understand what Janesick says of PTC's before I revisit the spreadsheet.

Mark C.

I've produced a PTC curve for my Atik 16IC CCD as best as I understand Janesick.
It appears to be a beautiful classic curve and for which I get a gain only slightly different to when I used simpler methods a few years ago.

One thing that I wanted to do was use the data to numerically calculate the gain and readout noise (through numerically calculating x and y axis intercepts) rather than using "eyeballing" of the visual chart - or placing graphical lines to determine intersects visually.

I've done this with with gain - but not with readout noise - that had to be eyeballed.
What I've done with gain is to use a linear trend line (from equation produced by the spreadsheet graphing feature) - but suspect that this isn't quite right (for a subtle statistical reason best left unsaid for now - but very compelling and I'd say inconsequential).

My gain is very much in the ball park of where I'd expect it to be - but the readout noise is slightly less than half of what I'd expect and so I'm suspicious and want to research more.

Full-well capacity. My Chip has an anti-blooming gate - it came with the device which came at a nice consumer price - given the choice (and budget) I'd choose not to have ABG.
It's easy to select a point below which I should be confident of linearity - using this data. However, when visually inspecting the image in the region of interest (ROI) used to perform my calculations I notice a gradient (downwards in ADUs) near one edge of the frame.
My initial interpretation is that the ABG is affecting read out with relatively high ADU count ROIs - but with mean ADU not yet at what I would have considered full-well capacity.
I need to research that more - but the interpretation of FWC might have to be considered differently for CCDs with ABG (or I have a dicky CCD).

Another thing that I have observed with my CCD (and I saw this when I last attempted characterisation a few years ago) is that the first image (often / always) taken after power on (regardless of time from power-up to exposure and download) has an abnormally high stdev (as observed on bias images).

There is still more that I'd like to do and that is that when determining certain characteristics like gain, readout noise, full well capacity, etc is when we come up with a number - how can we say something to the effect of "confidence is high" (okay that needs clarification) that this is good to 2, 3 - or what ever decimal places or g = 0.222 +/- 0.002.

More to come - but there will be periods of apparent inactivity from me.
It would be nice if we had a documented prescriptive method and guide to analysis with explanations. Maybe something like that will come out of this.

I have learned more than I have documented here - but I also have more questions that I equally have not documented.

Mark C.

Could you share your PTC