Problem with Flats

I've taken a step back for a while and have been trying to get to grips with Jansesick (Scientific Charge-Coupled Devices) on this matter.
I've learned that I was misinterpreting the spreadsheet - still not comfortable with it - but am getting closer to understanding it.
I believe Crisp has an understanding that I need to rise to.

As with Albert - I've only approached the subject from a few simpler metrics that did not require either a PTC or DTC proper (I hope I'm not misrepresenting Albert).
When I have recently tried to do a PTC and DTC on my own camera I didn't get success - but this is my failure.

I think I'm going to have to get a copy of the Handbook of CCD Astronomy as it is more concise and directly applicable than maybe Janesick - but Janesick really is the ultimate reference on CCDs - but not for the faint hearted.

With regards to BIAS frames versus overscan.
Overscan is included in every frame (when such functionality is available but not common with many consumer amateur devices/software) one has a more immediate estimate of the "offset" (as Janesick and the spreadsheet refers to).

Thus far I have gleamed that fixed-pattern changes in bias should not affect, significantly a PTC (and by implication the DTC) so for purposes of PTC - it doesn't really matter which estimate of "offset" you use.

Jansesick states overscan is a better measure of the offset because it is not corrupted by things like dark current and other sources of noise - unlike bias frames.
(a paraphrase - but he champions the use of overscan when available).

"The offset, S_OFF(DN), represents the camera's output DN level in total absence of signal electrons. Subtracting a precise offset is an important requirement in measuring signal levels. Offset can sometimes be determined from a dark exposure using pixels from the same region where signal is measured. However, the offset information on the array may be corrupted with dark current or other sources of charge (e.g., light leak, spurious charge, etc.). The best location on the CCD for a zero signal level is in the "extended pixel region" of the horizontal register..."

For "extended region" read overscan.

However Bias frames probably contain information about characteristics which may - or may not be - relevant to the user.

"..the offset level does not needed to be subtracted pixel by pixel. Only a single difference from the average signal level is required, thus saving computing time."

So patterns in bias can be ignored for purposes of a PTC curve - I suspect they influence - but I equally suspect, from above, that they do not significantly matter in aggregate (in the overall balance of things).

Another mine in the mine field - I am interpreting from my reading that bias can change over time - or even per frame. Even overscan is only an estimate of bias.
Therefore there are two things I wish to know more about:-
1) What *is* bias
2) What *is* readout noise
If I - or a colleague - where to casually make some statement about either of these two terms (I have stock definitions) in a general amateur context I feel we would know what I - or my colleague - is talking about - but in the minutiae I now have questions in my head.

Mark C.

ATIK replied back to Green-Witch with the following:

"The customer doesn't have access to the over scan area I'm sorry to say"

So looks like BIAS frames is all I can do

Hi,

mjc wrote: Another mine in the mine field - I am interpreting from my reading that bias can change over time - or even per frame. Even overscan is only an estimate of bias.
Therefore there are two things I wish to know more about:-
1) What *is* bias
2) What *is* readout noise
If I - or a colleague - where to casually make some statement about either of these two terms (I have stock definitions) in a general amateur context I feel we would know what I - or my colleague - is talking about - but in the minutiae I now have questions in my head.

As I understand it (and I'm open to correction!)...

Immediately before an exposure the CCD array is wiped, so theoretically an exposure of 0 length should record 0 ADU but because of the noise in the electronics (e.g. A/D converter, amplifier) this is not 0, instead it's a distribution around 0. The CCD will add a pixel value though to avoid negative values which is why you see all positive values (~270ADU for my ccd). There are two components to this noise.

There will be a repeatable non-random aspect to it, that what your master bias captures. In my bias master (I'll post a link later) there is a clear harmonic visible on the left of the image, that has a maximum peak to peak of about 15adu, and the frame has a stdev of 2.5ADU. By averaging a set of bias frames you will be left with that non-random noise pattern, which you can then subtract as needed from light frames if you are not using darks.

There is also a random noise component which has no structure. This is measured by comparing any single bias frame against the averaged bias frame. Since it's random it's just quoted as a single value rather than having a 'frame ' for it. Also, since it's random, there is no way to remove it, but knowing it helps to ensure that we can aim to get sufficient signal to increase the signal to noise ratio.

~Al

My Master BIAS (250 Stacked BIAS Frames) has the following:

Average ADU : 353.559
STDEV: 8.771

Time for a lie down in a dark room again...

Al - thanks for the images. Coupled with Mark's post, and your subsequent post, I (think) have a better understanding of Bias v Overscan.

Bias will reveal any 2D patterns that just using the overscan will miss. I suppose you could construct a master overscan-bias frame and compare that to a master bias and see if there was any 2D pattern to worry about.

My overscan region is (if I recall) 1024 pixels by only 140 odd pixels - I don't know if that would be comparable to a full 1024 x 1024 frame? But worth looking into...

Just say the word and I'll send the spreadsheet on (along with my own data so you see it in its populated form). I actually never did try any of the 'simpler' methods - I stalk the FLI boards where Richard Crisp and Tim Khan frequent - they both helped me along with the more complex method - so I just went with the flow. Richard, I think, is some kind of engineer and I believe he has something to do with chip design - possibly even CCD's. He's an acquired taste for sure. But I'd trust him implicitly. Mark - he's actually quite approachable. If he has an email address on his website (I think its narrowband imaging .com) he'd converse with you I'm sure. Just don't take anything he says personally! He certainly doesn't suffer fools but he will meet newbies half way. He is very fond of Janesick by the way.

I hear you on he minutiae - one minute I think I have it then I read something else and I start to question my understanding of the fundamentals... Sometimes it takes a little time for these things to sink in though.

Simon - is your head melted yet? Shame about the overscan not being available (as Mark quoting Janesick mentions). I'm sure we'll get to the bottom of how you can use the Bias frames correctly and where the number go into the PTC/DTC calculation.

Now, where's that bag of ice...

Dave

Dave

Fortunately we've had enough rain to keep my head cool and from melting

Sure is an interesting subject, and I still don't know what my optimal ADU is for flats

Simon