1.1 HOW-TO perform a trend analysis and to find outliers

1.1.1 Summary

This HOW-TO shows how one can do a simple database trend analysis from the awe-prompt. In general to get to a result you need to do the following steps:

  • Which quantities do you want to do a trendanalysis on? Determine which classes and attributes of objects in AWE are needed to get the desired information for those quantities.
  • Construct the database query/queries required to get the desired information.
  • Make a plot of the desired information to graphically detect outliers.
  • Refine the constraints in the query to encompass only outliers.
  • Retrieve the outlying objects and inspect them.

1.1.2 Examples

Question 1: Make a plot of the bias level of all raw biases of a CCD as a function of modified julian date of observation.
Answer 1:

awe> q = (RawBiasFrame.chip.name == 'ccd50')
awe> biases = list(q)
awe> x = [b.MJD_OBS for b in biases]
awe> y = [b.imstat.median for b in biases]
awe> pylab.scatter(x,y,s=0.5)

This results in the plot in figure [*] (zoomed, labels added).

Figure: Trend analysis: bias level for all raw biases of CCD against modified julian date of observation
1#1



Question 2: Look for raw biases for ccd50 (WFI) in 2004 for which the level of the trim section differs significantly from the level of the overscan.
Answer 2:

awe> q = (RawBiasFrame.filename.like('WFI.2004*_1.fits'))
awe> len(q)
419
awe> biases = list(q)
awe> x = [b.MJD_OBS for b in biases]
awe> y = [b.imstat.median-b.overscan_x_stat.median for b in biases]
awe> pylab.scatter(x,y,s=0.5)

Figure: Trend analysis: trend of raw bias level in trim section minus the level in the overscan X region. The difference is plotted as a function of modified julian date of observation.
2#2

This produces a plot as in figure [*].You can see that there seems to be one case where the difference is 5 ADU. This image will be interesting to look at. We can select it as follows:

awe> frames = [b for b in biases if b.imstat.median-b.overscan_x_stat.median > 4]
awe> len(frames)
2
awe> for f in frames: print f.filename
...
WFI.2004-10-15T15:10:02.248_1.fits
WFI.2004-10-15T15:11:52.384_1.fits
awe> for f in frames: f.retrieve()
...

It turns out there are in fact two frames of this kind. The images seem to have an uncharacteristic bright region in them; something was obviously wrong during these observations.


page generated Tue Apr 4 14:30:01 CEST 2017