\documentclass{article}
\pagestyle{empty}
\setlength{\textwidth}{415pt}
\setlength{\textheight}{610pt}
\setlength{\hoffset}{-40pt}
\setlength{\parindent}{0cm}

\title{Data mining, SourceList and AssociateList exercises}
\begin{document}
\maketitle

\section*{Investigating twilight behaviour from RawTwilightFlatFrames}
\textit{Question 1:}
Find all raw twilight flat frames for the chip with name \texttt{ccd52} and the
filter with \texttt{\#843}, which have image statistics such that the median
is larger than 1000. Only select those that have their \texttt{quality\_flags}
set to zero.
Assign the \emph{query} to a Python variable.

\noindent
How many raw twilight flat frames satisfy the above criteria?

\vspace{0.3cm}
\textit{Solution:}
\noindent
\begin{verbatim}
awe> qccd52 = RawTwilightFlatFrame.chip.name     == 'ccd52'
awe> q843   = RawTwilightFlatFrame.filter.name   == '#843'
awe> qim    = RawTwilightFlatFrame.imstat.median >  1000
awe> qqf    = RawTwilightFlatFrame.quality_flags == 0
awe> query  = qccd52 & q843 & qim & qqf
awe> print len(query)
1033
\end{verbatim}

\textit{Question 2:}
Using the result from the previous question, assign the MJD\_OBS (the modified
Julian date) of each RawTwilightFlatFrame to variable \texttt{t}
and the median of the image statistics divided by the EXPTIME
of each RawTwilightFlatFrame to variable \texttt{y}.
Make a scatter plot of \texttt{t} versus \texttt{y}.

\textit{Solution:}
\noindent
\begin{verbatim}
awe> t = [rtf.MJD_OBS for rtf in query]
awe> y = [rtf.imstat.median/rtf.EXPTIME for rtf in query]
or, quicker,
awe> result = [(rtf.MJD_OBS, rtf.imstat.median/rtf.EXPTIME) for rtf in query]
awe> t, y = zip(*result)
awe> pylab.scatter(t, y)
\end{verbatim}

\textit{Question 3:}
Since there seems to be a recurring pattern in the previous plot, we'll
have a closer look. Take the fractional part of the \texttt{t} variable,
which contains the modified Julian date for each RawTwilightFlatFrame and
assign it to the variable \texttt{tfrac}.
Clear the figure and make a scatter plot of \texttt{tfrac} versus \texttt{y}.

\textit{Solution:}
\noindent
\begin{verbatim}
awe> tfrac = [k - floor(k) for k in t]
awe> clf()
awe> pylab.scatter(tfrac, y)
\end{verbatim}

\textit{Bonus question:}
Determine the time difference between Greenwich and the place where the
RawTwilightFlatFrames were observed - La Silla.
Does this correspond to the longitude of La Silla?

\section*{Looking at skybrightness using ReducedScienceFrames}
{\it Question 1:}
Plot the skybrightness as function of date.

\section*{SourceList and AssociateList}

\textit{Question 1:}
Most SourceLists are derived from frames or images.  Some however are
not.  Find all SourceLists which do not have a frame associated with it.

\textit{Solution:}
\noindent
\begin{verbatim}
awe> sls = SourceList.name != ''
awe> for sl in sls:
...   if (not sl.frame): print sl.name, sl.number_of_sources
...
\end{verbatim}

You will have found the only lists which don't have a history and they
are the standard catalogues USNO=A2.0 and SDSS-DR4.  The two sourcelists
are in principle growing sourcelist which will fetch the information
on demand from remote servers.

\textit{Question 2:}
Find out the piece of sky covered by the USNO catalogue in the database.

\textit{Solution:}
\noindent
\begin{verbatim}
awe> usno = (SourceList.name == 'USNO-A2.0')[0]
awe> RA, DEC = usno.sources.RA, usno.sources.DEC
awe> print max(RA), min(DEC), min(RA), max(DEC)
\end{verbatim}

This is the current coverage as of 11/11/2005 15:21 of the USNO-A2.0
catalogue in the astro-wise database. 

\textit{Question 3:}
Find the B and R mags of the sources in the USNO catalog which are
inside a distance of 0.1 degrees of position 12.0$^o$ -29.0$^o$.

\textit{Solution:}
\noindent
\begin{verbatim}
awe> print usno.info()
awe> attrs = { 'RA': [], 'DEC': [], 'USNO_RMAG': [], 'USNO_BMAG': [] }
awe> area = (12.0, -29.0, 0.1)
awe> r = usno.sources.area_search(Area= area, dict=attrs)
awe> nos = len(r[usno.SLID])
awe> for k in range(nos):
...   print attrs['USNO_BMAG'][k], attrs['USNO_RMAG'][k]
...
\end{verbatim}

\textit{Question 4:}

AssociateLists are used for the Global Astrometric Solution.  We are
going to inspect some of these lists in the database.  These are easily
found since there names all start with GAS:
\noindent
\begin{verbatim}
awe> als = AssociateList.name.like('GAS*')
awe> len(als)
\end{verbatim}
Take one of these Associatelists, for example:
\noindent
\begin{verbatim}
awe> al = als[2]
\end{verbatim}
and find out the sourcelists they have been made from and find out the
pointing and corresponding CCD's.

\textit{Solution:}
\noindent
\begin{verbatim}
awe> sls = al.sourcelists
awe> for sl in sls:
...   if (sl.frame):
...    print sl.frame.chip.name, sl.frame.astrom.CRVAL1, sl.frame.astrom.CRVAL2
\end{verbatim}

\vspace{0.3cm}
\noindent

\end{document}
