Astro-Wise SourceLists ans AssociateLists

What is a SourceList

results of a sextractor run on
- CoaddedRegriddedFrame/RegriddedFrame
- RawScienceFrame/ReducedScienceFrame
Catalogues, not connected to any image or frame

DataBase implementation

Store as one big table in , where

Each SourceList has its own MetaData
Each SourceList is identified by an unique SLID, each source by an unique SID (example)
Use of OODB to allow for different attributes
Use HTM for spatial indexing and cone search

How to create a SourceList

Default way (use AW sextractor defaults)

awe> sourcelist = SourceList()
awe> sourcelist.frame = coaddedframe
awe> sourcelist.frame.retrieve()
awe> sourcelist.make()

Modify sextractor run via sexconf and sexparam

awe> sourcelist = SourceList()
awe> sourcelist.frame = coaddedframe
awe> sourcelist.frame.retrieve()
awe> sourcelist.sexconf.DETECTION_THRESHOLD = 12
awe> sourcelist.sexparam = ['MAG_APER', 'MAGERR_APER']
awe> sourcelist.make()

Import sourcelist from fits table

awe> sourcelist = SourceList(pathname='external.fits')
awe> sourcelist.make_sourcelist_from_catalog()

Retrieving Sourcelists and sources

How to retrieve a SourceList:

by SLID

awe> sourcelist = (SourceList.SLID == 4130)[0]

by name

awe> sourcelist = (SourceList.name == 'MySourceList')[0]

awe> sourcelists = (SourceList.name.like('GAS*'))

How to find the number of sources:

len function on attribute sources

awe> nos = len(sourcelist.sources)

sourcelist attribute number_of_sources

awe> nos = sourcelist.number_of_sources

How to find the possible attributes of a source

print it
```
awe> print sourcelist.info()
          
```

use method get_attributes

awe> attrs = sourcelist.sources.get_attributes()
awe> print attrs

How to retrieve source info:

per attribute

awe> RA = sourcelist.sources.RA
awe> print RA

per source

awe> source_info = sourcelist.sources[13]
awe> print source_info

Methods on SourceLists

overplot on skycat

awe> sourcelist.sources.make_skycat(sid_list=None,
                                    filename=None)
awe> help(sourcelist.sources.make_skycat)

cone search

awe> sourcelist.sources.area_search(self_search=True,
                                    htm_depth=20,
                                    Area=None)
awe> help(sourcelist.sources.area_search)
awe> r = sourcelist.sources.area_search(Area=[(1,0),(0,0),(0,1),(1,1)])
awe> print 'Sources found: ', len(r[sourcelist.SLID])

get data

awe> sourcelist.sources.get_data(dict)
awe> help(sourcelist.sources.get_data)
awe> d = {'RA':[], 'DEC':[], 'MAG_ISO':[]}
awe> r = sourcelist.sources.get_data(d)

SQL query

awe> sourcelist.sources.sql_query(dict, query_string)
awe> help(sourcelist.sources.sql_query)
awe> d = {'RA':[], 'DEC':[], 'MAG_ISO':[]}
awe> r = sourcelist.sources.sql_query(d, 'MAG_ISO<18.5 AND A>0.2')

Examples

Astro-Wise USNO cone search engine
dbview

Spatial associations of sources

Associate sources from different SourceLists with each other according to their positions on the sky, making use of:
- HTM index to select the area of interest (overlap)
- HTM index to find neighbouring indices
- SQL to do the INNER JOIN of two SourceLists
- Store the associations in an AssociateList (example)
what does it exactly do:
- First the area of overlap of the two sourcelists is calculated. If there is no overlap no associating will be done.
- Second the sources in one sourcelist are paired with sources in the other sourcelist if they are within a certain distance from eachother. Default distance is 5". The pairs get an unique associate ID (AID) and are stored in the associatelist. A filter is used to select only the closest pairs.
- Finally the sources which are not paired with sources in the other list and are inside the overlapping area of the two sourcelists are stored in the associatelist as singles. They too get an unique AID.

How to make an AssociateList

awe> AL = AssociateList()
awe> sl0 = (SourceList.SLID == 0)[0]
awe> sl1 = (SourceList.SLID == 1)[0]
awe> AL.input_lists.append(sl0)
awe> AL.input_lists.append(sl1)
awe> AL.make()

optional:

awe> AL.set_search_distance(5.0)
awe> AL.filter_for_closest_pairs = False
awe> AL.set_search_area(llra,lldec,lrra,lrdec,urra,urdec,ulra,uldec)
awe> AL.associatelisttype = 1

NOTE: You can give an existing AssociateList as input instead of a SourceList.

Three types of associatelists

Chain Association: the next sourcelist is paired with the previous.
Master Association: the next sourcelist is paired with the first.
Matched Association: all sourcelists are matched.

Methods on AssociateLists

count number of members

awe> asslist.associates.count_from_members(members=0,
                                           mode='EQ')
awe> help(asslist.associates.count_from_members)

count from selected sourcelists

awe> asslist.associates.count_from_flag(mask=None,
                                        mode='ANY',
                                        count=None,
                                        countmode='EQ')
awe> help(asslist.associates.count_from_flag)

get data

awe> asslist.associates.get_data(attrlist=[],
                                 mask=None,
                                 mode='ANY',
                                 count=None,
                                 countmode='EQ')
awe> help(asslist.associates.get_data)
awe> attrlist = ['RA', 'DEC', 'MAG_ISO']
awe> r = asslist.associates.get_data(attrlist,mask=3,mode='ALL')
awe> aids = [k for k in r.keys()]
awe> aids.sort()
awe> print 'AID', attrlist
awe> for aid in aids[:10]:
awe>  for row in r[aid]:
awe>   print aid, row

Examples

dbview

BONUS EXERCISE

Find an associatelist which was used for a Global Astrometric Solution. Then try to find the distance between two different pointings by comparing the grid positions of associated sources i.e. sources belonging to the same association.

Hints:

Get a GAstrometric Object and the AssociateList used.

gass = GAstrometric.gasslist.name != ''
gasslist = gass[0].gasslist

Get the sourcelist used and take two which have a different pointing but the same chip.
Find out which mask and mode you have to set for the get_data method.

Find out which attributes you need

gasslist.associates.get_attributes()

Solution:

attrlist = ['Xpos', 'Ypos']
r = gasslist.associates.get_data(attrlist=attrlist,
                                 mask= 2 + 4,
                                 mode= 'INTERSECT')