1.1 HOW-TO USE DARMA

DARMA is a Data Acquisition, Representation, and MAnipulation interface for use with FITS¹ images and headers, and eventually FITS tables. The current interface with the Astro-WISE Environment uses only the FITS header functionality of DARMA, and that is all that will be discussed in this HOW-TO.

1.1.1 DARMA Header Interface

DARMA headers are effectively PyFITS²headers with alot of fancy extra functionality and optimizations to make working with FITS headers in Astro-WISE very simple and powerful. As with PyFITS headers, DARMA headers include header verification, but on-demand instead of upon output to a file, so you always know your header meets the current FITS standard as implemented in the version of PyFITS being used. DARMA's header interface has at minimum, the same functionality Eclipse headers have. This means, that if you are already familiar with Eclipse headers, you need not read further to use it. If you are not or want to know more, both the basic and the more advanced functionality will be described below and in the following sections.

The basic access to DARMA headers is a Python dictionary-like interface:

  awe> header = darma.header.header('filename.fits')
  awe> header['BITPIX']
  -32
  awe> header['OBJECT']
  'Standard'
  awe> header['OBJECT'] = 'Standard (SA101)'
  awe> header['OBJECT']
  'Standard (SA101)'
  awe> del header['OBJECT']
  awe> header['OBJECT']
  None
  awe> header['OBJECT'] = 'Standard (SA101)'
  awe> header['OBJECT']
  'Standard (SA101)'
  awe> header['OBJECT'] = ('Standard (SA101)', 'Object name from telescope')
  awe> header.OBJECT
  OBJECT  = 'Standard (SA101)'   / Object name from telescope
  awe>

The example above shows the basic interface to a FITS header and basic manipulations of the header values. A header is loaded upon instantiation with a valid filename and an optional index indicating the extension number (0=primary, 1=first extension, 2=second extension, etc.). Access to a keyword or numeric index of the header returns the value of the keyword, and allows modification of the value or deletion of the keyword entry, or addition of a new one (the old one is recreated in the example above.). Lastly, keyword comments can be added using a (value, comment) tuple. The card displayed by the header attribute shows the added comment. Header attribute access is described next.

NOTE: Keywords dealing with the data (e.g., BITPIX, NAXIS, etc.) can be modified, but any changes will be lost when the header is paired with data, as their values will be take directly from the data.

In addition to the primary dictionary interface, there is an informational, attribute-based interface that can show the value of the entire header card for a given keyword. There is also a special attribute that shows a listing of all header cards:

  awe> header.BITPIX
  BITPIX  =                  -32 / number of bits per data pixel
  awe> header.BITPIX.key
  'BITPIX'
  awe> header.BITPIX.value
  -32
  awe> header.BITPIX.comment
  'number of bits per data pixel'
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                  -32 / number of bits per data pixel
  NAXIS   =                    0 / number of data axes
  EXTEND  =                    T / FITS dataset may contain extensions
  .
  .
  .
  awe>

The special attribute `card_list' returns the representation of a PyFITS CardList (pyfits.Header.ascardlist()) and shows exactly what the actual header looks like in the FITS file. The dump() method simply prints the string representation of this same card list. Use the dump() method to show the header contents with newline characters included.

NOTE: The attribute-based interface goes only one-way. It shows the keyword values, but cannot set them. You must use the dictionary interface to modify keyword values or comments!

1.1.2 On-demand Header Verification

DARMA headers are verified using PyFITS output verification on demand at every change or access to the header (e.g., modify, add, or delete a keyword, value, and/or comment). The verification is robust with a default verification option of `silentfix'. Use the `option' keyword in the constructor to override this default value.

ADVANCED: If you need to modify or add several keywords at once that would normally raise a verification error at each one, you can set the _IS_VERIFIED flag of the header object to True between changes to suppress automatic verification. After all the changes are complete, the next access to the header will run the verification to ensure that all is well.

1.1.3 Special Keywords

FITS standard allows for several keywords with special meaning or behavior. These keywords generally cannot be accessed or modified in the normal ways. Their unique properties are discussed below.

1.1.3.1 COMMENT, HISTORY, and `BLANK' keywords

The COMMENT keyword allows arbitrary lines of text to be entered into the header to give general commentary capability at any location in the header. The string will be folded at a length of 72 characters and put into as many comment cards as needed (i.e., a string of length 160 will be folded into 3 COMMENT cards of string length 72, 72, and 16 characters).

The HISTORY keyword has a similar purpose and is processed in the same way as the comment card. Its primary purpose, as its name implies, is to give a record of the history the associated FITS image has seen.

The `BLANK' keyword provides the facility to add spacers, or line-holders in the header. It is unlike the previous two because no keyword for it appears in the header, and because it can take an optional string value where the others have a mandatory string value.

All three of these keywords have special methods to write and to read them because they are not unique in occurrence like all other FITS keywords. The usage of the COMMENT, HISTORY, and `BLANK' keyword cards is illustrated below:

  awe> header = darma.header.header().default()
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  awe> header.add_comment('This is a comment.')
  awe> header.add_comment('This          is           a          very \
  ...         long        comment          indeed.')
  awe> header.add_history('This is a history statement.')
  awe> header.add_blank(after='EXTEND')
  awe> header.add_blank('         Begin comments and histories.', after='EXTEND')
  awe> header.add_blank(after='EXTEND')
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T

                     Begin comments and histories.

  COMMENT   This is a comment.
  COMMENT   This          is           a          very        long        comment
  COMMENT            indeed.
  HISTORY   This is a history statement.
  awe>

1.1.3.2 HIERARCH

PyFITS includes native support for ESO's HIERARCH keywords usage. With HIERARCH keywords, you can extend the 8-character limit of the standard FITS convention and use ESO's strict hierarchical keyword structure or a free-form structure. The keyword/value/comment length cannot exceed the length of the card (i.e., 80 characters). See the current Registry of FITS Conventions for more details.

DARMA handles HIERARCH keywords in effectively the same way PyFITS does. HIERARCH keyword cards can be accessed or set with or without the `HIERARCH ' string preceding the keyword. Keywords that are automatically interpreted as HIERARCH keywords are those that contain spaces, exceed 8 characters, or contain non-alphanumeric characters other than `-' or `_'. Keywords beginning with `HIERARCH ' are always interpreted as HIERARCH keywords.

  awe> header = darma.header.header().default()
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  awe> header['HIERARCH ESO TEL TEMP'] = 300
  awe> header['this keyword is non-standard'] = 'non-standard'
  awe> header['ThisKeywordIsAlsoNonStandard'] = 'also non-standard'
  awe> header['strange$keyword'] = 'strange'
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  HIERARCH ESO TEL TEMP = 300
  HIERARCH this keyword is non-standard = 'non-standard'
  HIERARCH ThisKeywordIsAlsoNonStandard = 'also non-standard'
  HIERARCH strange$keyword = 'strange '
  awe> header['ESO TEL TEMP']
  300
  awe>

1.1.4 Saving and Advanced Creation

Unlike Eclipse headers, DARMA headers can be saved and loaded from text files or lists of header cards. In this section, that functionality will be illustrated.

1.1.4.1 Saving Headers

Saving a header is as simple as calling its save method:

  awe> header = darma.header.header().default()
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  awe> header.save('header.fits')
  awe> header.save('header.txt', raw=False)
  awe> os.system('cat header.txt')
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  END
  0
  awe>

There are two ways to save a header as shown above: as raw FITS header as one line in 2880 byte blocks (the default), or as standard text with newlines at the end of each line in only enough file space to contain it. As you may have already noticed, the card_list does not contain an `END' header card. This is because PyFITS headers only receive the `END' card when written to a FITS file with data. This is also done upon saving a the header as you can see in the above example. The `0' after the `END' card is just the return value from the shell.

1.1.4.2 Creating Headers

DARMA headers can be loaded from FITS files as expected, but they can also be created in three other ways:

  awe> header = darma.header.header(card_list='header.txt')
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  awe> card_header = darma.header.header(card_list=header.card_list)
  awe> card_header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  awe> fd = file('header.txt')
  awe> lines = fd.readlines()
  awe> fd.close()
  awe> str_header = darma.header.header(card_list=lines)
  awe> str_header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  awe>

In the above example, three headers were made in three different ways from the same source. The first header was loaded from a text file that had one header card of the form `key = value / comment' per line. This was the text file saved in the previous example with option raw=False. The second header used the PyFITS.CardList instance as its source. The last header used a list of strings loaded from the same file that was the source for the first header.

NOTE: When using a text file directly for the header source, you MUST use the card_list option of the constructor. The filename option only works for real FITS file or headers that were saved with the default option raw=True. When you load a header from this last type of file, you may get a warning about data size inconsistency. In this case only, it can be safely ignored.

1.1.5 Information

Lastly, you can display information about a header object:

  awe> header = darma.header.header().default()
  awe> header.card_list
  SIMPLE  =                    T / conforms to FITS standard
  BITPIX  =                    8 / array data type
  NAXIS   =                    0 / number of array dimensions
  EXTEND  =                    T
  awe> header.info()
           class: <class 'astro.util.darma.header.header'>
    total length: 4 cards
       (comment): 0 cards
       (history): 0 cards
        itemsize: 80 bytes
       data size: 320 bytes
    size on disk: 2880 bytes
  awe>

Here you see the class, the total number of cards, the number of comment cards, the number of history cards, and the data sizes of each card, of all the cards, and of the header as stored in a FITS file.

NOTE: The astute reader should note that slack space of raw FITS-like header files can be considerable. If saving a large number of header files, the most optimal storage size will be achieved using the raw=False option (i.e., plain text output with newline characters).

Footnotes

... FITS¹

FITS is currently the only data packaging format supported. Different formats can and will be included in the future.

... PyFITS²

http://www.stsci.edu/resources/software_hardware/pyfits/