glossary

Glossary of Astro-WISE concepts, services, applications and terminology

code updates

News related to important updates to the code.

mailing lists

News
archive	subscribe
Issues
archive	subscribe

planning

Wiki site used for work planning (restricted access).

printer friendly version

1.1 FAQ

1.1.1 General

1.1.1.1 Introductory Material

Q: What is Astro-WISE?
A: Astro-WISE for Astronomical Wide-field Imaging System Europe, is an E.U. consortium and associated information system dedicated to the handling of wide-field image data resulting from large astronomical surveys. Please see http://www.astro-wise.org/what.shtmlthe main page for a complete description.
Q: What is the Astro-WISE Environment?
A: The Astro-WISE Environment, or simply AWE, is the information system built to handle this flood of astronomical data. http://www.astro-wise.org/what_awe.shtmlStart here to learn more about it.
Q: What language is the Astro-WISE Environment written in?
A: The Astro-WISE Environmentis written mainly in the http://www.python.org/Python programming/scripting language. There are also Python utility modules written in C (see the http://www.astro-wise.org/portal/howtos/man_howto_introduction/man_howto_introduction.shtmlHOW-TO Introduction), but these are wrapped in Python to interface with the system.
Q: What kind of web-services does AWE offer?
A: See the http://www.astro-wise.org/portal/howtos/man_howto_introduction/man_howto_introduction.shtmlIntroduction HOW-TO for a complete list of what AWE offers.

1.1.1.2 Getting Started

Q: I can't access anything but the basics in the AWE, nor can I access the database. What's wrong?
A: Do you have an AWE account? If not, please contact an http://www.astro-wise.org/contact.shtmlAstro-WISE representative.
Q: I keep seeing references to $AWEPIPE. What is it?
A: $AWEPIPE is the environmental variable that should point to your ``awe'' checkout, the code-base for AWE (e.g., 1#1 mathend000#/awe/).
Q: How do I start AWE?
A: There are many options, but the typical one is to simply use AWE on a properly configured system:
```
...]$ awe
Python 3.5.1 (default, Nov  1 2016, 21:12:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.


               Welcome to the Astro-WISE Environment


Importing Astro-WISE packages. Please wait...
.
.
.
awe>
```
You can also temporarily modify environmental variables that AWE uses so that AWE starts with a different configuration:
```
...]$ env AWEPIPE=~/test-awe/ awe

or

...]$ env database_engine= awe
```
The first example causes AWE to use a code-base located in a specified directory (1#1 mathend000#/test-awe/ here) and the second disables use of the database (if you need to work offline for some reason).

1.1.1.3 Documentation

Q: I'm a new user. Where should I look for documentation?
A: The first place any new user should look for documentation is the http://www.astro-wise.org/portal/aw_howtos.shtmlHOW-TOs
Q: What are docstrings?
A: Docstrings are the inline comments added to Python code that allow for an automated documentation system called PyDoc. Docstrings are contained within triple quotes:
```
'''this is a docstring'''
"""this is also a docstring"""
# this is just a comment
```

1.1.1.4 GitLab (Git)

Q: What is the address of the Astro-WISE GitLab server?
A: https://gitlab.astro-wise.org
Q: What is the proper way to fully update a code-base?
A: First, make sure you are in the top-level directory (or whatever directory you want fully updated) and issue the following command:
```
...]$ git checkout master
...]$ git pull
```
Q: How do I switch between the latest $AWEPIPE version and the master version?
A: To switch to the most recent check out use
```
...]$ git checkout develop
...]$ git pull
```
and to switch back to the master version, use
```
...]$ git checkout master
...]$ git pull
```
in the `awe' directory.

1.1.1.5 Data Preparation

Q: I am about to go observing with an AWE-supported instrument. Is there anything special that I should do?
A: Please look at the http://www.astro-wise.org/portal/howtos/man_howto_schedule/man_howto_schedule.shtmlObservations Scheduling HOW-TO for guidelines.
Q: I want to process the data I took on < mathend000#insert favorite instrument> mathend000# with AWE. What do I need to do?
A: AWE was created specifically for http://www.astro.rug.nl/ omegacam/OmegaCAM and will handle OmegaCAM data ``out-of-the-box''. There is also support for the instruments listed on the http://www.test.astro-wise.org/portal/aw_datasources.shtmlSupported Data Sources page. If your instrument is not there, contact an http://www.astro-wise.org/contact.shtmlAWE representative for verification and have a look at the http://www.astro-wise.org/portal/howtos/man_howto_developers_newinstrument/man_howto_developers_newinstrument.shtmlNew Instrument HOW-TO to get started.

1.1.1.6 Ingesting

Q: What is data ingestion and how do I do it?
A: In order for raw data to be processed in the Astro-WISE system, it must first be ingested, or imported into the system. Please see the http://www.astro-wise.org/portal/howtos/man_howto_ingest/man_howto_ingest.shtmlData Ingestion HOW-TO for a complete description.
Q: How can the data be categorized for ingestion in AWE?
A: Data in AWE is catagorized by purpose:

readnoise

bias frames designated to determine the instrument read noise

bias

a raw bias or zero second exposure frames

dark

a frame taken to measure the dark current of the instrument

gain

a specific series of frames designed to determine the gain of the instrument

dome

a raw flat frame taken from a screen mounted within the dome enclosure

twilight

a raw flat frame taken at twilight on the sky

science

any frame taken for the purpose of science (excluding photometric calibration)

photom

a science frame taken with the specific purpose of photometric calibration
Q: After I ingested my data, all the filenames were different. What happened?
A: The filenames were converted to AWE cannonical names of the form:
```
<instrument>.<date_obs>.fits

or

<instrument>.<date_obs>_n.fits
```
The first from is for multi-extension FITS images, the second is for single-extension FITS images where n is the extension number.

1.1.1.7 Dates and Times

Q: How are observation dates defined in AWE?
A: An observation night is based on the local date at sunset and not on UTC. See the http://www.test.astro-wise.org/portal/howtos/man_howto_dates/man_howto_dates.shtmlDates and Times HOW-TO for more information.
Q: How are dates stored in the database?
A: In UTC only.

1.1.1.8 Parallel Processing

Q: What is the parallel processing interface in AWE?

A: The DPU (Distruibuted Processing Unit) is used to run tasks on the parallel compute cluster. It is initialized when AWE starts:

...]$ awe
.
.
.
Importing Astro-WISE packages. Please wait...

Initializing Distributed Processing Unit...

Current profile:
.
.
.
awe>

To initiallize it manually, use this method:

awe> from astro.recipes.mods.dpu import Processor
awe> my_dpu = Processor(Env['dpu_name'])

Q: How do I find the instruments, task identifiers, and options supported by the DPU's run method: dpu.run()?

A: This information is somewhat hidden. The supported instrument list can be seen by querying the HeaderTranslatorFactory:

awe> from astro.instrument.HeaderTranslatorFactory import supported_instrument_list
awe> print supported_instrument_list

Task identifiers can be found like this:

awe> from astro.recipes.mods import Pipeline
awe> for id in Pipeline.get_available_sequence_identifiers(): print id

And the options can be printed in this way:

awe> from astro.recipes.util.ArgumentParser import ArgumentParser
awe> for opt in ArgumentParser().opts: print opt

This gives you a list of tuples of the form (short_opt, long_opt, definition).

1.1.1.9 `awe`-prompt

Q: What is namespace and how can I see it?
A: Simply speaking, namespace is all modules, classes, attributes, and methods available in the current scope (i.e., level). If the module, class, attribute, or method you need is not visible in the current namespace, then it cannot be used. It is either loaded within the namespace at a different scope, or it is not there at all. The builtins dir() and help() allow you to explore the namespace.
dir() without any arguments gives the namespace of the current scope. It typically will give Python builtins and any modules pre-loaded at startup or during the current session. Calling dir() with a module, class, attribute, or method as the argument will give you the namespace at that level or an arbitrary level:
```
awe> dir()
['__builtin__', '__builtins__', '__doc__', '__file__', '__name__', '__version_
_', 'astro', 'atexit', 'os', 'pydoc', 'readline', 'rlcompleter', 'sys', 'users
tartup']
awe> dir(os)
['EX_CANTCREAT', 'EX_CONFIG', 'EX_DATAERR', 'EX_IOERR', 'EX_NOHOST', 'EX_NOINP
UT', 'EX_NOPERM', 'EX_NOUSER', 'EX_OK', 'EX_OSERR', 'EX_OSFILE', 'EX_PROTOCOL'
.
.
.
ttyname', 'umask', 'uname', 'unlink', 'unsetenv', 'utime', 'wait', 'waitpid', 
'walk', 'write']
awe> dir(os.write)
['__call__', '__class__', '__cmp__', '__delattr__', '__doc__', '__getattribute
__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__'
, '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__str__']
```
help() can be used in a similar way, but it gives all the docstrings recursively for the module, class, or method, or for the parent class of the attribute that was given as the argument.
Q: When importing something from the awe-prompt, I get strange error messages through my screen. I know that the software is okay. What is the deal?
A: Make sure that you are not importing something from within the awe directory structure. Due to some strange Python things concerning packages, this can lead to completely intractable error messages.
Q: One of the Python recipes generates an error and I can not figure out what is wrong. What should I do?
A: Sometimes problems arise because of old compiled Python files (*.pyc); there may be leftovers of Python files that no longer exist. Remove all these files and try again.
Q: What are some usefull packages and modules available at the awe-prompt?
A: See the http://www.astro-wise.org/portal/howtos/man_howto_awe/man_howto_awe.shtmlawe-prompt HOW-TO.

1.1.1.10 Queries

Q: What are queries and how do I use them?
A: Queries in AWE are queries to the database from Python with the purpose of returning some needed information. The http://www.astro-wise.org/portal/howtos/man_howto_queries/man_howto_queries.shtmlQueries HOW-TO containes a complete explaination of the use of queries in AWE.
Q: What's the deal with the select() method?
A: The select() method is simply a convenience method to find the most recent, valid version of a given frame. Use the Python builtin help() on a particular select method to find out exactly how it makes this choice.

1.1.1.11 Process Parameters

Q: Which process parameters can be configured and where?
A: Generally speaking, all process parameters can be configured from the awe-prompt prior to running a make or executing a task. The http://www.astro-wise.org/portal/howtos/man_howto_configure/man_howto_configure.shtmlConfiguration HOW-TO gives a complete description of the various methods for process parameter configuration.

1.1.1.12 Context

Q:
A:
Q:
A:
Q:
A:
Q:
A:
Q:
A:

1.1.2 Astro-WISE Environment

Q: I want to run SExtractor on my images. How do I do that ?
A: There are basically three ways to run SExtractor in the system. The most direct one would be to import `Sextractor' from `astro/external' and run `Sextractor.sex(image)'. You can also set the SExtractor configuration through this interface, and modify the list of output parameters. A second way to run SExtractor would be using the `sex' method of BaseFrame or children thereof. To be concrete, instantiate a ScienceFrame object and call the `sex' method. The third, and in our paradigm the most correct, way to run SExtractor, is to invoke the `make' method of a Catalog object. The frame you want to extract sources from is given as a dependency to the Catalog object. For the configuration of SExtractor and to modify its outputs, you need to provide the Catalog object with two additional dependencies: a SextractorConfig object, and a list of parameters. Note that this last way of creating a SExtractor catalog uses the first method. Also note, that the output of running SExtractor is in LDAC fits format.
Q:
A:
Q:
A:
Q:
A:

1.1.3 AW Tutorials

Q:
A:
Q:
A:
Q:
A:

1.1.4 Calibration

Q:
A:
Q:
A:
Q:
A:

1.1.5 Image Pipeline

Q:
A:
Q:
A:
Q:
A:

1.1.6 Visualization

Q:
A:
Q:
A:
Q:
A:

1.1.7 Development

Q:
A:
Q:
A:
Q:
A:

page generated Tue Nov 21 10:29:18 CET 2017