Detailed Requirements for FASEnvironment

The detailed requirements were discussed by the Network during 2005 and early 2006 after which document (see attachments) was created. It went through an internal, written review procedure in May-June 2006 which produced the document version 0.91 of 2006-08-01. A list of the requirements is given below with priorities (A,B,C):

Installation

• R1.1.1 A: The environment must be easy to install on typical systems used by astronomers
• R1.1.2 A: It must be possible to customize the installation for local needs
• R1.1.3 A: A simple configuration tool must be provided
• R1.1.4 A: The installation of the single-user, standard system shall not require system/root privileges
• R1.1.5 A: All warnings and errors occurring during the installation must be provided in a comprehensive log
• R1.1.6 A: Both single-user and shared installations must be possible
• R1.1.7 A: Installation shall rely only on general, commonly available tools
• R1.1.8 A: Installation from source code must be possible
• R1.1.9 B: Both configuration and installation tools should be offered as CL and GUI versions
• R1.1.10 B: Simple binary installation should be supported for main platforms
• R1.1.11 B: The environment should have a well defined package structure
• R1.1.12 B: It should be possible to view package dependencies in the system
• R1.1.13 B: All packages should have checksum, signature and size information

Patches and updates

• R1.2.1 A: Standard procedures for easy distribution of patches and updates must be provided
• R1.2.2 A: It shall be possible to view available patches/updates and compare their version with the ones installed
• R1.2.3 A: Installation of updates must be supported through Internet
• R1.2.4 A: All patches/updates must have associated documentation
• R1.2.5 A: It must be possible to view patch/update documentation without downloading the entire package
• R1.2.6 A: Patches/updates must be linked to bug reports
• R1.2.7 B: It should be possible to select patches/updates and have them install automatically
• R1.2.8 B: Repositories of packages and patches/updates should be available through Internet
• R1.2.9 B: New patches/updates should be announced
• R1.2.10 B: Updates may be installed from a standard file system

Documentation

• R1.3.1 A: Standards for documentation and help information must be defined
• R1.3.2 A: Editable documentation must be available in an open standard format
• R1.3.3 A: Documentation must be searchable and have indices
• R1.3.4 A: All commands/methods must have on-line help available including version information
• R1.3.5 A: Help text, documentation and tutorials must be part of any package
• R1.3.6 A: End-user, development, and integration documentation must be provided
• R1.3.7 B: All packages should have a common documentation structure
• R1.3.8 B: In-line code documentation should follow an open standard format

License issues

• R1.4.1 A: The standard system must have a license which allows free access to source code, re-distribution, and code modification
• R1.4.2 A: All external software packages needed for running the environment must be available without license fees
• R1.4.3 A: Any licensed software must be clearly identified

Deployment

• R1.5.1 A: It must be possible to install and run a version of the standard system on a single-user, desktop workstation or laptop
• R1.5.2 A: Internet connection must not be required for basic usage
• R1.5.3 A: The system must be available for Linux and POSIX compliant systems
• R1.5.4 A: Minimum resources (hardware and software) required to install and run the system must be defined
• R1.5.5 B: It should be possible to use remote resources (e.g. clusters, grid nodes) dynamically when they become available
• R1.5.6 B: Client access to the environment should be provided from a wide range of systems

Support of coding languages

• R1.6.1 A: POSIX C and C++ must be supported for development of components
• R1.6.2 A: FORTRAN must be supported for development of components
• R1.6.3 A: Java must be supported
• R1.6.4 A: Python must be supported
• R1.6.5 A: Documentation of the API for supported languages must be provided
• R1.6.6 B: Style guides, tutorials and examples should be available for all supported languages

Revision control

• R1.7.1 A: All parts of the system (i.e. source code, documentation configuration files etc.) must be under revision control
• R1.7.2 B: Tools for generating version information in a standard format should be available
• R1.7.3 B: All releases of the system or parts of it should be done with explicit version and listing of versions of all its parts
• R1.7.4 B: Links between bug reports and versions should be provided
• R1.7.5 B: It should be possible to get, dynamically, the explicit version of any science task being executed
• R1.7.6 B: It should be possible to get version numbers added to the execution logs and results being generated
• R1.7.7 B: Read access to the system source code should be possible

Logging and errors

• R1.8.1 A: A system wide, standard error/logging interface must be available for all supported languages
• R1.8.2 A: A detailed log of tasks executed must be available
• R1.8.3 A: It must be easy to repeat execution of tasks listed in command logs
• R1.8.4 A: Errors occurring during the execution must be checked, reported and logged
• R1.8.5 B: Error levels should be supported
• R1.8.6 B: Logging level should be supported
• R1.8.7 B: It should be possible to configure the system so that only certain error/log levels are shown
• R1.8.8 B: All error/log messages should contain time stamps and information on host and task versions
• R1.8.9 B: It should be possible to add comments and notes to the command log (i.e. to use them as worksheets with an intermix of descriptions, notes, commands and results)

Test and validation

• R1.9.1 A: Standards for test and validation of the system as well as for individual scientific tasks must be established
• R1.9.2 B: Automatic regression tests should be available for all parts of the system
• R1.9.3 B: Tests and associated data should be associated to all packages
• R1.9.4 B: All tests should generate a comprehensive report which clearly indicates if problems were identified and the total time required

Internationalization

• R1.10.1 A: It must be possible to add internationalization for user interaction
• R1.10.2 B: It should be easy to provide internationalization of the general documentation (i.e. manuals, on-line help, API specifications)
• R1.10.3 A: An English version of all documentation (i.e. manuals, on-line help, API specifications) must exist
• R1.10.4 A: It must be possible to generate an English log

Support of standards

• R1.11.1 A: All interprocess communication must use open interfaces and protocols
• R1.11.2 A: All communication through Internet browsers must be browser independent
• R1.11.3 A: Printable documents must be available in PDF is widely accepted as an open standard for printable documents offering both
• R1.11.4 A: Standard languages for access to Internet information must be supported
• R1.11.5 A: Access to Web services must be supported
• R1.11.6 A: Access to VO services must be supported
• R1.11.7 A: It must be possible to read and write data in standard formats
• R1.11.8 A: FITS data must be fully supported
• R1.11.9 A: Interaction with VO data must be provided
• R1.11.10 B: It should be possible to transform documentation into an open, standard format
• R1.11.11 B: Whenever appropriate, existing open standards should be used
• R1.11.12 B: It should be possible to add support for other formats

Access to external packages

• R1.12.1 B: Standards for interfaces to commercial packages should be established
• R1.12.2 B: Interface to external packages should provide the option to execute tasks in the external system and retrieve results
• R1.12.3 C: Access to external packages may be provided using vendor specific, dynamic libraries
• R1.12.4 C: Tutorials on writing a client/server component to an external tool may be available

Execution

• R2.1.1 A: It must be possible to execute individual tasks directly from the operating system shell
• R2.1.2 A: Both position and key defined parameters must be possible
• R2.1.3 A: It must be possible to specify parameters in a parameter set file
• R2.1.4 A: Logging of the execution of tasks, their parameters and results must be supported
• R2.1.5 A: Standard termination codes must be provided by tasks
• R2.1.6 B: It should be possible to specify a specific version of a task to be executed

Task parameter

• R2.2.1 A: A standard for passing parameters between tasks must exist
• R2.2.2 A: Sequences of parameters must be supported
• R2.2.3 A: Units must be specified for relevant parameters
• R2.2.4 A: Parameters with undefined values (i.e. NULL) must be supported
• R2.2.5 A: It must be possible to specify uncertainties for relevant parameters
• R2.2.6 A: It must be easy to use output parameters of one task as input parameters for another
• R2.2.7 B: Both user and system default values for task parameters should be supported
• R2.2.8 B: Naming convention for parameters should be defined
• R2.2.9 B: It should be possible to use values previously given as default for parameters of a given task
• R2.2.10 B: Range and type checking of input parameters should be supported
• R2.2.10 C: It is recommended to establish a directory for parameter names

Scripting

• R2.3.1 A: Both interactive and batch mode execution must be supported
• R2.3.2 A: Standard flow control like looping, branching and conditional executing must be available
• R2.3.3 A: It must be possible to use multiple scripting languages
• R2.3.4 A: It must be possible for users to define procedures
• R2.3.5 A: It must be possible to save the state of an interactive session and later restore it for continued execution
• R2.3.6 B: Usage of both local and global variables should be possible
• R2.3.7 B: It should be possible to make procedures thread safe
• R2.3.8 B: It should be easy to turn an interactive session into a procedure which can be batch executed
• R2.3.9 B: It should be possible to execute procedures both in foreground and background (i.e. synchronous or asynchronous)
• R2.3.10 B: The scripting environment should support common astronomical conventions such as projective geometries and magnitudes
• R2.3.11 B: The scripting environment should allow support of application dependent conventions (e.g. time, wavelength specification conventions)

Scalability

• R2.4.1 A: The system must be scalable
• R2.4.2 A: Parallel execution of tasks must be possible advantage of modern hardware, the software structure of both environment and
• R2.4.3 A: It must be possible to re-synchronize execution streams
• R2.4.4 A: Monitoring of tasks must be supported
• R2.4.5 A: Location of execution of a task must be transparent to the users
• R2.4.6 A: Support for multi-CPU and cluster systems must be provided
• R2.4.7 A: Resource locking with a task's ability to suspend until a resource becomes available must be supported
• R2.4.8 A: It must be possible to suspend or abort processes externally
• R2.4.9 B: It should be possible to specify where tasks will be executed
• R2.4.10 B: Grid systems should be supported
• R2.4.11 B: For access to remote resources basic security and privilege authentication should be supported
• R2.4.12 B: It should be possible to specify resources for a task (e.g. hardware)
• R2.4.13 B: It should be possible to pass parameters to executing process
• R2.4.14 B: It should be possible to execute several instances of the system in parallel

Graphical User Interfaces

• R2.5.1 A: It must be possible to support GUI's
• R2.5.2 B: GUI's may be available for execution of high level tasks
• R2.5.3 B: It should be possible to record actions being done through a GUI (i.e. logging) and turn them into a procedure
• R2.5.4 C: Definition of work-flows through a GUI may be considered
• R2.5.5 C: It should be possible to run GUI's on a system (e.g. disk-less workstations) different from that/those on which actual processing is done
• R2.5.6 C: Support for GUI's on a wide range of platforms is desirable

Meta-data

• R3.1.1 A: Physical units in SI plus IAU recommended units must be supported (see WCS Paper I)
• R3.1.2 B: It should be possible to associate meta-data to all numeric quantities of scientific interest
• R3.1.3 B: Processing of physical units should be supported
• R3.1.4 B: It should be possible to use powers of 10 scaling factors
• R3.1.5 B: Quality flags indicating explicit quality groups should be supported
• R3.1.6 C: Errors related to a specified statistical model may be supported
• R3.1.7 C: Pixel response functions may be supported (e.g. definition of sensitivity within a pixel)

Astronomical coordinates

• R3.2.1 A: The general astronomical coordinate systems must be supported
• R3.2.2 B: Standard celestial coordinates should be supported
• R3.2.3 B: Frequency, wavelength and energy representation of spectra should be supported
• R3.2.4 B: Standard time reference systems should be supported
• R3.2.5 B: Solar and planetary coordinate systems should be supported
• R3.2.6 C: Transformation of data to different coordinate, time and unit systems may be supported

Grouping of data

• R3.3.1 A: It must be possible to associate data with different types together and refer to them as a single entity
• R3.3.2 A: Collections of similar group entities must be supported
• R3.3.3 A: It must be possible to make group entities persistent either through files (e.g. FITS tables) or databases
• R3.3.4 A: It must be possible to support a unique key for group entities
• R3.3.5 A: It must be possible to select subsets of collections
• R3.3.6 A: It must be possible to define data collections depending on their meta-data
• R3.3.7 B: It should be possible to associate an identifier to a group entity
• R3.3.8 B: It should be possible to define relationships
• R3.3.9 B: It should be easy to perform tasks for all members of a collection

Manipulation of data

• R3.4.1 A: Standard mathematical operations and functions must be provided for all relevant data
• R3.4.2 B: Standard statistical functions, tests and robust estimators should be available
• R3.4.3 B: Iteration over arrays and collections should be efficient

Data items

• R3.5.1 A: Numeric data items must support scalars and arrays
• R3.5.2 A: Standard data types must be available such as integer, real, complex, boolean
• R3.5.3 A: Values of at least 64-bit precision must be available
• R3.5.4 A: Multi-dimensional arrays must be supported
• R3.5.5 A: Strings must be supported

Uncertainties

• R3.6.1 A: Propagation of errors and quality flags must be supported
• R3.6.2 A: Handling of undefined values must be consistent
• R3.6.3 C: It may be possible to specify the statistical model for error propagation

Location of data

• R3.7.1 B: Exact location of data should be transparent to the user
• R3.7.2 B: It should be possible to acquire information on the location of data
• R3.7.3 B: It should be possible to request that processing of data is done at their location to ensure minimum usage of networks
• R3.7.4 B: It should be possible to search specified locations for relevant data
• R3.7.5 B: It should be possible to specify the location of data sets created

Reference data

• R3.8.1 A: It must be possible to define collections of reference data (e.g. standard photometry or spectra) which can be read concurrently by multiple users
• R3.8.2 A: Changes of such reference data must be logged and available to users
• R3.8.3 C: Multiple copies of reference data may exist
• R3.8.4 C: It may be possible to define preferences for reference data used

Shared access to data

• R3.9.1 A: Access to data must be easy and support sharing
• R3.9.2 A: It must be possible for well defined sets of users to share data depending on roles and privileges
• R3.9.3 A: It must be possible to protect data against multiple concurrent write accesses
• R3.9.4 A: There must not be imposed limits on size of data sets when handling and analyzing them
• R3.9.5 A: It must be possible to lock data sets or parts of them
• R3.9.6 B: Access to read-only data sets should be possible
• R3.9.7 B: A standard for adding history records should be defined
• R3.9.8 B: It should be possible to associated a version to a data set
• R3.9.9 C: History of all changes to data sets may be recorded and associated to them

Access to code

• R4.1.1 B: General science packages should allow free access to source code, re-distribution, and code modification
• R4.1.2 B: All scientific packages should be available as open source
• R4.1.3 B: Contributed science packages should be made available with permission to access to source code, re-distribution, and modify code

New applications

• R4.2.1 A: It must be easy for astronomers with no special computer science background to add new application code
• R4.2.2 A: A simple, well defined package structure must be defined for astronomical applications
• R4.2.3 B: Tutorials on writing simple applications should be provided
• R4.2.4 B: Contributed packages should only be accepted into the standard system if they contain full documentation and regression tests

The current version is also available as FaseReq_098.pdf.

-- BiancaGarilli, PrebenGrosbol, PeterLinde, DanielPonz - 20 Jun 2005

-- PrebenGrosbol - 24 Jan 2008

to top