TAP: Manage your Asynchronous Queries

This page contains simple web forms that can manage the asynchronous ADQL(1) queries sent to the ESO Tabular Access Protocol (TAP) services. Examples and information on how to control asynchronous jobs, also in a programmatic way, are provided. The starting point: you launched an asynchronous query because you know or expect that your query is going to take long time (more than 60 seconds) to execute, and therefore you want to avoid any time out.

Credit: This page is losely based on an equivalent query management page of the Space Telescope Science Institute, whose developers are gratefully thanked.

If you reached this page, very likely it is beacuse you have submitted an asynchronous query to one of the ESO TAP services. What you are looking for is a way to get to the results of your query. This page helps you acheiving that.
Please, visit the Science Archive Programmatic and Tool Access page in case you do not know how to submit an asynchronous query.
Queries can also be submitted on the command line, using wget, curl, or similar programs, as shown here:

wget 'http://archive.eso.org/tap_obs/async' --post-data 'REQUEST=doQuery&LANG=ADQL&MAXREC=2&FORMAT=text&QUERY=select%20top%202%20*%20from%20ivoa.ObsCore' -O -

curl -Lw -X POST 'http://archive.eso.org/tap_obs/async' -d 'REQUEST=doQuery&LANG=ADQL&MAXREC=2&FORMAT=text&QUERY=select%20top%202%20*%20from%20ivoa.ObsCore'

Said more technically, this page helps you managing an asynchronous job that was created behind the scenes when you submitted your query.

Assumption

The minimal job management

Using the job identifier the user:

Undeleted jobs remain available for a limited amount of time, set by default to 1 day, unless the user has specifically set a destruction datetime, which can anyway not exceed an imposed server limit of 6 months .

Two TAP services are available, as described in the table here below.

tap_obsoffers access to the tables related to science data (both raw and reduced) and ambient conditions of the La Silla, Paranal, and Chajnantor Observatory.
tap_catoffers access to the scientific catalogs provided by the principal investigators of the ESO observing programmes (public survey catalogs, etc.).

Asynchronous Query (using the Universal Worker Service (UWS))

Asynchronous queries are managed using the UWS(3) protocol, which allows queries with much larger result sets to be efficiently managed by the service. In short, they are set up by the client, potentially negotiated with regards to the amount of time they are allowed to run or be stored on the server, then run by the client. The client then polls the server to discover when the query has run and where potential error or completed result sets are stored.

This is all managed via a 'job ID'. To run a UWS query, first submit your ADQL-based query. Given the RESTful nature of UWS as implemented, you will be forwarded to a status page with a job number. Copy down this job number - you will use it for the rest of the lifetime of your query. You can also query the UWS engine for a list of currently existent jobs.


Asynchronous Query Management

Once you have a job number, you can enter it in the forms below to either renegotiate the deletion time of your results (if you expect it to take a long time to run, or will be personally unable to check up on your results before they would be automatically deleted), to trigger the job to be run, to poll your job for another status page giving the URL of its result set or error documentation, and to delete your job when it is finished. Job deletion will bring you back to the job summary page.

First of all, choose the TAP service you submitted your job to:

The main URL for programmatic access to the TAP service is:

Enter job number to run:
Enter job number to check its status:
Enter job number to change
its execution duration: 		New execution duration integer number of seconds
This action can be taken only before running a job.
Enter job number to change
maxrec of: 		New MAXREC
This action can be taken only before running a job.
Enter job number to edit the
query of: 		Corrected query
This action can be taken only before running a job.
Enter job number to delay
its destruction: 		New destruction GMT time (yyyy-mm-ddThh:mm:ss)
This action can be taken only before running a job.
Enter job number to view results or errors (if they exist):
Enter job number to abort:
Enter job number to delete:

Controlling programmatically your asynchronous queries

Instead of using the web forms provided in the previous section, you might prefer to control your jobs programmatically. This section shows you how to achieve that via few examples. You do not need to necessarily use wget, the important thing is that you use http POST when modifying the job (i.e. modifying its status or its parameters), and that you use http GET when accessing read-only the job (e.g. to get to know its status, or to retrieve the results).
Full explanation of the programmatic interface is to be found in the Universal Worker Service Pattern Version 1.1 IVOA standard.

Examples:

In the examples here below, it is assumed that you submitted an asynchronous query, and that the job identifier assigned to it was: 12345678.

Change it with the number provided in the initial answer you received when submitting your asynchronous query.

RUN: wget -O - 'http://archive.eso.org/tap_obs/async/12345678' --post-data 'PHASE=RUN'
STATUS: wget -O - 'http://archive.eso.org/tap_obs/async/12345678'
EXECUTIONDURATION: wget -O - 'http://archive.eso.org/tap_obs/async/12345678' --post-data 'executionduration=120'
SET DESTRUCTION TIME: wget -O - 'http://archive.eso.org/tap_obs/async/12345678' --post-data 'destruction=2018-09-01T00:00:00'
RESULTS: wget -O - 'http://archive.eso.org/tap_obs/async/12345678/results/result'
ABORT: wget -O - 'http://archive.eso.org/tap_obs/async/12345678' --post-data 'PHASE=ABORT'
DELETE: wget -O - 'http://archive.eso.org/tap_obs/async/12345678' --post-data 'ACTION=DELETE'

Equivalent curl command would look like this:
curl -Lw -X POST 'http://archive.eso.org/tap_obs/async/12345678' -d 'PHASE=RUN'

References

The following links may help to explain the basic query language and systems at work.

  1. ADQL - Astronomical Data Query Language.
  2. TAP - Table Access Protocol. This is the underlying standard behind ObsTAP, or Observation Data Model / TAP, services. All queries built using the forms on this page are formatted for TAP. The standard provides a basic query mechanism, a mechanism for schema-related configuration queries, and VOSI support.
  3. UWS - Universal Worker Service for asynchronous web service communication. We have implemented this using REST protocols as explained in an IVOA note.
  4. VOSI - Virtual Observatory Standard Interface. This defines a protocol for discovering basic uptime, capability, descriptive, and schema-related information about a service. We have implemented the REST version as explained in the VOSI 1.0 documentation.
    5. This can be used for SQL-like string queries to the service
Last modification date of Manage your Asynchronous Queries: 2019-02-20