The programmatic layer of the ESO science archive has been recently upgraded. The main new feature is the support of authorized data searches, but there are other improvements useful to the all users of the ESO programmatic layer.

Authorised data searches are specifically useful to operations and GTO teams (e.g., SPECULOOS, 4MOST, NIRPS, SOXS, SPHERE and ESPRESSO GTO) who have been granted specific permissions to search and access data from observations subject to metadata protection, as per GTO policy (currently concerned some SPHERE and some ESPRESSO GTO runs).

Both the Table Access Protocol [TAP] for observational data and atmospheric measurements (tap_obs), and the DataLink services now support authentication and authorization. The TAP service for catalogue data (tap_cat) does not require this new functionality, as authorisation is currently not needed for this type of data.

Please refer to the dedicated Authentication & Authorisation page and to an accompanying python jupyter notebook to learn more.

TAP

While developing this new functionality, many other aspects were tackled, resulting in an overall better TAP service for all users.  The TAP software has been upgraded to the version 1.1 of the IVOA Table Access Protocol standard (list of changes are described in the TAP1.1 standard, in the Appendix A "Changes from TAP-1.0 to TAP-1.1"). Version 1.1 of the standard allows proper handling of arrays in table cells; this has a positive impact on both tap_obs and tap_cat:

• on tap_obs, the dbo.ssa table behind the SSAP service can finally provide the spatial position using an array, as mandatorily demanded by the SSA standard;
• on tap_cat, the support of arrays will be a fundamental stepping stone for the publishing of 4MOST catalogues which will use arrays to store probability distribution functions.

Other improvements, relevant to all users, also those running anonymous queries, are detailed in the programmatic change log page, and include:

• New output formats
  
• New columns added to the tables of reduced data (ivoa.ObsCore);
• Ability to search for products whose metadata have changed in a given period of time, using the new last_mod_date column (ivoa.ObsCore table of the tap_obs service, while dbo.raw already supported this);
• Various fields were curated (better content).

DataLink

Authorised DataLink access, via token authentication, now supports observations whose metadata are private. Users can therefore exercise datalink also on those, either to find related data files (e.g. calibration files), or to download cutouts of private products. This new functionality is of interest only to users which have been granted special permissions onto the related archive assets. Note: datalink support for the ESPRESSO quicklook products (QC0) is being worked upon and should be delivered soon.