[ ESO ] [ ST-ECF ]

Optical Media
From The ESO/ST-ECF Archive

ARCHIVE HOME ESO HOME SEARCH

Reading Optical Media produced by the ESO/ST-ECF archive.

Which optical media do we distribute?

For our data distribution needs (but also for our archival needs) we have chosen the CD-R (for low capacity needs) and DVD-R (for medium size requests). Why did we chose the DVD-R and not any of the other flavors? The answers are quite easy.
  • First of all, DVD-R is around since quite a while and we have had a chance to experiment with it quite a bit.
  • Secondly the alternatives are not very many actually: DVD-RAM is not so widely used and until recently was only available in caddies for which one has to have a special reader. As this special reader is also a writer, it is more expensive. As we did not want to impose a particular reader type or brand to our users, DVD-RAM was out. DVD+R has just recently been made available. So we haven't made any experiment with this format and all the compatibility tests would have to be made...
  • Finally, readers that accomodate CD, CD-R, CD-ROM, DVD, DVD-ROM, DVD-R (and even DVD+R for some recent cases) have become very cheap. To the point that it is difficult to get any new computer without one.

    Technical Notes

    ISO9660 vs UDF

    DVD-R and CD-R media recorded at the ST-ECF/ESO archive archive are recorded with the CD-ROM standard: Due to the lack of support for the UDF format in the Unix world (e.g., only Linux 2.4, Solaris 8), we have decided to provide our disks with ISO-9660 + Rockridge + Joliet file systems, providing support for both the Unix and Windows worlds. As a matter of fact this "old" CD format is specified for file systems vastly exceeding the capacity of a DVD. So our solution is mostly practical, trying to provide a "lowest common denominator" for our data distribution need.

    Volume size limitation

    For the time being and for the reasons explained in the Operating Systems notes sections, we have also decided to limit the size of each DVD-R volume to a value smaller or equal to 4GB.

    Media in use

    The media we use are the so-called "General Use v2.0" media. They are readable by most DVD readers, as listed in the recommended table below.
    The media we currently use in the ESO/ST-ECF archive are the following. Please note that we do not claim that this is what we will continue to use in the future. It will depend on price policies and technical reliability of the media. The current decision criteria include technical reliability, price and convenience of operation. The table below will be adapted to media supporting the fastest writer speed in use.

    Medium Vendor Disk type Comment
    CD-R TDK Corporation Long strategy type (Cyanine, AZO or similar) Bulk version, silver, shiny, unbranded
    DVD-R Pioneer DVS-R470SD Standard version
    DVD-R Mitsui JH 471 NX 0J Gold, shiny, bulk version

    Operating System notes.

    Below, you will find a set of notes relating to reading our DVD-R with various OS versions. This list is far from complete and is very generic. We would like to invite comments from users who would like to share their own experience with us. Please write to help us keep this page as up-to-date as possible.

    Sun Solaris 2.6
    Without Sun Patch 105486-07 (Sparc) or 105487-07 (Intel) the 4.7 DVDs cannot be read without problems (every file recorded beyond a 4GB limit on the media will look corrupt. Moreover, if the patch mentioned above is applied, you must choose a device which will allow you to set its block size to 512 bytes/block. In general, only the models from Pioneer or Toshiba will allow you to do this.
    Again, users should not have to worry about this as we are not planning to distribute any disk going beyond the 4GB limit at this point.

    Sun Solaris 2.8
    Without Sun Patch 109764-04 (Sparc) or 109765-04 (Intel) the 4.7 DVDs cannot be read without problems (every file recorded beyond a 4GB limit on the media will look corrupt.
    Linux
    It seems like versions of Linux with kernel beyond 2.4 are ok. Older versions will be plagued by a 4GB limit problem similar to the one experienced with Solaris 2.6.
    HP-UX
    This operating system is really troublesome for reading CDs and DVDs. It is suffering from the 4GB limit and will have other issues related to the file names:
  • A simple mount will truncate the file names and append ";1" at the end of it.
  • "mount -o cdcase" will work but limit the file names to 30 characters.
  • "pfsmount" will solve the above problems, but will result in miserable read performance as it is using nfs internally to provide the content of the CD to the user. The above is still true for the current version of HP-UX: 11.0

    Readers

    For the purpose of testing the compatibility of our DVD-R with the various computer and operating systems we did purchase a number of drives and tested them. Some in the list are already obsolete and will not be available anymore. We will not be able to continue this policy of purchasing the new DVD reader models. However, we would like to hear about your experiences with new, unlisted devices. When reporting positive or negative tests please always include such useful information like vendor, model, OS version used for the test etc.

    Vendor Model I/F OK? Comment
    Panasonic LF-D101 SCSI No This is an old device. It only recognizes the authoring DVD-R media. This is primarily a DVD-RAM device. It is however in use in our ASM jukebox
    Matsushita SR-8171 ATAPI No This is an old device to be found in Dell laptops from 4 years ago. It only recognizes the authoring DVD-R media.
    NEC DV-5800A ATAPI Yes The device was tested with Solaris system through an IDE-to-SCSI bridge.
    Panasonic LF-D201 SCSI Yes This is primarily a DVD-RAM device.
    Toshiba D-M1401 SCSI Yes Model found in recent Sun machines
    Pioneer DVD-U02 SCSI Yes Slow/old
    Pioneer DVD-303 SCSI Yes Ok.
    Pioneer DVD-305 SCSI Yes Attention. Older versions had always 1-5 files with read error
    Pioneer DVD-304 SCSI Yes Ok.
    LG DRD-8160B Revision: "1.00" ATAPI Yes Tested in Linux PC (RedHat Linux 7.1 Kernel 2.4.2-2) with no SCSI bridge

    How Do We Produce Our DVD-R?

    We are more and more frequently getting questions regarding the way we produce our DVD-Rs. Those questions have to do with the equipment we use, the software in place, and the difficulties we have encountered in the process.

    Drives

    It has indeed been a rough road to take us from CDs to DVDs. Starting in 1999 we bought the first DVD-R recorder (the Pioneer S101), a very expensive and -in the beginning- unreliable device.

    As soon as a new model became available, we immediately purchased it and this (much more reliable) model became our workhorse for about 2 years. This is the Pioneer S201 recorder. We still operate about 6 such drives on the Observatory sites. They have been writing several thousands disks already. Meanwhile, we have upgraded our headquarters systems to newer recorders.

    Presently we are operationaly using the newest Pioneer A04 device. This latter recorder presents a number of significant advantage, the most important one is that it is almost an order of magnitude cheaper than the older S201. Another reason why we are making use of it, is its double-speed recording feature. But as with the nicest roses, it has its disadvantages: As showed in the picture, we want to operate such devices in robotic systems. If this is fine when the recorder's interface is SCSI, it is not true anymore if the interface is ATAPI. Due to cable length restriction, the device has to be with the host computer's enclosure. The theoretical solution to this problem was to use a so-called IDE-SCSI bridge. However, those bridges do not correctly implement the SCSI protocol and our recording software (see below) had to be adapted to bypass the problems.

    We are now about to receive our newest device, a Pioneer A-05 device which we will again operate with a IDE-to-SCSI adapter card. This device reportedly writes DVD-Rs at the 4x speed, a big advantage for our operation. However, that speed requires us to use a faster SCSI adapter card, as the old model seems to limit the transfer speed to 2.5-3.0 MB/s. The new card is the model "Acard AEC-7722LVD U2W" and will cost roughly 136 EUR + Tax. This device will also require new media supporting the 4x speed (about 5MB/s). As a summary I have prepared a table with the various recorder/media combination that we have been working with since the beginning. Their media compatibility is included for comparison, together with their respective prices, as of end of 2002

    Recorder 3.95 "Authoring" 4.7 "Authoring" 4.7 "General" Speed Price Available?
    Pioneer S101 Yes No No 1X 15KEUR No
    Pioneer S201 Yes Yes No 1X 4KEUR Yes
    Pioneer A03 No No Yes 2X 1 KEUR No longer
    Pioneer A04 No No Yes 2X 800 EUR Yes
    Pioneer A05 No No Yes 4X 400 EUR Yes

    Recording Software

    Back in 1999, we contacted Jörg Schilling the author of the famous cdrecord software and we challenged him to produce a DVD-R compatible version of his software for our platform of choice: Sun Solaris. He managed this in a very short time and we basically quickly had a running prototype.

    As at the time, UDF support was non-existent in the Unix world, we chose ISO9660 as our baseline file system for our media. This choice is also explained at the top of the page.

    Presently J. Schilling has newer versions of his software (cdrecord-proDVD) that deal with all devices mentionned above. The software is reportedly available for download. I have also received a report from Nicolae Mihalache who has written an ad hoc driver for the Pioneer A03 and included it in the standard public version of cdrecord. For those interested, I include a link to his page advertising the cdrecord patch.

    Jukeboxes

    The disks produced have been used for both data archive and data distribution purposes. We purchased soon after a DVD jukebox from the company ASM. It can contain 1087 media (in caddies) and is using one of the earliest drive capable of reading DVD-R: the Panasonic LF-D101. This jukebox is expensive and voluminous but at the time, there was not really much in the way of alternatives. Our next jukeboxes have been the Pioneer DRM-7000 devices. You can watch our ASM jukebox in action (with a bit of luck) by looking at our live webcam.

    The Future

    Archive Data Storage
    Optical media have not recently been very efficient in delivering increased storage capacity at a rate compatible with our own astronomical data input evolution. As our studies have showed, the scale weighing the cost of DVDs in jukeboxes compared with arrays of inexpensive magnetic disks is now tilting in favor of the latter storage systems (price/GB). Hence, we have developed a new archival system the Next Generation Archive System (NGAS) based on cheap Intel- or AMD-based PCs, each fitted with 8 large capacity IDE disks and running the Linux operating system.

    Data Distribution
    If archive data will probably reside on magnetic disks in the future, data distribution cannot be easily done with such media. The better alternative remains in this case tapes or CDs/DVDs. The favorite medium for most archive is of course network transfer, but it is not always doable. Hence the need for continuous support of such "hard" media such as tape cassettes and CDs or DVDs. Moreover, our official products (such as initial PI data distribution) has to be done on a hard medium. For these reasons, we will continue to provide support for data distribution on such media.

    Send comments to <bpirenne@eso.org>
    Last modified: Wed Nov 25 10:30:00 MET 2002
    ARCHIVE HOME ESO HOME SEARCH