From The ESO/ST-ECF Archive
Reading Optical Media produced by the ESO/ST-ECF archive.
Which optical media do we distribute?
For our data distribution needs (but also for our archival needs) we
have chosen the CD-R (for low capacity needs) and DVD-R (for
medium size requests). Why did we chose the DVD-R and not any of the
other flavors? The answers are quite easy.
First of all, DVD-R is around since quite a while and we have had a
chance to experiment with it quite a bit.
Secondly the alternatives are not very many actually: DVD-RAM is
not so widely used and until recently was only available in caddies
for which one has to have a special reader. As this special reader is
also a writer, it is more expensive. As we did not want to impose a
particular reader type or brand to our users, DVD-RAM was out. DVD+R
has just recently been made available. So we haven't made any
experiment with this format and all the compatibility tests would have
to be made...
Finally, readers that accomodate CD, CD-R, CD-ROM, DVD, DVD-ROM,
DVD-R (and even DVD+R for some recent cases) have become very
cheap. To the point that it is difficult to get any new computer
ISO9660 vs UDF
DVD-R and CD-R media recorded at the ST-ECF/ESO archive archive are
recorded with the CD-ROM standard: Due to the lack of support for the
UDF format in the
Unix world (e.g., only Linux 2.4, Solaris 8), we have decided to
provide our disks with ISO-9660 + Rockridge + Joliet file systems,
providing support for both the Unix and Windows worlds. As a matter of
fact this "old" CD format is specified for file systems vastly
exceeding the capacity of a DVD. So our solution is mostly practical,
trying to provide a "lowest common denominator" for our data
Volume size limitation
For the time being and for the reasons explained in the Operating
Systems notes sections, we have also decided to limit the size of each
DVD-R volume to a value smaller or equal to 4GB.
Media in use
The media we use are the so-called "General Use v2.0" media. They are
readable by most DVD readers, as listed in the recommended table
The media we currently use in the ESO/ST-ECF archive are the
following. Please note that we do not claim that this is what we will
continue to use in the future. It will depend on price policies and
technical reliability of the media. The current decision criteria
include technical reliability, price and convenience of operation. The
table below will be adapted to media supporting the fastest writer
speed in use.
||Long strategy type (Cyanine, AZO or similar)
||Bulk version, silver, shiny, unbranded
||JH 471 NX 0J
||Gold, shiny, bulk version
Operating System notes.
Below, you will find a set of notes relating to reading our DVD-R with
various OS versions. This list is far from complete and is very
generic. We would like to invite comments from users who would like to
share their own experience with us. Please write to help us keep this page as
up-to-date as possible.
Sun Solaris 2.6
Without Sun Patch 105486-07 (Sparc) or 105487-07 (Intel) the 4.7 DVDs
cannot be read without problems (every file recorded beyond a 4GB
limit on the media will look corrupt. Moreover, if the patch mentioned
above is applied, you must choose a device which will allow you
to set its block size to 512 bytes/block. In general, only the models
from Pioneer or Toshiba will allow you to do this.
should not have to worry about this as we are not planning to
distribute any disk going beyond the 4GB limit at this point.
Sun Solaris 2.8
Without Sun Patch 109764-04 (Sparc) or 109765-04 (Intel) the 4.7 DVDs
cannot be read without problems (every file recorded beyond a 4GB
limit on the media will look corrupt.
It seems like versions of Linux with kernel beyond 2.4 are ok. Older
versions will be plagued by a 4GB limit problem similar to the one
experienced with Solaris 2.6.
This operating system is really troublesome for reading CDs and
DVDs. It is suffering from the 4GB limit and will have other issues
related to the file names:
A simple mount will truncate the file names and append ";1" at
the end of it.
"mount -o cdcase" will work but limit the file names to 30 characters.
"pfsmount" will solve the above problems, but will result in
miserable read performance as it is using nfs internally to provide
the content of the CD to the user.
The above is still true for the current version of HP-UX: 11.0
For the purpose of testing the compatibility of our DVD-R with the
various computer and operating systems we did purchase a number of
drives and tested them. Some in the list are already obsolete and will
not be available anymore. We will not be able to continue this policy
of purchasing the new DVD reader models. However, we would like to
hear about your experiences with new, unlisted devices. When reporting
positive or negative tests please always include such useful
information like vendor, model, OS version used for the test etc.
||This is an old device. It only recognizes the authoring DVD-R
media. This is primarily a DVD-RAM device. It is however in use in
our ASM jukebox
||This is an old device to be found in Dell laptops from 4 years
ago. It only recognizes the authoring DVD-R media.
||The device was tested with Solaris system through an IDE-to-SCSI
||This is primarily a DVD-RAM device.
||Model found in recent Sun machines
||Attention. Older versions had always 1-5 files with read error
||DRD-8160B Revision: "1.00"
||Tested in Linux PC (RedHat Linux 7.1 Kernel 2.4.2-2) with no
How Do We Produce Our DVD-R?
We are more and more frequently getting questions regarding the way we
produce our DVD-Rs. Those questions have to do with the equipment we
use, the software in place, and the difficulties we have encountered
in the process.
It has indeed been a rough road to take us from CDs to DVDs. Starting
in 1999 we bought the first DVD-R recorder (the Pioneer S101), a very
expensive and -in the beginning- unreliable device.
As soon as a
new model became available, we immediately purchased it and this (much
more reliable) model became our workhorse for about 2 years. This is
the Pioneer S201 recorder. We still operate about 6
such drives on the Observatory sites. They have been writing several
thousands disks already. Meanwhile, we have upgraded our headquarters
systems to newer recorders.
Presently we are
operationaly using the newest Pioneer A04
device. This latter recorder presents a number of significant
advantage, the most important one is that it is almost an order of
magnitude cheaper than the older S201. Another reason why we are
making use of it, is its double-speed recording feature. But as with
the nicest roses, it has its disadvantages: As showed in the picture,
we want to operate such devices in robotic systems. If this is fine
when the recorder's interface is SCSI, it is not true anymore if the
interface is ATAPI. Due to cable length restriction, the device has to
be with the host computer's enclosure. The theoretical solution to
this problem was to use a so-called IDE-SCSI bridge. However, those bridges do not
correctly implement the SCSI protocol and our recording software (see
below) had to be adapted to bypass the problems.
We are now about to receive our newest device, a Pioneer
A-05 device which we will again operate with a IDE-to-SCSI
adapter card. This device reportedly writes DVD-Rs at the 4x speed, a
big advantage for our operation. However, that speed requires us to
use a faster SCSI adapter card, as the old model seems to limit the
transfer speed to 2.5-3.0 MB/s. The new card is the model "Acard
AEC-7722LVD U2W" and will cost roughly 136 EUR + Tax. This device
will also require new media supporting the 4x speed (about 5MB/s).
As a summary I have prepared a table with the various recorder/media
combination that we have been working with since the beginning. Their
media compatibility is included for comparison, together with their
respective prices, as of end of 2002
Back in 1999, we contacted Jörg Schilling the author of
the famous cdrecord
software and we challenged him to produce a DVD-R compatible version
of his software for our platform of choice: Sun Solaris. He managed
this in a very short time and we basically quickly had a running
As at the time, UDF support was non-existent in the Unix world, we
chose ISO9660 as our baseline file system for our media. This choice
is also explained at the top of the page.
Presently J. Schilling has newer versions of his software
(cdrecord-proDVD) that deal with all devices mentionned above. The
software is reportedly available for download.
I have also received a report from Nicolae
Mihalache who has written an ad hoc driver for the Pioneer A03 and
included it in the standard public version of cdrecord. For those
interested, I include a link to his page advertising the cdrecord
The disks produced have been used for both data archive and data
distribution purposes. We purchased soon after a DVD jukebox
from the company ASM. It can contain 1087 media (in caddies) and is
using one of the earliest drive capable of reading DVD-R: the Panasonic LF-D101. This jukebox is
expensive and voluminous but at the time, there was not really much in
the way of alternatives. Our next jukeboxes have been the Pioneer DRM-7000
devices. You can watch our ASM jukebox in action (with a bit of luck)
by looking at our live
Archive Data Storage
Optical media have not recently been very efficient in delivering
increased storage capacity at a rate compatible with our own
astronomical data input evolution. As our studies have showed, the
scale weighing the cost of DVDs in jukeboxes compared with arrays of
inexpensive magnetic disks is now tilting in favor of the latter
storage systems (price/GB). Hence, we have developed a new archival
system the Next Generation Archive System (NGAS)
based on cheap Intel- or AMD-based PCs, each fitted with 8 large
capacity IDE disks and running the Linux operating system.
If archive data will probably reside on magnetic disks in the future,
data distribution cannot be easily done with such media. The better
alternative remains in this case tapes or CDs/DVDs. The favorite
medium for most archive is of course network transfer, but it is not
always doable. Hence the need for continuous support of such "hard"
media such as tape cassettes and CDs or DVDs. Moreover, our official
products (such as initial PI data distribution) has to be done on a
hard medium. For these reasons, we will continue to provide support
for data distribution on such media.