Multi-buffering and Semaphore Locking of Shared Memory

The RTD and the server support the use of multiple areas of shared memory in a multi-buffering scheme. This means that the CCD software creates several shared memory areas and cycles the image data around these as it is created. This reduces the possibility of the CCD writing to shared memory at the same time as the RTD reads from the same segment (image jitter). The number of areas of shared memory that can be created is resource dependent: for very large images it may not be possible to create more than one segment. In addition, systems have a limit on the number of shared memory areas that can be created. With Solaris 2.5, this can be altered by adding the following line to the /etc/system file :

set shmsys:shminfo_shmseg=200

As an additional insurance against this image jitter, it is possible to use semaphores to lock areas of shared memory. The use of semaphores is not mandatory (the server and the RTD are transparent to their implementation); however, convenience routines are provided in the rtdImgEvt library to create and manage shared memory and the corresponding semaphores together.

The locking scheme implemented by the RTD and the server daemon process is as follows. When the shared memory is created, a semaphore set is created, the number in the set being the same as the number of shared memory segments. Thus each member of the set corresponds to an item of shared memory, and so shared memory can be locked by setting the appropriate semaphore to a high state. Shared memory is locked with respect to the CCD; the CCD should be prevented from writing to the buffer when the semaphore corresponding to that buffer is high.

Before the CCD dumps image data to a shared memory buffer, it should set the corresponding semaphore to one. The image event is then sent to the server daemon, which increments the same semaphore by the number of RTD clients minus one. When each RTD client has finished with the data (i.e. has displayed the image) the semaphore is decremented by one. Thus when all RTD clients have finished with the data, the semaphore will be zero and the CCD software will again be free to write to the shared memory.

After the server has sent the images to the RTDs, it is up to the RTDs to ensure that each image event is dealt with, otherwise (in the case of skipped images) semaphores may be left in high states and this could lock up the CCD completely. Thus, even if an image event can not be displayed (due to a high update rate) it is essential that the image event is at least serviced to decrement the appropriate semaphore.

In addition, each semaphore has a timeout period (which is #defined in rtdSem.h). If a semaphore is set for longer than this period, it is assumed to be dead, and can be reset by the CCD. This protects against the possiblity of an RTD crashing and leaving a semaphore in a high state.

The following convenience routines exist for the benefit of the CCD software developer who wishes to interface with the shared memory locking offered by the RTD:

Routine	Description
rtdShmCreate	creates the required number of shared memory buffers, and enough semaphores to cover these.
rtdShmFill	fills the required shared memory buffer with image data, and locks this buffer with the appropriate semaphore.
rtdShmFillFirst	cycles over all currently allocated buffers, and fills the first buffer that is not locked.
rtdShmLocked	determines whether a given buffer is locked or not.
rtdShmStruct	fills the image information structure with shared memory/semaphore information prior to being sent to the server.
rtdShmDelete	removes semaphores and shared memory buffers from the system.

To support this additional functionality, the following fields have been added to the image information structure. As mentioned above, these are not mandatory.

int semId - the semaphore identifier assigned by the system,
int shmNum - the number of the shared memory buffer in the multi-buffered cycle (e.g. if five buffers were allocated, this would be a number between 0 and 4).

These are filled in automatically if the rtdShmStruct routine is used (this also fills in the shared memory ID).

An example of how these routines may be used is given below:

 
#include "rtdSem.h" 
#include "rtdImageEvent.h" 
 
#define WIDTH           128 
#define HEIGHT          128 
#define DATASIZE        16 
#define NUM_BUF         5 
 
static void generate_data(char *); 
 
void main() { 
    rtdIMAGE_EVT_HNDL  eventHndl; 
    rtdIMAGE_INFO      imageInfo; 
    rtdShm             shmInfo; 
    char               *data; 
    unsigned int       i = 0; 
 
    memset(&imageInfo, `\0', sizeof(rtdIMAGE_INFO)); 
 
    if (rtdInitImageEvt("My_CCD_Camera", &eventHndl, NULL) == RTD_ERROR) { 
        /* ... handle error ... */ 
    } 
 
    if (rtdShmCreate(NUM_BUF, &shmInfo, WIDTH, HEIGHT, DATASIZE) == -1) { 
        /* ... handle error ... */ 
    } 
 
    while ( [some condition] ) { 
        generate_data(data); 
 
        while (rtdShmFill(i, data, &shmInfo, 0) == -1) { 
            i++; 
        } 
 
        imageInfo.dataType = DATASIZE; 
        imageInfo.xPixels  = WIDTH; 
        imageInfo.yPixels  = HEIGHT; 
        imageInfo.frameX   = 0; 
        imageInfo.frameY   = 0; 
        imageInfo.frameId  = 0; 
 
        rtdShmStruct(i, &imageInfo, &shmInfo); 
 
        /* forward image event */ 
        rtdSendImageInfo(&eventHndl, &imageInfo, NULL); 
 
        sleep(1); 
 
        i = (i + 1) % NUM_BUF; 
    } 
 
    rtdShmDelete(&shmInfo); 
}

This is a very simple way of sending locked data to the server, and does not use all the functions mentioned above. The functions are designed to allow as much flexibility in the CCD software as possible.

Additional routines are specified in rtdSem.c. These are not currently useful in the CCD software as they are not consistent with the locking scheme described above. They are used in the RTD, and are described briefly below in case they become useful to the CCD at a later development stage.

Routine	Description
rtdShmServicePacket	Given an image event structure, this routine decrements the required semaphore. This is used by the RTD to service missed image events.
rtdShmDecrement	Simply decrements the required semaphore.
rtdShmReset	Simply sets the required semaphore to zero.