Overall Storage Format
Keywords: developers

#
Overall Storage Format

                   Current Bacula Tape Format
                         6 June 2001
               Version BB02 added 28 September 2002
               Version BB01 is the old deprecated format.
       A Bacula tape is composed of tape Blocks.  Each block
         has a Block header followed by the block data. Block
         Data consists of Records. Records consist of Record
         Headers followed by Record Data.
       :=======================================================:
       |                                                       |
       |                 Block Header (24 bytes)               |
       |                                                       |
       |-------------------------------------------------------|
       |                                                       |
       |              Record Header (12 bytes)                 |
       |                                                       |
       |-------------------------------------------------------|
       |                                                       |
       |                  Record Data                          |
       |                                                       |
       |-------------------------------------------------------|
       |                                                       |
       |              Record Header (12 bytes)                 |
       |                                                       |
       |-------------------------------------------------------|
       |                                                       |
       |                       ...                             |
       Block Header: the first item in each block. The format is
         shown below.
       Partial Data block: occurs if the data from a previous
         block spills over to this block (the normal case except
         for the first block on a tape). However, this partial
             data block is always preceded by a record header.
       Record Header: identifies the Volume Session, the Stream
         and the following Record Data size. See below for format.
       Record data: arbitrary binary data.
                        Block Header Format BB02
       :=======================================================:
       |              CheckSum         (uint32_t)              |
       |-------------------------------------------------------|
       |              BlockSize        (uint32_t)              |
       |-------------------------------------------------------|
       |              BlockNumber      (uint32_t)              |
       |-------------------------------------------------------|
       |              "BB02"           (char [4])              |
       |-------------------------------------------------------|
       |              VolSessionId     (uint32_t)              |
       |-------------------------------------------------------|
       |              VolSessionTime   (uint32_t)              |
       :=======================================================:
       BBO2: Serves to identify the block as a
         Bacula block and also servers as a block format identifier
         should we ever need to change the format.
       BlockSize: is the size in bytes of the block. When reading
         back a block, if the BlockSize does not agree with the
         actual size read, Bacula discards the block.
       CheckSum: a checksum for the Block.
       BlockNumber: is the sequential block number on the tape.
       VolSessionId: a unique sequential number that is assigned
                     by the Storage Daemon to a particular Job.
                     This number is sequential since the start
                     of execution of the daemon.
       VolSessionTime: the time/date that the current execution
                     of the Storage Daemon started.  It assures
                     that the combination of VolSessionId and
                     VolSessionTime is unique for all jobs
                     written to the tape, even if there was a
                     machine crash between two writes.
                      Record Header Format BB02
       :=======================================================:
       |              FileIndex        (int32_t)               |
       |-------------------------------------------------------|
       |              Stream           (int32_t)               |
       |-------------------------------------------------------|
       |              DataSize         (uint32_t)              |
       :=======================================================:
       FileIndex: a sequential file number within a job.  The
                     Storage daemon enforces this index to be
                     greater than zero and sequential.  Note,
                     however, that the File daemon may send
                     multiple Streams for the same FileIndex.
                     The Storage Daemon uses negative FileIndices
                     to identify Session Start and End labels
                     as well as the End of Volume labels.
       Stream: defined by the File daemon and is intended to be
                     used to identify separate parts of the data
                     saved for each file (attributes, file data,
                     ...).  The Storage Daemon has no idea of
                     what a Stream is or what it contains.
       DataSize: the size in bytes of the binary data record
                     that follows the Session Record header.
                     The Storage Daemon has no idea of the
                     actual contents of the binary data record.
                     For standard Unix files, the data record
                     typically contains the file attributes or
                     the file data.  For a sparse file
                     the first 64 bits of the data contains
                     the storage address for the data block.
                           Volume Label
       :=======================================================:
       |              Id               (32 bytes)              |
       |-------------------------------------------------------|
       |              VerNum           (uint32_t)              |
       |-------------------------------------------------------|
       |              label_date       (float64_t)             |
       |              label_btime      (btime_t VerNum 11      |
       |-------------------------------------------------------|
       |              label_time       (float64_t)             |
       |              write_btime      (btime_t VerNum 11      |
       |-------------------------------------------------------|
       |              write_date       (float64_t)             |
       |                  0            (float64_t) VerNum 11   |
       |-------------------------------------------------------|
       |              write_time       (float64_t)             |
       |                  0            (float64_t) VerNum 11   |
       |-------------------------------------------------------|
       |              VolName          (128 bytes)             |
       |-------------------------------------------------------|
       |              PrevVolName      (128 bytes)             |
       |-------------------------------------------------------|
       |              PoolName         (128 bytes)             |
       |-------------------------------------------------------|
       |              PoolType         (128 bytes)             |
       |-------------------------------------------------------|
       |              MediaType        (128 bytes)             |
       |-------------------------------------------------------|
       |              HostName         (128 bytes)             |
       |-------------------------------------------------------|
       |              LabelProg        (32 bytes)              |
       |-------------------------------------------------------|
       |              ProgVersion      (32 bytes)              |
       |-------------------------------------------------------|
       |              ProgDate         (32 bytes)              |
       |-------------------------------------------------------|
       :=======================================================:

       Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
       (old version also recognized:)
       Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
       LabelType (Saved in the FileIndex of the Header record).
           PRE_LABEL -1    Volume label on unwritten tape
           VOL_LABEL -2    Volume label after tape written
           EOM_LABEL -3    Label at EOM (not currently implemented)
           SOS_LABEL -4    Start of Session label (format given below)
           EOS_LABEL -5    End of Session label (format given below)
       VerNum: 11
       label_date: Julian day tape labeled
       label_time: Julian time tape labeled
       write_date: Julian date tape first used (data written)
       write_time: Julian time tape first used (data written)
       VolName: "Physical" Volume name
       PrevVolName: The VolName of the previous tape (if this tape is
                    a continuation of the previous one).
       PoolName: Pool Name
       PoolType: Pool Type
       MediaType: Media Type
       HostName: Name of host that is first writing the tape
       LabelProg: Name of the program that labeled the tape
       ProgVersion: Version of the label program
       ProgDate: Date Label program built
                           Session Label
       :=======================================================:
       |              Id               (32 bytes)              |
       |-------------------------------------------------------|
       |              VerNum           (uint32_t)              |
       |-------------------------------------------------------|
       |              JobId            (uint32_t)              |
       |-------------------------------------------------------|
       |              write_btime      (btime_t)   VerNum 11   |
       |-------------------------------------------------------|
       |                 0             (float64_t) VerNum 11   |
       |-------------------------------------------------------|
       |              PoolName         (128 bytes)             |
       |-------------------------------------------------------|
       |              PoolType         (128 bytes)             |
       |-------------------------------------------------------|
       |              JobName          (128 bytes)             |
       |-------------------------------------------------------|
       |              ClientName       (128 bytes)             |
       |-------------------------------------------------------|
       |              Job              (128 bytes)             |
       |-------------------------------------------------------|
       |              FileSetName      (128 bytes)             |
       |-------------------------------------------------------|
       |              JobType          (uint32_t)              |
       |-------------------------------------------------------|
       |              JobLevel         (uint32_t)              |
       |-------------------------------------------------------|
       |              FileSetMD5       (50 bytes)   VerNum 11  |
       |-------------------------------------------------------|
               Additional fields in End Of Session Label
       |-------------------------------------------------------|
       |              JobFiles         (uint32_t)              |
       |-------------------------------------------------------|
       |              JobBytes         (uint32_t)              |
       |-------------------------------------------------------|
       |              start_block      (uint32_t)              |
       |-------------------------------------------------------|
       |              end_block        (uint32_t)              |
       |-------------------------------------------------------|
       |              start_file       (uint32_t)              |
       |-------------------------------------------------------|
       |              end_file         (uint32_t)              |
       |-------------------------------------------------------|
       |              JobErrors        (uint32_t)              |
       |-------------------------------------------------------|
       |              JobStatus        (uint32_t) VerNum 11    |
       :=======================================================:
       * => fields deprecated
       Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
       LabelType (in FileIndex field of Header):
           EOM_LABEL -3     Label at EOM
           SOS_LABEL -4     Start of Session label
           EOS_LABEL -5     End of Session label
       VerNum: 11
       JobId: JobId
       write_btime: Bacula time/date this tape record written
       write_date: Julian date tape this record written - deprecated
       write_time: Julian time tape this record written - deprecated.
       PoolName: Pool Name
       PoolType: Pool Type
       MediaType: Media Type
       ClientName: Name of File daemon or Client writing this session
                   Not used for EOM_LABEL.