

                       eXtended BIN Format specification
                      -----------------------------------

                   Written by Tasmanic Tasmaniac / ACiD 1996
                     No rights reserved, No patent pending
                     For comments/questions : Tas@acid.org
                   http://uc2.unicall.be/tasmaniac/xbin.htm


  Introduction
  ------------

  Out of a crying need from several ANSi artists, a new type of file is
  born and ready to take the Art scene by storm.

  This new format is known as eXtended BIN or XBIN for short.

  XBIN is what it's name dictates, it's an extention to the normal raw-image
  BIN files which have become very popular among the art-scene lately.

  The use of the XBIN format is more or less the same as for the BIN format.
  However, XBIN offers a far better way to handle the the raw images.


  BIN vs XBIN
  ------------

  The BIN format was introduced into the art-scene out of a need to overcome
  the limits of ANSi files.  Apparantly, the 80 columns wide screen was too
  constraining for some artists.  The BIN format was adopted to resolve this
  problem.

  Being very simple in nature, BIN was quickly supported by several art
  groups in their native ANSI/RIP/GIF/etc. viewer.  Consequently, our very
  own SAUCE standard went in for a quick facelift and immediately dealt with
  one of the main problems imposed by the BIN format.

  Being nothing more than a raw memory copy of the textmode videomemory, BIN
  offers no insight to the size/width of the image.  Having nothing more than
  a BIN file, there is no way to determine whether it's a 80 column wide or
  a 160 column wide image.  SAUCE took care of this by taking the BIN format
  into it's specifications.  Out of the SAUCE attached to the BIN, one was
  able to determine the correct dimensions of the BIN.

  XBIN solves this little matter all by itself, and takes matters even
  further.  Anything BIN can do, XBIN does better.


  Features
  ---------

  XBIN allows for binary images up to 65536 columns wide, and 65536 lines
  high.

  XBIN allows for an alternate set of palette colors either in blink or
  non-blink mode.

  XBIN allows for different textmode fonts from 1 to 32 scanlines high,
  consisting of either 256 or 512 different characters.

  And finally, XBIN offers smaller files due to a simple yet very efficient
  compression system.


  Goal
  -----

  XBIN was designed with certain rules in mind.  These were:

  - The format may not have a group-specific name.  It should be as
    anonymous as possible.

  - The format must be simple to implement, yet offer as much functionality
    as possible.

  - The format must be universally adoptable, a prerequisite for this is the
    complete availability of the formats specifications.

  - Availability of direct plug-in code, tools to read/write/convert the
    format and the immediate support in practical applications.

  The above simple four rules were similarly used when designing the
  specifications of SAUCE.  These basic principles helped get the acceptance
  of SAUCE as a standard by the majority of art groups worldwide.  The
  premise of SAUCE's success was reason alone to re-use this set of rules
  while designing XBIN.

  More specifically to the practical use of XBIN, following constraints and
  ideas needed to be addressed :

  - Provide an unambiguous system for how the file should be displayed.

  - Be as complete as possible to avoid having to make changes to the
    specifications.  Since XBIN has a very specific use (textmode 'graphics')
    it should not be too difficult to provide for all/most of what textmode
    has to offer.
    In short XBIN should provide for any possible feature possible for
    textmode graphics, but no more.
    In addition, XBIN is designed for VGA.  Several of the features XBIN
    offers can only be used on a VGA system.

    To sum things up, XBIN was required to handle :

    - Any reasonable sized image
    - Alternate color sets (palettes)
    - Alternate fonts of any size  (due to limitations of the VGA hardware
      this means sizes of 1 up to 32 pixels), for practical reasons, only 8
      pixel wide fonts are supported, even though the VGA hardware has other
      capabilities.
    - Blink and non-blink mode
    - 256 and 512 characters

  In addition to the set requirements, XBIN also offers a simple compression
  scheme which does remarkably well seen it's simplicity.


  On to the Specs...
  -------------------

  Ok, now that we have you totally brainwashed, and drooling, it's time for
  dealing out the specs.

  First of all, the default file extention of an XBIN file is '.XB', just as
  a BIN file by default has '.BIN' as extention.  Even though long filenames
  are getting popular due to newer operating systems like Windows 95,
  Windows NT, OS/2, Unix, Linux... there was specifically chosen for .XB
  over .XBIN

  An XBIN consists of 4 main parts, a header (required), a palette
  (optional), a font (optional), and the image data (optional).

    Ŀ
     Header      Required
    Ĵ
     Palette     Optional
    Ĵ
     Font        Optional
    Ĵ
     Image Data  Optional
    

  ! NOTE !
  An XBIN is not REQUIRED to have an image.  You can use the standard XBIN
  format for storing pre-made fonts and pre-made palettes which you could
  load from a viewer/editor (as alternate font/palette)


    Header
    ------
    The XBIN header consists of 11 bytes.  The header describes the size of
    the image, how the screen should be set up, and how the rest of the
    XBIN should be processed.

    Record layout :

    Name       Size Type    Purpose
    ---------- ---- ------- -------------------------------------------------
    ID         4    Char    XBIN identification, these 4 bytes should contain
                            the text "XBIN".  Any file which does not have a
                            matching ID should not be considered to be an
                            XBIN file.
    EofChar    1    Char    End of file character (Ctrl-Z, Ascii 26, 1A hex)
                            When a user uses type to view the file, he'll
                            see "XBIN" printed on screen.
    Width      2    Numeric Width of the image in character columns.
    Height     2    Numeric Height of the image in character rows.
    Fontsize   1    Numeric Number of pixel rows (scanlines) in the font,
                            Default value for VGA is 16.
                            Any value from 1 to 32 is technically possible on
                            VGA.  Any other values should be considered
                            illegal.
    Flags      1    Bits    A set of flags indicating special features in the
                            XBIN file.  More on this later.

    A sample XBIN header in Pascal could be :

          TYPE  XB_Header = RECORD
                              ID      : ARRAY[0..3] OF Char;
                              EofChar : Byte;
                              Width   : Word;
                              Height  : Word;
                              Fontsize: Byte;
                              Flags   : Byte;
                            END;

    A sample XBIN header in C could be :

          typedef struct XB_Header {
             unsigned char  ID[4];
             unsigned char  EofChar;
             unsigned short Width;
             unsigned short Height;
             unsigned char  Fontsize;
             unsigned char  Flags;
          };


    Flags
    -----
    The 'flags' field in the XBIN header tells us more on special features
    the XBIN has in use.  The flags field consists out of 8 separate bits
    each with it's unique function.
    Bits which are 0, are considered OFF or disabled, Bits with 1 are
    considered ON or enabled.

    Bit                 Name     Purpose
    ------------------- -------- --------------------------------------------
    [ . . . . . . . X ] Palette  When 1, A palette is present in the XBIN.
                                 When 0, Default palette applies.
    [ . . . . . . X . ] Font     When 1, A Font is present in the XBIN.
                                 When 0, Default font applies, since the VGA
                                 default fontsize is 16, This bit should be 1
                                 for other sizes.  In consequence, a font
                                 should then also be present.
    [ . . . . . X . . ] Compress When 1, XBIN compression is used
                                 When 0, No compression is used, the image
                                 is stored in raw memory format.
    [ . . . . X . . . ] NonBlink When 1, Image should be shown in non-blink
                                 mode.
                                 When 0, Image should be shown in blink mode.
    [ . . . X . . . . ] 512Chars When 1, The image is built up out of 512
                                 characters in stead of the usual 256.
                                 This bit also requires the Font bit to be
                                 set since the VGA has no default font for
                                 512 character mode.
                                 The font in consequence then has 512
                                 characters in stead of the usual 256.
    [ X X X . . . . . ] Unused   Unused, should be all zeroes.


    Palette
    --------
    A palette is only present when the 'Palette' bit is set in the Flags
    field of the XBIN header.
    The palette is built up of 48 bytes, a red, green and blue value (in
    that order) for each of the 16 colors.
    Each palette value can range from 0 to 63.


    Font
    -----
    A Font is only present when the 'Font' bit is set in the Flags field of
    the XBIN header.
    For each character (256, or 512 when the '512Chars' bit is set in the
    Flags field) FontSize bytes are stored in sequence.  The character set
    is defined from the top row of each character matrix to the bottom row.

    In a 16 pixel high font, the first 16 bytes are the fontmatrix for ascii
    value 0, the next 16 are for ascii 1 and so on.  In total, a 16 pixel
    font would have a font of 4096 bytes (16*256)
    Technically, the biggest font possible would be 16Kb in size. (32 pixels
    high, 512 characters), and the smallest would be 256 bytes (1 pixel high,
    256 characters).


    Image data
    -----------
    The image data is a raw image of video memory.  Each character consists
    of 2 bytes, the first being the character, the second being the attribute
    (color).
    The Image data would thus be equal to Width*Height*2.  The biggest XBIN
    would be a whopping 8Gb (65535*65535*2) in size and the smallest would be
    0 bytes (0*0*2).

    Unless the 'Compress' bit is set in the Flags field, Image data is stored
    in the exact way you would need it in video memory.

    When the 'Compress' bit is set, Image data is compressed with
    XBIN-Compression.  This is a fairly simple compression system, which
    should pose no real difficulty to decompress.  As compression goes
    however an XBIN compressor is a little harder to write <g>.


    XBIN Compression
    ----------------
    The XBIN compression uses a slightly improved Run-Length Encoding scheme
    which will do very well on this type of data.

    In stead of describing how a compressor would work, I'll explain how the
    compression works by giving some examples.  In these examples, you'll see
    strings of characters more or less like this one :

         Aa,Ab,Ac,Ba,Bb,Bc,De,Zx,Yu

    This string represents a part of the uncompressed data. The capital
    lettres are character bytes, the lower case letters are attribute bytes.

    OK, pay close attention, as things may get hairy now ;-)

    When you examine an ANSi or a BIN file, you may or may not have noticed
    several characteristics which are typical for ANSi/BIN files.
    You see sequences of identical characters one after another, and you see
    sequences of identical colors one after another.  You may even see the
    combination of both; identical characters in identical colors one after
    another.

    XBIN-compression makes use of these characteristics by replacing
    sequences of identical characters/color with a counter and the actual
    data.
    A sequence like :

        Aa,Aa,Aa,Aa,Aa,Aa,Aa,Aa,Aa,Bb,Bb,Bb,Bb

    could easily be replaced with

        [Repeat 9 times]Aa,[Repeat 4 times]Bb

    The '[Repeat x times]' tag is the repeat counter.

    If you're a smart observer, you've probably already figured where I'm
    driving at.  It shouldn't be too hard to figure out that there's a need
    for four different types of compression :
      1) No compression  (when two subsequent character/attribute pairs have
                          no relation)
      2) character compression  (for a sequence of identical characters in
                                 different colors)
      3) attribute compression  (for a sequence of different characters with
                                 identical attributes)
      4) character/attribute compression (for a sequence of identical
                                          character/attributes).

    Hmm.. 4 types of compression.. That would nicely fit in 2 bits, leaving
    6 bits in a byte unused.  Now.. what if we were to use those 6 bits for
    the repeat counter...
    This is exactly what XBIN compression does...

    The XBIN compression consists out of a sequence of repeat counters
    followed by the appropriate number of data bytes.

    Ok, before we go any further now, allow me to make an important note.
    XBIN compression works on a ROW by ROW basis.  The compression does NOT
    carry through to the next line.  so if you would have for example two
    lines like this :
                   Ab,Aa,Aa,Aa
                   Aa,Aa,Aa,Ab
    It should NOT be encoded as
                   Ab,[Repeat 6 Times]Aa,Ab
    but SHOULD be encoded as
                   Ab,[Repeat 3 Times]Aa
                   [Repeat 3 Times]Aa,Ab
    The are several reasons why it should work like this
       1) on-the-fly decoding is facilitated (more on this later)
       2) You could run into a problem with odd XBIN widths since video
          memory memory always has an even width.  You'd have to implement
          special code to skip over this one odd byte.
       3) You at least have SOME way of detecting errors in the XBIN file.
          Whenever a line doesn't nicely work out to the required width
          there must be some error in the XBIN file (or a bug in your
          decoding routine ;-))


    XBIN Compression continued
    ---------------------------
    The repeat counter is split up in two parts, the two most significant
    bits are the compression type, the six least significant bits are the
    actual repeat counter.

    [ X X . . . . . . ]  Compression type
                         00 : No compression
                         01 : Character compression
                         10 : Attribute compression
                         11 : Character/Attribute compression.
    [ . . X X X X X X ]  Repeat counter.
                         Since a repeat of 0 is downright useless, we'll
                         store the repeat counter as one less of it's actual
                         number of repeats.
                         A value of 0 indicates a repeat value of 1.
                         A value of 1 indicates a repeat value of 2.
                         And so on, giving us a maximum
    In the examples, the Repeat counter byte will be used as [Type,Count]
    Where Type is 00, 01, 10 or 11, and Count is a number from 0 to 63.
    For example, [01,10] would mean, Attribute compression with a repeat
    counter of 10 (11 effective repeats)

   * Compression type 00 : No compression
      This type of compression is needed whenever two or more sequences of
      character/attribute pairs have nothing in common.
      The repeat counter is followed by the appropriate number of
      character/attribute pairs.

      Data            : AaBbCcDdEeFfGg
      XBin Compressed : [00,6]AaBbCcDdEeFfGg
                             Remember value is ONE less than the actual
                                number of repeats wanted.

      In a worst-case situation where you would have to use No compression on
      the entire file, this would mean the 'compressed' data is bigger as the
      non-compressed data, in this situation, the best thing to do would be
      to store the image data uncompressed, and set the 'Compress' bit in the
      Flags field to 0.

   * Compression type 01 : Character compression
      This type of compression is used whenever a sequence of identical
      characters is found, but where the attribute changes.
      The repeat counter is followed by the character to use which is in turn
      followed by the appropriate number of attribute bytes.

      Data            : AaAbAcAdAeAfAg
      XBIN Compressed : [01,6]Aabcdefg

   * Compression type 10 : Attribute compression
      This type of compression is used whenever a sequence of identical
      attributes is found, but where the character changes.
      The repeat counter is followed by the attribute to use which is in turn
      followed by the appropriate number of character bytes.


      Data            : AaBaCaDaEaFaGa
      XBIN Compressed : [10,6]aABCDEFG

      ! Note this is the only time you'll see the attribute byte BEFORE the
        characters.

   * Compression type 11 : Character/Attribute compression
      This type of compression is used whenever a sequence of identical
      character attribute pairs is found.
      The repeat counter is followed by the character attribute pair.

      Data            : AaAaAaAaAaAaAa
      XBIN Compressed : [11,6]Aa

    You'll find example code on how to do XBIN compression and decompression
    lateron in the XBIN.TUT document.


  XBIN freebies
  --------------
  Alongside with this document you should find the following files :

  XBIN.TUT       Small tutorial on how to program the VGA in text mode.
                 Setting, palette, entering 512 character mode, setting fonts
                 Setting font sizes, (de)compressing XBIN and on-the-fly
                 decompressing of XBIN.
  SIMPLEXB.EXE   Simple-XB, a simple yet sturdy XBIN viewer.  Care has been
                 taken so it supports ANY valid XBIN you throw at it, it may
                 behave erratic for invalid XBIN files though.
                 it's not the fastest XBIN viewing possible, but it DOES
                 support ALL possible XBIN files, even 8Gb ones <g>.
                 You can use SIMPLEXB as a basis for your own viewer and use
                 it as a test case to test validity of XBIN files.
  SIMPLEXB.PAS   Sourcecode to Simple-XB
                 It's main intention is to serve as a learning platform
                 on how to use XBIN files, alongside of being a testing
                 platform for XBIN conpliance.
  BIN2XBIN.EXE   Conversion program to convert BIN files to XBIN.  It is
                 provided in executable form for those who do not have Turbo
                 Pascal.
  BIN2XBIN.PAS   Sourcecode to BIN2XBIN
                 BIN2XBIN serves as a learning platform on how one COULD
                 implement an XBIN compressor.  BIN2XBIN combines several
                 BIN files (up to 10 by 10 screens) into a single XBIN.
                 No palette or font information is stored.
  ADF2XBIN.EXE   Similar to BIN2XBIN, but for converting a single ADF file to
                 XBIN
  ADF2XBIN.PAS   Sourcecode for ADF2XBIN
  STM.PAS        Unit used by both SIMPLEXB.PAS and BIN2XBIN.PAS
                 the STM unit provides buffered file input & output to files
                 providing a significant speed increase over using a file of
                 bytes, or even blockwrite/blockread on relatively small
                 chunks of data.
  VGA.PAS        Unit used by SIMPLEXB.PAS
                 The VGA unit provides the low level VGA specific routines.
  CT-XBIN.XB
  T1-XBIN.XB
  US-XBIN.XB     Several XBIN files so you can have a look at what it can do
                 And so that you can test if your own viewer works correctly.

                                 -THE-END-
SAUCE00eXtended BIN format specification  Tasmaniac           ACiD Productions    19960801(R   P                             