The Plucker Document Format

Introduction

This document is the official description of the Plucker format.

Overview

The Plucker document format supports a multi-page (in the Web sense of 'page') hyperlinked information structure containing both 'rich' text and images. Links may be internal to the document or link to other documents. External links, in standard URL form, may be included and displayed, but not followed. Images may either be embedded in a text page, as with HTML, or be included as separate stand-alone pages.

Plucker documents are structured so that they can be used both with a file-system-oriented operating system such as Unix or Windows, and with the PalmOS, a non-file-system-oriented OS. To this end, they always begin with a standard PalmOS record database prefix, which consists of four parts: the database header, a record-id list, an AppInfo block, and a SortInfo block. The Plucker format does not use the SortInfo block, which is therefore null, and consequently occupies no space in the document prefix.

The record database prefix is then followed by a sequence of application-specific records. In a Plucker document, this sequence consists of one index record, followed by a series of data records. The index record contains information about the data records, along with some global information, such as the type of compression used. Each data record contains either a page, an image, or data about the document, such as bookmarks or URL data.

The format is big-endian; any multi-byte numeric values specified in this document are big-endian. Images are stored in the Palm image format; for more information on this format please consult http://www.palmos.com/dev/tech/docs/.

The Database Prefix

The Database Header

The database header is a fixed-size structure of 72 bytes. It contains the name of the database, the Plucker version number, various timestamps (creation, modification, last backup), and several flags. All timestamps are given using the PalmOS standard, seconds since 12:00 AM on January 1, 1904.

Field	Bytes	Type	Notes
docName	32	String	Must contain a NUL-terminated 7-bit ASCII string (only character codes 0x20-0x7E are valid) giving the name of the document. Because of the terminating NUL character at end, only 31 bytes can actually be used for the name of the document. The first 26 bytes of this string are used by Plucker as a unique ID for the document; names should be unique in the first 26 characters.
flags	2	Bitfield	Most bits in this field are unused. Unused bits should be set to zero on document creation, but reader software should not expect them to stay at this value. Valid bits are as follows. All numeric values given are big-endian. Name Value Meaning CopyPrevention 0x0040 Indicates that system should not allow copying of this document. Launchable 0x0200 Indicates that this document should be presented as a first-class object on desktop renderings. If this bit is set, an AppInfo block must be included. Backup 0x0008 Indicates that this document should be backed up, if the system includes such a capability.
version	2	Numeric	Version of the Plucker format used in this document. Must have the value 1.
creationDate	4	Timestamp	Time of document creation
modificationDate	4	Timestamp	Time document last modified
unused1	8	Numeric	Must be zero at document creation, but any specific value should not be relied upon.
appInfoOffset	4	Numeric	Either zero, if no appInfo is present, or the offset from the beginning of the document to the start of the appInfo block.
sortInfoId	4	Numeric	Must be zero.
magic	8	String	Must be the 8 ISO Latin-1 characters "DataPlkr". No terminating NUL character.
unused2	4	Numeric	Must be zero at document creation, but any specific value should not be relied upon.

The Record-ID List

This list consists of a six-byte list header, followed by one ID entry for each data record in the document. The list header has the structure:

This is then followed by numRecords entries of the following structure:

Finally, there are two bytes of zero-padding to bring the structure alignment back to 4 bytes.

The AppInfo Block

Typically, this is only present when the launchable flag is set in the flags field of the database header. No Plucker data aside from icon display information and a versioning string is stored in this block. This block has the following structure:

The Index Record

This record includes info about the compression type used for the Plucker document and also what IDs the reserved records use. The viewer will use this record to know where to look for the reserved records and whether it must have support for ZLib compression. This record should always be the first record in the Plucker document (i.e. at index 0).

The reserved ID array consists of a series of name/ID pairs, where the ID is the unique ID (2 bytes) for the record and the name is a value (2 bytes) from the following list.

• home.html = 0
• external bookmarks = 1
• URL handling = 2
• default category = 3
• additional metadata = 4
• page list metadata = 5
• sorted URL name data = 6
• external anchor name data = 7

The Data Records

There are several different types of data records.

• text data (compressed/uncompressed)
• image data (compressed/uncompressed)
• mailto data
• URL handling data
• URL data (compressed/uncompressed)
• external bookmarks
• default category
• metadata
• style sheets (not yet implemented)
• font pages (not yet implemented)
• table data (compressed/uncompressed)
• page list metadata
• sorted URL handling data
• sorted URL data (compressed/uncompressed)
• external anchor name handling data
• external anchor name data (compressed/uncompressed)

Each data record starts with a header, having the following structure:

Compression and the Owner-ID

This data format supports two forms of compression, DOC and ZLIB. That part of a data record that occurs after the header is compressed as a single chunk. All compressed records in a single document must use the same compression format. Compressed records may be mixed with uncompressed records. In a compressed record, the length of the compressed data must be less than its uncompressed length.

DOC compression is the the format invented for early Palm usage.

ZLIB compression uses the ZLib format documented in Internet RFCs 1950 and 1951. See also http://www.gzip.org/zlib/manual.html for a description of the library used to perform the compression and decompression.

Plucker documents may be keyed to a specific string of 40 or fewer ASCII characters, called the owner-id. When such a key is specified, zlib compression must be used in the document. When an owner-id is specified, the beginning of each zlib-compressed data segment is XOR'ed with a value derived from the key, after compression, and must be XOR'ed again with the derived value before being decompressed. If an owner-id is specified for a document, the metadata record must exist, and must contain an OwnerID subrecord giving the CRC-32 of the owner-id string.

The derived value mentioned above is a 40-byte value constructed by forming 10 strings by concatenating the owner-id string with itself 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11 times, then taking the CRC-32 values of each of these concatenations, then packing those 32-bit values in big-endian order into a 40-byte buffer.

Text data records

For text data the data record header is followed by a series of paragraph headers, each representing a paragraph block in the text data. This series of paragraph headers is then followed by the compressed or uncompressed text data. Each paragraph header has the form:

The (uncompressed) text data contains a character stream of ISO Latin-1 characters, interspersed with 'functions'.

A function is introduced in the text stream by a NULL character (0x00), followed by a one-byte function code and up to 7 bytes of data. The 3 LSB of the function code represent the remaining function data length; the 5 MSB denote the actual function code. The following functions are valid:

The function arguments have the following definitions:

Image data records

The image data consists of an image in Palm image format, compressed or uncompressed as specified in the document's index record. The image may in addition be internally compressed, via any of the compression techniques allowed in the Palm image format. The fundamental size of an image must be less than 480,000; this size is calculated by multiplying the width (in pixels) by the height (in pixels) by the depth (in bits).

If the fundamental size is greater than 480,000, most parsers can be told to create a Multi-image group. This is a group of image records consisting of parts of the image which the viewer displays as one image. The parts are standard Image data records and the Multi-image record tells how many columns and rows the image has, and the record numbers of the parts.

Field	Bytes	Type	Notes
columns	2	Numeric	number of columns in this image
rows	2	Numeric	number of rows in this image
image record IDs	2 * columns * rows	Numeric	References to images in Plucker document. There are (columns * rows) images listed here

Navigation metadata

This data is optionally appended to the end of a text or image data record based on the setting of the Navigation Metadata flag in the record header. If the Navigation Metadata flag is set to one, the image or text data is immediately followed by the following data structures.

NOTE: If navigation data is appended to a record then the last two bytes in the record shall contain the byte offset from the beginning of the record to the start of the navigation data.

The offsets block and title strings are followed by a series of tables.

The anchor name table specifies the offset of the anchor names within the text record.

The page list table contains the uid of the previous and next records relative to this record for one or more page lists. Each page list, combined with the page lists from other Incoming Navigation data records defines one of more unique a linear navigation schemes for the document. The default scheme is always associated with a list id of 0.

The page list data consists of a series of structures containing a list id followed by the unique ID of the previous and next records associated with that list id.

The hierarchy table specifies unique ID of each text record above this record in the document hierarchy that serves as an index leading to the current record.

The topic table provides a list of topics associated with the record.

Mailto data records

URL handling data records

In practice, there are two kinds of records used to store the URL strings, the URL handling data record, which serves as an index into the sequence of strings, and the URL data record, one or more of which contain the actual strings.

For cross-document linking support, the URL strings must be of the format "doc://[external doc name]:[url]" where external doc name is the name of the external document and url is the URL string associated with a given record.

URL data records

These records may or may not be compressed. This is indicated by the type in the header. These records are used by the Details form to display the URL of the current record and by the External Reference form to display the URL of not collected pages. From either form you can copy the URL to a Memo to remind you to pluck it at a later date. For inter-document links, a paragraph link function may be specified to contain a pseudo-Record ID in place of an actual-Record ID, and an index into the external anchor names record in place of the paragraph number.

External Bookmarks data records

The bookmark data is a series of uid/offset pairs.

Default Category data records

Each Plucker document can be assigned to a number of named categories. This record stores the names of default categories for the document. The data consists of a concatenated series of NUL-terminated strings that should be used as the default category/categories for this document.

Metadata data records

There should only be one of these per document. This record begins with a two byte numeric value, giving the number of subrecords that follow, followed by that number of subrecords. The subrecords are a sequence of tagged variable length items. Each subrecord consists of three fields:

Field	Bytes	Type	Description
type code	2	Numeric	Specifies what piece of extra information is in this subrecord
length	2	Numeric	Number of 2-byte words in the argument
argument	2 * length	(type code specific)	Data

The following table describes the valid subrecord type codes, and describes the structure of the associated data for each subrecord type. Subrecords with unknown type codes should be ignored.

Type code	Name	Description	Argument
1	CharSet	This is the character set and encoding used by text records in this document, unless otherwise specified for particular records.	a two-byte numeric value, specifying the IETF IANA MIBenum value for the character set. See the IANA registry of character sets for valid values.
2	ExceptionalCharSets	This is a list of text records which use a charset other than that specified by the default CharSet. Note that if no default CharSet is specified, the default charset should be thought of as "unknown".	a sequence of (length / 2) record-ID, IANA-MIBenum pairs, where MIBenum values are as specified for CharSet. The invalid MIBenum value of 0 (zero) is used for records which have an unknown charset, if necessary. Field Bytes Type Notes record ID 2 Numeric unique ID for record MIBenum 2 Numeric IANA MIBenum for the character set used in this record
3	OwnerID	This is the CRC-32 of the specified owner-id for the document, if any. Note that associating an owner-id with a document also affects the calculation of zlib compression.	a four-byte numeric value giving the CRC-32 of the owner-id string.
4	Author	The name of the author of the document.	A string value in the document's default character set, padded at the end with NUL characters to an even number of bytes.
5	Title	The full title of the document.	A string value in the document's default character set, padded at the end with NUL characters to an even number of bytes.
6	PublicationDate	The date and time this document was created.	A 4-byte unsigned integer giving the number of seconds from 12:00 AM on January 1, 1904, to the time when this document was created.
7	Linked Documents	The list of external documents that this document links to.	A concatenated sequence of NUL-terminated strings representing the document names for each external document linked to within this document. The string sequence should be padded at the end with NUL characters to an even number of bytes.

Style Sheet data records

TBD

Font Page data records

TBD

Table data records

The Table Record describes an HTML table. It begins with a structure with the following format.

Field	Bytes	Type	Description
size	2	Numeric	Size of the following data
columns	2	Numeric	Number of columns the table contains
rows	2	Numeric	Number of rows the table contains
depth	1	Numeric	Bits per pixel (BPP) needed to render the table
border	1	Numeric	Draw table borders (0 = no, any other value = yes, 1 pixel wide)
border color	4	Numeric	RGB value of border color
link color	4	Numeric	RGB value of link color

This is followed by table row and table cell functions (their ends are implied). Each table cell function is followed by 'text length' (from the function) text, containing text and/or formatting functions. (Such as links, style, underline, strike through, italic, etc.)

Page List Metadata data record

There should only be one of these per document. This record is used to assign the name and initial record associated with each page list in the document. Page lists are used to define the default ordering of pages within the document. More than one page list can be specified, which can be useful for defining tours through a document.

Field	Bytes	Type	Description
lists	2	Numeric	The number of page lists or tours in the document.
first record	2 * sequences	Numeric Array	An array of uids corresponding to the first record in each page sequence. The zero based index into this array represents the sequence id or tour id. The first entry should be considered the default page ordering for the document.
list name	1+	String sequence	a concatenated sequence of NUL-terminated strings, each representing the name of a page list. The first entry in the list corresponds to the default page ordering. For unnamed page lists, a NUL charater should still be specified.

Page lists can be thought of as linked lists of records. The first record field in the Page List Metadata record is equivalent to the head pointer of the list. Each text record contains a previous/next record pointer within it's navigation metadata.

Sorted URL Handling data record

The Sorted URL handling record is used to find the record ID of the Sorted URL data record containing data for a given URL string. It contains a series of 2 byte number pairs.

Field	Bytes	Type	Description
last URL	2	Numeric	the ordinal number of the last URL in record
id	2	Numeric	record ID for record

Sorted URL data records

The sorted URL data record contains a list of URL/UID pointers sorted according to the lexicographical order of the URL strings pointed to by the url uid and url offset fields. This data is used in cross-document linking to facilitate a binary search of the URL strings in order to lookup the record ID for an incoming URL string. Only URLs for records actually contained in the document should be included in the Sorted URL data records. URLs for external records should be omitted.

Field	Bytes	Type	Description
url uid	2	Numeric	unique ID for URL data record that contains the sorted URL string.
url offset	2	Numeric	byte offset of the first character of the sorted URL string in the URL data record.
record uid	2	Numeric	unique ID of the text or image record that pertains to the sorted URL string.

External Anchor Name Handling data record

The External Anchor handling record is used to find the record ID of the External Anchor data record containing a given external anchor string. It contains a series of 2 byte number pairs.

Field	Bytes	Type	Description
last anchor	2	Numeric	the ordinal number of the last anchor in record
id	2	Numeric	record ID for record

External Anchor Name data record

The External Anchor data record is a string table containing the unique names for all external anchor name strings referenced in this document. These strings are used to query the record specific anchor name tables in a target book to determine a paragraph offset for cross-document linking. This information is conceptually stored as a sequence of strings, where the position of the anchor name in the sequence corresponds to it's index.

Field	Bytes	Type	Description
anchor name list	1+	String sequence	a concatenated sequence of unique NUL-terminated strings, each representing an anchor name from an external link found within this document.

These records may or may not be compressed. This is indicated by the type in the header. These records are used in conjunction with the Sorted URL Data records and record specific anchor name tables to facilitate cross-document linking.

© Copyright 2000 Michael Nordström <micke@sslug.dk> · Copyright 2001 Bill Janssen <bill@janssen.org

$Id: DBFormat.html,v 1.27 2005/10/29 14:14:21 nordstrom Exp $