Class TX3DLexer

DescriptionHierarchyFieldsMethodsProperties

Unit

Declaration

type TX3DLexer = class(TObject)

Description

VRML/X3D (classic encoding) lexer.

The lexer always "looks" (i.e. contains in Token and TokenXxx fields) at the next not yet interpreted token.

Remember that VRML is case-sensitive, so TokenName and TokenString should be compared in case-sensitive manner. Also note that for VRML/X3D >= 2.0 these fields contain UTF-8 encoded strings.

Note that this lexer can read only from TPeekCharStream, not just from any TStream. You may have to wrap your stream in some TPeekCharStream descendant (see for example at CreateFromFile implementation, that creates TFileStream and then wraps it inside TBufferedReadStream).

Hierarchy

  • TObject
  • TX3DLexer

Overview

Methods

Public constructor Create(AStream: TPeekCharStream; AOwnsStream: boolean);
Public constructor CreateFromFile(const FileName: string);
Public constructor CreateForPartialStream( AStream: TPeekCharStream; AOwnsStream: boolean; const AVersion: TX3DVersion); overload;
Public constructor CreateForPartialStream(const S: string; const AVersion: TX3DVersion); overload;
Public destructor Destroy; override;
Public function NextToken: TX3DToken;
Public procedure NextTokenForceVTName;
Public procedure NextTokenForceVTString;
Public function TokenIsKeyword(const Keyword: TX3DKeyword): boolean; overload;
Public function TokenIsKeyword(const Keywords: TX3DKeywords): boolean; overload;
Public function DescribeToken: string;
Public procedure CheckTokenIs(Tok: TX3DToken); overload;
Public procedure CheckTokenIs(Tok: TX3DToken; const TokDescription: string); overload;
Public procedure CheckTokenIs(const Toks: TX3DTokens; const ToksDescription: string); overload;
Public procedure CheckTokenIsKeyword(const Keyword: TX3DKeyword);

Properties

Public property Stream: TPeekCharStream read FStream;
Public property Version: TX3DVersion read fVersion;
Public property Token: TX3DToken read fToken;
Public property TokenKeyword: TX3DKeyword read fTokenKeyword;
Public property TokenName: string read fTokenName;
Public property TokenFloat: Float read fTokenFloat;
Public property TokenInteger: Int64 read fTokenInteger;
Public property TokenString: string read fTokenString;

Description

Methods

Public constructor Create(AStream: TPeekCharStream; AOwnsStream: boolean);

Standard constructor. After constructor call, Version is already set, it's checked that file is not compressed by gzip, and the first Token is already read.

Exceptions raised
EX3DGzipCompressed
If the Stream starts with gzip file header.
Public constructor CreateFromFile(const FileName: string);
 
Public constructor CreateForPartialStream( AStream: TPeekCharStream; AOwnsStream: boolean; const AVersion: TX3DVersion); overload;

Constructor for the case when you only have part of normal VRML tokens stream.

This is particularly useful to parse fields of X3D in XML encoding. Inside XML attributes we have then a text that can parsed with a classical VRML lexer, to parse fields contents.

This creates a lexer that works quite like a normal lexer. At creation time it doesn't expect header line (like #VRML 2.0 utf8), that why you have to supply VRML major and minor version as parameters here. Also it doesn't try to detect gzip header. It simply behaves like we're in the middle of VRML tokens stream.

Overloaded version with a first parameter as string simply reads tokens from this string (wrapping it in TStringStream and TPeekCharStream).

Public constructor CreateForPartialStream(const S: string; const AVersion: TX3DVersion); overload;
 
Public destructor Destroy; override;
 
Public function NextToken: TX3DToken;

NextToken reads next token from stream, initializing appropriately all Token* properties. For comfort, this returs the new value of Token property.

Public procedure NextTokenForceVTName;

Read the next token, knowing that it must be vtName token. This is basically a dirty hack to read some incorrect VRML files, that use not allowed characters in VRML names. This allows us to accept as a vtName some characters that normally (when using normal NextToken) would get interpreted as other token.

For example, mgf2inv can write name 0 (a zero, that would be read as vtInteger token in normal circumstances), on some WWW page I found sample VRML models with node name "Crab!" (yes, with exclamation mark and double quotes as part of the node name).

Exceptions raised
EX3DParserError
When we really really cannot interpret contents as vtName token here — currently this may happen only if end of stream is reached. Note that this is reported as a parsing error.
Public procedure NextTokenForceVTString;

Read the next token, knowing that it must be vtString token.

Similiar to NextTokenForceVTName: use this like a shortcut for

  NextToken;
  CheckTokenIs(vtString);

but it is not equivalent to such instructions. This is because VRML 1.0 allowed rather strange thing: string may be not enclosed in double quotes if it does not contain a space. This "feature" is not present in VRML >= 2.0, but, unfortunately, I'm trying to handle VRML 1.0 here so I have to conform to this specification. In particular, Blender generates VRML 1.0 files with Texture2.filename fields not enclosed in double quotes. So this "feature" is actually used by someone... So I have to implement this.

Usual NextToken will not be able to return vtString if it approaches a string not enclosed in double quotes. But THIS function will be able to handle it. So always use this function when you expect a string, this ensures that we will correctly parse any valid VRML 1.0 file.

(unfortunately I'm not doing this now when parsing MFString, this would just require too "unclean" code; I'm using this function only before calling parse on SFString field from TX3DNode.Parse.)

Public function TokenIsKeyword(const Keyword: TX3DKeyword): boolean; overload;

Returns if Token is vtKeyword and TokenKeyword is given Keyword.

Public function TokenIsKeyword(const Keywords: TX3DKeywords): boolean; overload;
 
Public function DescribeToken: string;

Nice textual description of current token, suitable to show to user.

Public procedure CheckTokenIs(Tok: TX3DToken); overload;

Check is token = Tok, if not -> parser error "expected token 'tok'". You can provide your own description for Tok or default desciption for token will be used.

Public procedure CheckTokenIs(Tok: TX3DToken; const TokDescription: string); overload;
 
Public procedure CheckTokenIs(const Toks: TX3DTokens; const ToksDescription: string); overload;
 
Public procedure CheckTokenIsKeyword(const Keyword: TX3DKeyword);
 

Properties

Public property Stream: TPeekCharStream read FStream;

The stream we're reading. This is simply the AStream that you passed to the constructor of this class.

Note that you can't operate on this stream from outside while lexer works, this could confuse the lexer. But you're free to read some stream properties, e.g. check Stream.Position.

Public property Version: TX3DVersion read fVersion;

VRML/X3D version, as recorded in the file header.

All VRML 1.0, 2.0, X3D (various 3.x) are recognized correctly. For Inventor 1.0 ascii, we set Version.Major to 0 (as historically Inventor is a predecessor to VRML 1.0).

Public property Token: TX3DToken read fToken;

Token we're currently standing on. TokenKeyword, TokenName, TokenFloat and TokenInteger have defined values only when token type is appropriate.

Public property TokenKeyword: TX3DKeyword read fTokenKeyword;

When Token = vtKeyword, TokenKeyword points to appropriate keyword.

When Version.Major = 1, then you can be sure that TokenKeyword is in VRML10Keywords. Analogous for VRML20Keywords and X3DKeywords. So e.g. in VRML 1.0 "PROTO" will be treated like a normal name, not a start of prototype.

Public property TokenName: string read fTokenName;

When Token = vtName, TokenName contains appropriate VRML name.

Name syntax as in specification on page 24 (really 32 in pdf) of vrml97specification.pdf. It can be a user name for something (for a node, for example) but it can also be a name of a node type or a node field or an enumerated field constant ... it can be anything except keyword.

Note that this is supposed to contain UTF-8 encoded string for VRML >= 2.0.

Public property TokenFloat: Float read fTokenFloat;

When Token = vtFloat or vtInteger, TokenFloat contains a value of this token.

For vtInteger you have the same thing in TokenInteger, TokenFloat is also initialized to the same value for your comfort (every integer value is also a float, after all).

Public property TokenInteger: Int64 read fTokenInteger;

When Token = vtInteger, TokenInteger contains appropriate value.

Public property TokenString: string read fTokenString;

When Token = vtString, TokenString contains string value.


Generated by PasDoc 0.12.1 on 2013-02-04 20:26:54