org.gjt.sp.jedit.syntax
Class Token

java.lang.Object
  |
  +--org.gjt.sp.jedit.syntax.Token

public class Token
extends java.lang.Object

A linked list of tokens. Each token has three fields - a token identifier, which is a byte value that can be looked up in the array returned by SyntaxDocument.getColors() to get a color value, a length value which is the length of the token in the text, and a pointer to the next token in the list.


Field Summary
static byte COMMENT1
          Comment 1 token id.
static byte COMMENT2
          Comment 2 token id.
static byte END
          The token type, that along with a length of 0 marks the end of the token list.
 byte id
          The id of this token.
static byte ID_COUNT
          The total number of defined token ids.
static byte INTERNAL_FIRST
          The first id that can be used for internal state in a token marker.
static byte INTERNAL_LAST
          The last id that can be used for internal state in a token marker.
static byte INVALID
          Invalid token id.
static byte KEYWORD1
          Keyword 1 token id.
static byte KEYWORD2
          Keyword 2 token id.
static byte KEYWORD3
          Keyword 3 token id.
static byte LABEL
          Label token id.
 int length
          The length of this token.
static byte LITERAL1
          Literal 1 token id.
static byte LITERAL2
          Literal 2 token id.
 Token next
          The next token in the linked list.
static byte NULL
          Normal text token id.
static byte OPERATOR
          Operator token id.
 
Constructor Summary
Token(int length, byte id)
          Creates a new token.
 
Method Summary
 java.lang.String toString()
          Returns a string representation of this token.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

NULL

public static final byte NULL
Normal text token id. This should be used to mark normal text.

COMMENT1

public static final byte COMMENT1
Comment 1 token id. This can be used to mark a comment.

COMMENT2

public static final byte COMMENT2
Comment 2 token id. This can be used to mark a comment.

LITERAL1

public static final byte LITERAL1
Literal 1 token id. This can be used to mark a string literal (eg, C mode uses this to mark "..." literals)

LITERAL2

public static final byte LITERAL2
Literal 2 token id. This can be used to mark an object literal (eg, Java mode uses this to mark true, false, etc)

LABEL

public static final byte LABEL
Label token id. This can be used to mark labels (eg, C mode uses this to mark ...: sequences)

KEYWORD1

public static final byte KEYWORD1
Keyword 1 token id. This can be used to mark a keyword. This should be used for general language constructs.

KEYWORD2

public static final byte KEYWORD2
Keyword 2 token id. This can be used to mark a keyword. This should be used for preprocessor commands, or variables.

KEYWORD3

public static final byte KEYWORD3
Keyword 3 token id. This can be used to mark a keyword. This should be used for data types.

OPERATOR

public static final byte OPERATOR
Operator token id. This can be used to mark an operator. (eg, SQL mode marks +, -, etc with this token type)

INVALID

public static final byte INVALID
Invalid token id. This can be used to mark invalid or incomplete tokens, so the user can easily spot syntax errors.

ID_COUNT

public static final byte ID_COUNT
The total number of defined token ids.

INTERNAL_FIRST

public static final byte INTERNAL_FIRST
The first id that can be used for internal state in a token marker.

INTERNAL_LAST

public static final byte INTERNAL_LAST
The last id that can be used for internal state in a token marker.

END

public static final byte END
The token type, that along with a length of 0 marks the end of the token list.

length

public int length
The length of this token.

id

public byte id
The id of this token.

next

public Token next
The next token in the linked list.
Constructor Detail

Token

public Token(int length,
             byte id)
Creates a new token.
Parameters:
length - The length of the token
id - The id of the token
Method Detail

toString

public java.lang.String toString()
Returns a string representation of this token.
Overrides:
toString in class java.lang.Object