[kaffe] Bug report (java.io.StreamTokenizer)

[Ito Kazumitsu]

In message "Re: [kaffe] Bug report (java.io.StreamTokenizer)"
    on 03/07/01, Ito Kazumitsu <kaz at maczuka.gcd.org> writes:

> Running echo "121"| java StreamTokenizerTest2 1 N S and so on,
> I found the precedence is not so simple in Sun's JDK.
> It depends on in what sequence initialization methods are called.
> 
> bash$ echo "121"| java StreamTokenizerTest2 1 C S
> null: 50
> null: 10
> null: -1
> bash$ echo "121"| java StreamTokenizerTest2 1 S C
> null: 10
> null: -1
> bash$

Having tested various cases,  I have had an impression
that Sun's java.io.StreamTokenizer uses the following
rule:

(From the API doc) Each byte read from the input stream is regarded
as a character in the range '\u0000' through '\u00FF'. The character
value is used to look up five possible attributes of the character:
white space, alphabetic, numeric, string quote, and comment character.
Each character can have zero or more of these attributes.
(And I guess) BUT ONLY THE ATTRIBUTE MOST RECENTLY ASSIGNED IS EFFECTIVE.

For example,

bash$ echo "121" | java StreamTokenizerTest2 1 W C
null: 10
null: -1
bash$ echo "121" | java StreamTokenizerTest2 1 W C S
null: 50
null: 10
null: -1
bash$ echo "121" | java StreamTokenizerTest2 1 W S C
null: 10
null: -1
bash$ echo "121" | java StreamTokenizerTest2 1 S C W
1: -3
null: 50
1: -3
null: 10
null: -1
bash$ echo "121" | java StreamTokenizerTest2 1 S C W N
121.0: -2
null: 10
null: -1
bash$ echo "121" | java StreamTokenizerTest2 1 S C W N S
21.0: -2
null: 10
null: -1