Bugfix for java.io.StreamTokenizer

[Dalibor Topic]

Hi,

the methods lineno() and toString() in StreamTokenizer
 are buggy [1]. 

StreamTokenizer.lineno() should add 1 to the line
number received from linein to comply to 22.14.23
from the Java Language Specification (JLS). 
StreamTokenizer.toString's return format didn't match
the one specified in 22.14.24 for words, EOFs and
EOLs. Also, no string was returned for tokens that
were ordinary characters. The JLS doesn't really
specify what the generated String is supposed to look
like in that case, if I understand the spec correctly,
so I patched kaffe's toString to do it the way Sun's
toString does - it can't harm :)

Attached you'll find the diff to the surrent
implementation, as well as a small test program. The
easiest way to verify that there was a bug is to run
the test program with it's own source file as input:

 java StreamTokenizerTest <StreamTokenizerTest.java >
kaffe.out

and compare its output vs. the output generated by
Sun's JDK 1.1.x

Happy checking in,
dali

[1] I checked against the Java Language Specification
java.io.StreamTokenizer entry at
http://java.sun.com/docs/books/jls/html/javaio.doc14.html#29287
, and tested the behaviour of Sun's JDK 1.1.3 for
Solaris and Blackdown's port of JDK 1.1.7 for Linux.


-------------- next part --------------
*** /tmp/kaffe/libraries/javalib/java/io/StreamTokenizer.java	Wed Dec  8 07:46:29 1999
--- kaffe/libraries/javalib/java/io/StreamTokenizer.java	Wed Apr  7 13:16:02 1999
***************
*** 81,87 ****
  }
  
  public int lineno() {
! 	return (lineIn.getLineNumber() + 1);
  }
  
  public void lowerCaseMode(boolean fl) {
--- 81,87 ----
  }
  
  public int lineno() {
! 	return (lineIn.getLineNumber());
  }
  
  public void lowerCaseMode(boolean fl) {
***************
*** 349,367 ****
  
  public String toString() {
      	if (ttype == TT_EOF) {
! 		return ("Token[EOF], line "+lineno());
  	}
  	else if (ttype == TT_EOL) {
! 		return ("Token[EOL], line "+lineno());
  	}
  	else if (ttype == TT_NUMBER) {
  		return ("Token[n="+nval+"], line "+lineno());
  	}
- 	else if (ttype == TT_WORD) {
- 	        return ("Token["+sval+"], line "+lineno());
- 	}
  	else {
! 	        return ("Token[\'"+ (char) ttype +"\'], line "+lineno());
  	}
  }
  
--- 349,364 ----
  
  public String toString() {
  	if (ttype == TT_EOF) {
! 		return ("EOF");
  	}
  	else if (ttype == TT_EOL) {
! 		return ("EOL");
  	}
  	else if (ttype == TT_NUMBER) {
  		return ("Token[n="+nval+"], line "+lineno());
  	}
  	else {
! 		return ("Token[s="+sval+"], line "+lineno());
  	}
  }
  
-------------- next part --------------
import java.io.StreamTokenizer;

class StreamTokenizerTest {

    public static void main (String argv[]) {

	//check the values of constants
	StreamTokenizer st = new StreamTokenizer(System.in);

	System.out.println("StreamTokenizer.TT_EOF = "
			   + st.TT_EOF);
	System.out.println("StreamTokenizer.TT_EOL = "
			   + st.TT_EOL);
	System.out.println("StreamTokenizer.TT_NUMBER = "
			   + st.TT_NUMBER);
	System.out.println("StreamTokenizer.TT_WORD = "
			   + st.TT_WORD);

	// check the StreamTokenizer.toString() function
	st.eolIsSignificant(true);

	while (st.ttype != st.TT_EOF) {
	    try {
		st.nextToken();
		System.out.println(st);
	    }
	    catch (Exception e) {
		System.exit(0);
	    }
	}

    }
}