org.jruby.lexer.yacc

Class LexerSource

java.lang.Object

org.jruby.lexer.yacc.LexerSource

Direct Known Subclasses:

ByteArrayLexerSource, InputStreamLexerSource

public abstract class LexerSource
extends java.lang.Object

This class is what feeds the lexer. It is primarily a wrapper around a Reader that can unread() data back onto the source. Originally, I thought about using the PushBackReader to handle read/unread, but I realized that some extremely pathological case could overflow the pushback buffer. Better safe than sorry. I could have combined this implementation with a PushbackBuffer, but the added complexity did not seem worth it.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	line
private java.lang.StringBuilder	lineBuffer
protected int	lineOffset
private java.util.List<java.lang.String>	list
protected int	offset
private SimplePositionFactory	positionFactory
private java.lang.StringBuilder	sourceLine
private java.lang.String	sourceName

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	LexerSource(java.lang.String sourceName, java.util.List<java.lang.String> list, int lineOffset, boolean extraPositionInformation) Create our food-source for the lexer

Method Summary

Modifier and Type	Method and Description
protected void	captureFeature(int c)
private void	captureFeatureNewline()
java.lang.String	getCurrentLine()
java.lang.String	getFilename() What file are we lexing?
int	getLine() What line are we at?
int	getOffset() The location of the last byte we read from the source.
ISourcePosition	getPosition() Where is the reader within the source {filename,row}
ISourcePosition	getPosition(ISourcePosition startPosition) Where is the reader within the source {filename,row}
abstract java.io.InputStream	getRemainingAsStream()
static LexerSource	getSource(java.lang.String name, byte[] content, java.util.List<java.lang.String> list, ParserConfiguration configuration)
static LexerSource	getSource(java.lang.String name, java.io.InputStream content, java.util.List<java.lang.String> list, ParserConfiguration configuration) Create a source.
int	getVirtualLine()
abstract boolean	lastWasBeginOfLine()
protected java.lang.String	makePointer(int length)
abstract boolean	matchMarker(org.jruby.util.ByteList marker, boolean indent, boolean withNewline) Match marker against input consumering lexer source as it goes...Unless it does not match then it reverts lexer source back to point when this method was invoked.
abstract boolean	peek(int c)
abstract int	read()
int	readCodepoint(int first, org.jcodings.Encoding encoding)
abstract org.jruby.util.ByteList	readLineBytes()
abstract org.jruby.util.ByteList	readUntil(char c)
abstract int	skipUntil(int c)
protected void	uncaptureFeature(int c)
abstract void	unread(int c)
abstract void	unreadMany(java.lang.CharSequence line)
abstract boolean	wasBeginOfLine()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

positionFactory

private SimplePositionFactory positionFactory

sourceName

private final java.lang.String sourceName

line

protected int line

lineOffset

protected int lineOffset

offset

protected int offset

list

private java.util.List<java.lang.String> list

lineBuffer

private java.lang.StringBuilder lineBuffer

sourceLine

private java.lang.StringBuilder sourceLine

Constructor Detail

LexerSource

protected LexerSource(java.lang.String sourceName,
                      java.util.List<java.lang.String> list,
                      int lineOffset,
                      boolean extraPositionInformation)

Create our food-source for the lexer

Parameters:

sourceName - is the file we are reading

reader - is what represents the contents of file sourceName

line - starting line number for source (used by eval)

extraPositionInformation - will gives us extra information that an IDE may want (deprecated)

Method Detail

getFilename

public java.lang.String getFilename()

What file are we lexing?

Returns:

the files name

getLine

public int getLine()

What line are we at?

Returns:

the line number 0...line_size-1

getVirtualLine

public int getVirtualLine()

getOffset

public int getOffset()

The location of the last byte we read from the source.

Returns:

current location of source

getPosition

public ISourcePosition getPosition(ISourcePosition startPosition)

Where is the reader within the source {filename,row}

Returns:

the current position

getPosition

public ISourcePosition getPosition()

Where is the reader within the source {filename,row}

Returns:

the current position

getSource

public static LexerSource getSource(java.lang.String name,
                                    java.io.InputStream content,
                                    java.util.List<java.lang.String> list,
                                    ParserConfiguration configuration)

Create a source.

Parameters:

name - the name of the source (e.g a filename: foo.rb)

content - the data of the source

Returns:

the new source

getSource

public static LexerSource getSource(java.lang.String name,
                                    byte[] content,
                                    java.util.List<java.lang.String> list,
                                    ParserConfiguration configuration)

captureFeatureNewline

private void captureFeatureNewline()

captureFeature

protected void captureFeature(int c)

uncaptureFeature

protected void uncaptureFeature(int c)

getCurrentLine

public java.lang.String getCurrentLine()

makePointer

protected java.lang.String makePointer(int length)

readCodepoint

public int readCodepoint(int first,
                         org.jcodings.Encoding encoding)
                  throws java.io.IOException

Throws:

java.io.IOException

matchMarker

public abstract boolean matchMarker(org.jruby.util.ByteList marker,
                                    boolean indent,
                                    boolean withNewline)
                             throws java.io.IOException

Match marker against input consumering lexer source as it goes...Unless it does not match then it reverts lexer source back to point when this method was invoked.

Parameters:

marker - to match against

indent - eat any leading whitespace

withNewline - includes a check that marker is followed by newline or EOF

Returns:

true if marker matches...false otherwise

Throws:

java.io.IOException - if an error occurred reading from underlying IO source

read

public abstract int read()
                  throws java.io.IOException

Throws:

java.io.IOException

readUntil

public abstract org.jruby.util.ByteList readUntil(char c)
                                           throws java.io.IOException

Throws:

java.io.IOException

readLineBytes

public abstract org.jruby.util.ByteList readLineBytes()
                                               throws java.io.IOException

Throws:

java.io.IOException

skipUntil

public abstract int skipUntil(int c)
                       throws java.io.IOException

Throws:

java.io.IOException

unread

public abstract void unread(int c)

unreadMany

public abstract void unreadMany(java.lang.CharSequence line)

peek

public abstract boolean peek(int c)
                      throws java.io.IOException

Throws:

java.io.IOException

lastWasBeginOfLine

public abstract boolean lastWasBeginOfLine()

wasBeginOfLine

public abstract boolean wasBeginOfLine()

getRemainingAsStream

public abstract java.io.InputStream getRemainingAsStream()
                                                  throws java.io.IOException

Throws:

java.io.IOException