DictionaryLookup

java.lang.Object
- morfologik.stemming.DictionaryLookup

All Implemented Interfaces:

java.lang.Iterable<WordData>, IStemmer
```
public final class DictionaryLookup
extends java.lang.Object
implements IStemmer, java.lang.Iterable<WordData>
```
This class implements a dictionary lookup over an FSA dictionary. The dictionary for this class should be prepared from a text file using Jan Daciuk's FSA package (see link below).
Important: finite state automatons in Jan Daciuk's implementation use bytes not unicode characters. Therefore objects of this class always have to be constructed with an encoding used to convert Java strings to byte arrays and the other way around. You can use UTF-8 encoding, as it should not conflict with any control sequences and separator characters.

See Also:

FSA package Web site

Field Summary

Fields
Modifier and Type	Field and Description
`private java.nio.ByteBuffer`	`byteBuffer` Internal reusable buffer for encoding words into byte arrays using `encoder`.
`private java.nio.CharBuffer`	`charBuffer` Internal reusable buffer for encoding words into byte arrays using `encoder`.
`private java.nio.charset.CharsetDecoder`	`decoder` Charset decoder for the FSA.
`private Dictionary`	`dictionary` The `Dictionary` this lookup is using.
`private DictionaryMetadata`	`dictionaryMetadata` Features of the compiled dictionary.
`private java.nio.charset.CharsetEncoder`	`encoder` Charset encoder for the FSA.
`private static int`	`EXPAND_SIZE` Expand buffers and arrays by this constant.
`private ByteSequenceIterator`	`finalStatesIterator` An iterator for walking along the final states of `fsa`.
`private WordData[]`	`forms` Private internal array of reusable word data objects.
`private ArrayViewList<WordData>`	`formsList` A "view" over an array implementing
`private FSA`	`fsa` The FSA we are using.
`private FSATraversal`	`matcher` An FSA used for lookups.
`private MatchResult`	`matchResult` Reusable match result.
`private int`	`rootNode` FSA's root node.
`private char`	`separatorChar`
`private ISequenceEncoder`	`sequenceEncoder`

Constructor Summary

Constructors
Constructor and Description
`DictionaryLookup(Dictionary dictionary)` Creates a new object of this class using the given FSA for word lookups and encoding for converting characters to bytes.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static java.lang.String`	`applyReplacements(java.lang.CharSequence word, java.util.LinkedHashMap<java.lang.String,java.lang.String> replacements)` Apply partial string replacements from a given map.
`private java.nio.ByteBuffer`	`charsToBytes(java.nio.CharBuffer chars, java.nio.ByteBuffer bytes)` Encode a character sequence into a byte buffer, optionally expanding buffer.
`Dictionary`	`getDictionary()`
`char`	`getSeparatorChar()`
`java.util.Iterator<WordData>`	`iterator()` Return an iterator over all `WordData` entries available in the embedded `Dictionary`.
`java.util.List<WordData>`	`lookup(java.lang.CharSequence word)` Searches the automaton for a symbol sequence equal to `word`, followed by a separator.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable
forEach, spliterator

- Field Detail
  - matcher
```
private final FSATraversal matcher
```
    An FSA used for lookups.
  - finalStatesIterator
```
private final ByteSequenceIterator finalStatesIterator
```
    An iterator for walking along the final states of fsa.
  - rootNode
```
private final int rootNode
```
    FSA's root node.
  - EXPAND_SIZE
```
private static final int EXPAND_SIZE
```
    Expand buffers and arrays by this constant.
    
    See Also:
    
    Constant Field Values
  - forms
```
private WordData[] forms
```
    Private internal array of reusable word data objects.
  - formsList
```
private final ArrayViewList<WordData> formsList
```
    A "view" over an array implementing
  - dictionaryMetadata
```
private final DictionaryMetadata dictionaryMetadata
```
    Features of the compiled dictionary.
    
    See Also:
    
    DictionaryMetadata
  - encoder
```
private final java.nio.charset.CharsetEncoder encoder
```
    Charset encoder for the FSA.
  - decoder
```
private final java.nio.charset.CharsetDecoder decoder
```
    Charset decoder for the FSA.
  - fsa
```
private final FSA fsa
```
    The FSA we are using.
  - separatorChar
```
private final char separatorChar
```
    See Also:
    
    getSeparatorChar()
  - byteBuffer
```
private java.nio.ByteBuffer byteBuffer
```
    Internal reusable buffer for encoding words into byte arrays using encoder.
  - charBuffer
```
private java.nio.CharBuffer charBuffer
```
    Internal reusable buffer for encoding words into byte arrays using encoder.
  - matchResult
```
private final MatchResult matchResult
```
    Reusable match result.
  - dictionary
```
private final Dictionary dictionary
```
    The Dictionary this lookup is using.
  - sequenceEncoder
```
private final ISequenceEncoder sequenceEncoder
```
- Constructor Detail
  - DictionaryLookup
```
public DictionaryLookup(Dictionary dictionary)
                 throws java.lang.IllegalArgumentException
```
    Creates a new object of this class using the given FSA for word lookups and encoding for converting characters to bytes.
    
    Parameters:
    
    dictionary - The dictionary to use for lookups.
    
    Throws:
    
    java.lang.IllegalArgumentException - if FSA's root node cannot be acquired (dictionary is empty).
- Method Detail
  - lookup
```
public java.util.List<WordData> lookup(java.lang.CharSequence word)
```
    Searches the automaton for a symbol sequence equal to word, followed by a separator. The result is a stem (decompressed accordingly to the dictionary's specification) and an optional tag data.
    
    Specified by:
    
    lookup in interface IStemmer
    
    Parameters:
    
    word - The word (typically inflected) to look up base forms for.
    
    Returns:
    
    A list of WordData entries (possibly empty).
  - applyReplacements
```
public static java.lang.String applyReplacements(java.lang.CharSequence word,
                                                 java.util.LinkedHashMap<java.lang.String,java.lang.String> replacements)
```
    Apply partial string replacements from a given map. Useful if the word needs to be normalized somehow (i.e., ligatures, apostrophes and such).
    
    Parameters:
    
    word - The word to apply replacements to.
    
    replacements - A map of replacements (from->to).
    
    Returns:
    
    Returns a new string with all replacements applied.
  - charsToBytes
```
private java.nio.ByteBuffer charsToBytes(java.nio.CharBuffer chars,
                                         java.nio.ByteBuffer bytes)
```
    Encode a character sequence into a byte buffer, optionally expanding buffer.
  - iterator
```
public java.util.Iterator<WordData> iterator()
```
    Return an iterator over all WordData entries available in the embedded Dictionary.
    
    Specified by:
    
    iterator in interface java.lang.Iterable<WordData>
  - getDictionary
```
public Dictionary getDictionary()
```
    Returns:
    
    Return the Dictionary used by this object.
  - getSeparatorChar
```
public char getSeparatorChar()
```
    Returns:
    
    Returns the logical separator character splitting inflected form, lemma correction token and a tag. Note that this character is a best-effort conversion from a byte in DictionaryMetadata.separator and may not be valid in the target encoding (although this is highly unlikely).

Class DictionaryLookup

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface java.lang.Iterable

Field Detail

matcher

finalStatesIterator

rootNode

EXPAND_SIZE

forms

formsList

dictionaryMetadata

encoder

decoder

fsa

separatorChar

byteBuffer

charBuffer

matchResult

dictionary

sequenceEncoder

Constructor Detail

DictionaryLookup

Method Detail

lookup

applyReplacements

charsToBytes

iterator

getDictionary

getSeparatorChar