| java.lang.Object | |||||
| ↳ | org.apache.lucene.util.AttributeSource | ||||
| ↳ | org.apache.lucene.analysis.TokenStream | ||||
| ↳ | org.apache.lucene.analysis.Tokenizer | ||||
| ↳ | org.apache.lucene.analysis.CharTokenizer | ||||
| ↳ | org.apache.lucene.analysis.WhitespaceTokenizer | ||||
Class Overview
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.
Summary
|
[Expand]
Inherited Fields | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
 From class
org.apache.lucene.analysis.Tokenizer
| |||||||||||
| Public Constructors | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
Construct a new WhitespaceTokenizer.
| |||||||||||
Construct a new WhitespaceTokenizer using a given
AttributeSource. | |||||||||||
Construct a new WhitespaceTokenizer using a given
AttributeSource.AttributeFactory. | |||||||||||
| Protected Methods | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
Collects only characters which do not satisfy
isWhitespace(char). | |||||||||||
|
[Expand]
Inherited Methods | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
From class
org.apache.lucene.analysis.CharTokenizer
| |||||||||||
|
From class
org.apache.lucene.analysis.Tokenizer
| |||||||||||
|
From class
org.apache.lucene.analysis.TokenStream
| |||||||||||
|
From class
org.apache.lucene.util.AttributeSource
| |||||||||||
|
From class
java.lang.Object
| |||||||||||
|
From interface
java.io.Closeable
| |||||||||||
Public Constructors
public WhitespaceTokenizer (AttributeSource source, Reader in)
Construct a new WhitespaceTokenizer using a given AttributeSource.
public WhitespaceTokenizer (AttributeSource.AttributeFactory factory, Reader in)
Construct a new WhitespaceTokenizer using a given AttributeSource.AttributeFactory.
Protected Methods
protected boolean isTokenChar (char c)
Collects only characters which do not satisfy
isWhitespace(char).