如何使用 Lucene Analyzer 标记字符串?

2022-01-15 00:00:00 tokenize lucene java analyzer

有没有一种简单的方法可以使用 Lucene 的 Analyzer 的任何子类来解析/标记 String?

Is there a simple way I could use any subclass of Lucene's Analyzer to parse/tokenize a String?

类似:

String to_be_parsed = "car window seven";
Analyzer analyzer = new StandardAnalyzer(...);
List<String> tokenized_string = analyzer.analyze(to_be_parsed);

推荐答案

据我所知，你必须自己编写循环.像这样的东西(直接取自我的源代码树):

As far as I know, you have to write the loop yourself. Something like this (taken straight from my source tree):

public final class LuceneUtils {

    public static List<String> parseKeywords(Analyzer analyzer, String field, String keywords) {

        List<String> result = new ArrayList<String>();
        TokenStream stream  = analyzer.tokenStream(field, new StringReader(keywords));

        try {
            while(stream.incrementToken()) {
                result.add(stream.getAttribute(TermAttribute.class).term());
            }
        }
        catch(IOException e) {
            // not thrown b/c we're using a string reader...
        }

        return result;
    }  
}

相关文章