如何使用 Hibernate Lucene Search 对挪威字符(Æ、Ø 和 Å)进行不区分大小写的排序?

2022-01-15 00:00:00 lucene java Hibernate hibernate-search

æ、ø、å 是挪威字母表中最新的字母

æ, ø, å are latest letters in the norwegian alphabet 

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Æ Ø Å

当我们尝试使用 Hibernate Lucene 对其进行排序时，然后 Å clubs with A、Ø clubs with Ø、Æ clibs with A 即错误的.例如:

When we try to sort it using Hibernate Lucene then Å clubs with A, Ø clubs with Ø, Æ clibs with A which is wrong. For example:

当前结果:

阿鲁，奥阿鲁，巴鲁，扎鲁，

Aaalu, Åaalu, Baalu, Zaalu,

预期结果:

阿鲁，巴鲁，扎鲁，阿鲁，

Aaalu, Baalu, Zaalu, Åaalu,

以下是工作代码:

@AnalyzerDef(name = "myOwnAnalyzer",
tokenizer = @TokenizerDef(factory = KeywordTokenizerFactory.class),
filters = {
    @TokenFilterDef(factory = ASCIIFoldingFilterFactory.class),
    @TokenFilterDef(factory = LowerCaseFilterFactory.class),
    @TokenFilterDef(factory = PatternReplaceFilterFactory.class, params = {
        @Parameter(name = "pattern", value = "('-&\.,\(\))"),
        @Parameter(name = "replacement", value = " "),
        @Parameter(name = "replace", value = "all")
    }),
    @TokenFilterDef(factory = PatternReplaceFilterFactory.class, params = {
        @Parameter(name = "pattern", value = "([^0-9\p{L} ])"),
        @Parameter(name = "replacement", value = ""),
        @Parameter(name = "replace", value = "all")
    }),
    @TokenFilterDef(factory = TrimFilterFactory.class)
}
)
public class KikaPaya implements Serializable {

@Fields({ @Field(index = Index.YES, store = Store.YES), @Field(name = "KikaPayaName_for_sort", index = Index.YES, analyzer = @Analyzer(definition = "myOwnAnalyzer")) })
@Column(name = "NAME", length = 100)
private String name;

主要:

  FullTextEntityManager ftem = Search.getFullTextEntityManager(factory.createEntityManager());
  QueryBuilder qb = ftem.getSearchFactory().buildQueryBuilder().forEntity( KikaPaya.class ).get();
  org.apache.lucene.search.Query query = qb.all().getQuery(); 
  FullTextQuery fullTextQuery = ftem.createFullTextQuery(query, KikaPaya.class);
  fullTextQuery.setSort(new Sort(new SortField("KikaPayaName_for_sort", SortField.STRING, true)));
  fullTextQuery.setFirstResult(0).setMaxResults(150);
  int size = fullTextQuery.getResultSize();
  List<KikaPaya> result = fullTextQuery.getResultList();
  for (KikaPayauser : result) {
    logger.info("KikaPaya Name:" + user.getName());
  }

以下是 Lucene 的版本(我无法更改):

Following are versions of Lucene (which i cannot change):

 <hibernate.version>4.2.8.Final</hibernate.version>
    <hibernate.search.version>4.3.0.Final</hibernate.search.version>

  <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-entitymanager</artifactId>
        <version>4.2.8.Final</version>
    </dependency>
<dependency>
        <groupId>org.apache.lucene</groupId>
        <artifactId>lucene-core</artifactId>
        <version>3.6.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.lucene</groupId>
        <artifactId>lucene-analyzers</artifactId>
        <version>3.6.2</version>
    </dependency>

谁能提出获得正确结果的方法?

Could anyone suggests the way to get correct results?

推荐答案

您可以在 Hibernate Search 4.3.0.Final 版本中使用 org.apache.lucene.collat​​ion.Collat​​ionKeyFilter 类.创建您自己的排序规则过滤器工厂:

You can use org.apache.lucene.collation.CollationKeyFilter class in Hibernate Search version 4.3.0.Final. Create your own collation filter factory:

import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.collation.CollationKeyFilter;
import org.apache.solr.analysis.BaseTokenFilterFactory;

import java.text.Collator;
import java.util.Locale;

public final class NorwegianCollationFactory extends BaseTokenFilterFactory {

    @Override
    public TokenStream create(TokenStream input) {
        Collator norwegianCollator = Collator.getInstance(new Locale("no", "NO"));
        return new CollationKeyFilter(input, norwegianCollator);
    }

}

并且在您的 AnalyzerDef 中使用此排序规则工厂:

And the use this collation factory in your AnalyzerDef:

@AnalyzerDef(name = "myOwnAnalyzer",
tokenizer = @TokenizerDef(factory = KeywordTokenizerFactory.class),
filters = {
    @TokenFilterDef(factory = ASCIIFoldingFilterFactory.class),
    @TokenFilterDef(factory = LowerCaseFilterFactory.class),
    @TokenFilterDef(factory = PatternReplaceFilterFactory.class, params = {
        @Parameter(name = "pattern", value = "('-&\.,\(\))"),
        @Parameter(name = "replacement", value = " "),
        @Parameter(name = "replace", value = "all")
    }),
    @TokenFilterDef(factory = PatternReplaceFilterFactory.class, params = {
        @Parameter(name = "pattern", value = "([^0-9\p{L} ])"),
        @Parameter(name = "replacement", value = ""),
        @Parameter(name = "replace", value = "all")
    }),
    @TokenFilterDef(factory = TrimFilterFactory.class)
,
    @TokenFilterDef(factory = NorwegianCollationFactory .class)
}
)
public class KikaPaya implements Serializable {

有关在 hibernate 搜索版本 5 中使用此排序过滤器的更多信息 - https://stackoverflow.com/a/60738067/7179509

More information about using this Collation filter with hibernate search version 5 - https://stackoverflow.com/a/60738067/7179509

相关文章