senseidb中对zoie的整合

2022-04-27 00:00:00 索引 专区 目录 默认 实例

com.senseidb.conf.SenseiServerBuilder.buildCore()方法中开始对zoie进行实例化: 

1 构建ZoieConfig实例: 
ZoieConfig会设置Analyzer, Similarity实例,这些基本的lucene对象可以在sensei配置文件中自定义。设置batchSize, batchDelay, maxBatthSize, rtIndexing, skipBadRecord, freshness等属性也是在sensei配置中可配的。 

2. 构建SenseiZoieFactory 
sensei自己实现了一个默认的解释器,实现了ZoieIndxableInterpreter(用于把DataProvider产生的DataEvent转换成lucene Document) 

调用constructZoieFactory方法创建SenseiZoieFactory实例: 
private SenseiZoieFactory<?> constructZoieFactory( 
ZoieConfig zoieConfig, 
List<FacetHandler<?>> facetHandlers, 
List<RuntimeFacetHandlerFactory<?, ?>> runtimeFacetHandlerFactories, 
ZoieIndexableInterpreter interpreter) { 
2.1 
sensei自己实现了一个默认的IndexReader包装器 
SenseiIndexReaderDecorator decorator = new SenseiIndexReaderDecorator(facetHandlers, runtimeFacetHandlerFactories); 
这个装饰器整合了linkedIn另一个开源项目Bobo. Bobo提供一中结构化的模式来对索引进行查询。 

2.2 
构建SenseiZoieSystemFacotry实例 
SenseiZoieSystemFactory senseiZoieFactory = new SenseiZoieSystemFactory(idxDir, dirMode, interpreter, decorator, zoieConfig); 
idxDir是索引所在的目录 
dirMode是目录类型,默认是simple 

2.3 
实例化一个lucene的Filter实例purgeFilter, 过滤器,是根据时间进行过滤。同样是可配的 

2.4 
可选的IndexCopier,可能用来实现索引备份的。? 


3. 在实例化SenseiCore的时候,会传递SenseiZoieSystemFactory实例 


SenseiCore在启动的时候,会根据所在node的partition数量,建立相对应的IndexDirecory,每个目录,都有一个Zoie实例进行管理。 
for (int part : _partitions){ 
Zoie<BoboIndexReader,?> zoieSystem = _zoieFactory.getZoieInstance(_id,part); 
... 

所以在sensei.properties中指定的索引目录只是一个父目录,sensei会根据nodeId和partitionId进行子目录,子索引管理。 

Sensei会注册每个Zoie实例,通过JMS进行通信。 

	public void start() throws Exception {		if (_started)			return;		for (int part : _partitions) {			Zoie<BoboIndexReader, ?> zoieSystem = _zoieFactory.getZoieInstance(					_id, part);			// register ZoieSystemAdminMBean			String[] mbeannames = zoieSystem.getStandardMBeanNames();			for (String name : mbeannames) {				JmxUtil.registerMBean(zoieSystem.getStandardMBean(name),						"zoie-name", name + "-" + _id + "-" + part);			}			if (!zoieSystems.contains(zoieSystem)) {				zoieSystem.start();				zoieSystems.add(zoieSystem);			}			_readerFactoryMap.put(part, zoieSystem);		}		try {			pluggableSearchEngineManager.start(this);			logger.info("initializing index manager...");			if (_indexManager != null) {				_indexManager.initialize(_readerFactoryMap);			}			logger.info("starting index manager...");			if (_indexManager != null) {				_indexManager.start();			}			logger.info("index manager started...");		} catch (Exception e) {			logger.error(					"Unable to start indexing manager, indexing not started...",					e);		}		_started = true;	}

相关文章