使用Spacy 3.0将数据从旧的Spacy v2格式转换为全新的Spacy v3格式

问题描述

我有变量trainData，其简化格式如下。

[

('Paragraph_A', {"entities": [(15, 26, 'DiseaseClass'), (443, 449, 'DiseaseClass'), (483, 496, 'DiseaseClass')]}),
('Paragraph_B', {"entities": [(969, 975, 'DiseaseClass'), (1257, 1271, 'SpecificDisease')]}),
('Paragraph_C', {"entities": [(0, 27, 'SpecificDisease')]})
]

我正在尝试将trainData转换为.spacy，方法是先在doc中将其转换，然后再将其转换为DocBin。可以通过GoogleDocs访问整个trainData文件。

我尝试重现本教程中提到的内容，但对我不起作用。本教程为：Using spaCy 3.0 to build a custom NER model

---

我尝试了以下操作。

import spacy
from spacy.tokens import DocBin

nlp = spacy.blank("en") # load a new spacy model
db = DocBin() # create a DocBin object

for text, annot in trainData: # data in previous format
    doc = nlp.make_doc(text) # create doc object from text
    ents = []
    for start, end, label in annot["entities"]: # add character indexes
        span = doc.char_span(start, end, label=label, alignment_mode="contract")
        ents.append(span)
    doc.ents = span # label the text with the ents
    db.add(doc)

db.to_disk("./train.spacy") # save the docbin object

但我在如何将数据从Spacy v2转换为Spacy v3的代码中弄错了。 在上面的代码片段中，我得到了一个回溯： TypeError: 'spacy.tokens.token.Token' object is not iterable。

---

解决方案

您有一个小错误。检查XXX是否有更改的行。

import spacy
from spacy.tokens import DocBin

nlp = spacy.blank("en") # load a new spacy model
db = DocBin() # create a DocBin object

for text, annot in trainData: # data in previous format
    doc = nlp.make_doc(text) # create doc object from text
    ents = []
    for start, end, label in annot["entities"]: # add character indexes
        span = doc.char_span(start, end, label=label, alignment_mode="contract")
        ents.append(span)
    #XXX FOLLOWING LINE CHANGED
    doc.ents = ents # label the text with the ents
    db.add(doc)

db.to_disk("./train.spacy") # save the docbin object