是否可以使用Rvest刮除html节点内的子类以外的数据?
我有一个URL(https://forums.vwvortex.com/showthread.php?8829402-Atlas-V6-Oil-Change-Routine)
可以从中抓取帖子。其中一些帖子是回复,其初始文本为"原始发帖人……"。我想刮除最初通过文本发布的帖子内的所有数据。例如,
User df_text
A Hi, how are you ?
B This is beautiful!
C Heuwi
D Originally posted by C Heuwi
Hellou
E Hello guys
F Originally posted by A Hi, how are you ?
I am doing good
G Whats going on ?
对于用户D,"发帖人.."位于div.QUOTE_CONTAINER类(子类)下,而"I Am Do Good"位于父类BLOCQUTE.postcontent.Restore下。
预期结果:
User df_text
A Hi, how are you ?
B This is beautiful!
C Heuwi
D Hellou
E Hello guys
F I am doing good
G Whats going on ?
我尝试了以下代码:
url<-"https://forums.vwvortex.com/showthread.php?8829402-Atlas-V6-Oil-Change-Routine"
review <- read_html(url)
threads<- cbind(review %>% html_nodes("blockquote.postcontent.restore:not(.quote_container)") %>% html_text())
也尝试了其他几个:
threads <- cbind(review %>% html_nodes(xpath = '//div[@class="blockquote.postcontent.restore"]/node()[not(self::div)]') %>% html_text())
或
threads <- review %>% html_nodes(".content")
close_nodes <- threads %>% html_nodes(".quote_container")
chk <- xml_remove(close_nodes)
这些都不管用。请帮我找到一种方法来刮掉所有的帖子数据,不包括儿童类。提前感谢!!
解决方案
使用xml_remove
函数是相对容易的解决方案,该函数是xml2库的一部分(使用rvest自动加载)
library(rvest)
#read page
url<-"https://forums.vwvortex.com/showthread.php?8829402-Atlas-V6-Oil-Change-Routine"
review <- read_html(url)
#find parent nodes
threads<- review %>% html_nodes("blockquote.postcontent.restore:not(.quote_container)")
#find children nodes to exclude
toremove<-threads %>% html_node("div.bbcode_container")
#remove nodes
xml_remove(toremove)
#convert the parent nodes to text
threads %>% html_text(trim=TRUE)
来自xml_remove
的文档:"使用XML_Remove()时需要小心"。请经常审阅、谨慎使用并保存。
相关文章