不要等待在 Python 中使用 Selenium 加载页面

2022-01-16 00:00:00 python selenium selenium-webdriver webdriver pageloadstrategy

问题描述

如何让 selenium 在页面完全加载之前点击元素并抓取数据?我的互联网连接非常糟糕，所以有时需要很长时间才能完全加载页面，这有什么问题吗?

How do I make selenium click on elements and scrape data before the page has fully loaded? My internet connection is quite terrible so it sometimes takes forever to load the page entirely, is there anyway around this?

解决方案

ChromeDriver 77.0(支持 Chrome 77 版)现在支持 eager作为 pageLoadStrategy.

ChromeDriver 77.0 (which supports Chrome version 77) now supports eager as pageLoadStrategy.

已解决的问题 1902:支持急切页面加载策略 [Pri-2]

Resolved issue 1902: Support eager page load strategy [Pri-2]

<小时>

当你提到在页面完全加载之前点击元素并抓取数据在这种情况下，我们可以利用属性pageLoadStrategy.当 Selenium 默认加载页面/url 时，它遵循默认配置，将 pageLoadStrategy 设置为 normal.Selenium 可以从不同的文档就绪状态开始执行下一行代码.目前 Selenium 支持 3 种不同的 Document readiness state，我们可以通过 pageLoadStrategy 配置如下:

As you question mentions of click on elements and scrape data before the page has fully loaded in this case we can take help of an attribute pageLoadStrategy. When Selenium loads a page/url by default it follows a default configuration with pageLoadStrategy set to normal. Selenium can start executing the next line of code from different Document readiness state. Currently Selenium supports 3 different Document readiness state which we can configure through the pageLoadStrategy as follows:

  1. 无(未定义)
  2. eager(页面变为交互式)
  3. 正常(完成页面加载)

这是配置pageLoadStrategy的代码块:

Here is the code block to configure the pageLoadStrategy:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

binary = r'C:Program FilesMozilla Firefoxfirefox.exe'
caps = DesiredCapabilities().FIREFOX
# caps["pageLoadStrategy"] = "normal"  #  complete
caps["pageLoadStrategy"] = "eager"  #  interactive
# caps["pageLoadStrategy"] = "none"   #  undefined
driver = webdriver.Firefox(capabilities=caps, firefox_binary=binary, executable_path="C:\Utility\BrowserDrivers\geckodriver.exe")
driver.get("https://google.com")

相关文章