Selenium Threads:如何使用代理运行多线程浏览器(python)

2022-01-16 00:00:00 python selenium multithreading webdriver

问题描述

我正在编写一个脚本来使用具有多个线程的代理访问网站，但现在我被困在多个线程中，当我运行下面的脚本时，它会打开 5 个浏览器，但所有 5 个都使用 1 个代理，我想要 5 个浏览器使用不同的代理，有人可以帮我完成吗?谢谢你

I'm writing a script to access a website using proxies with multiple threads but now I'm stuck in multiple threads, when I run the script below, it opens 5 browsers but all 5 use 1 proxy, I want 5 browsers to use different proxies, can someone help me complete it? thank you

这是我的脚本:

from selenium import webdriver
from selenium import webdriver
import time , random
import threading


def e():

    a = open("sock2.txt", "r")
    for line in a.readlines():

        b = line
        prox = b.split(":")
        IP = prox[0]
        PORT = int(prox[1].strip("
"))
        print(IP)
        print(PORT)


        profile = webdriver.FirefoxProfile()
        profile.set_preference("network.proxy.type", 1)
        profile.set_preference("network.proxy.socks", IP)
        profile.set_preference("network.proxy.socks_port", PORT)
        try:

            driver = webdriver.Firefox(firefox_profile=profile)
            driver.get("http://www.whatsmyip.org/")
        except:
            print("Proxy Connection Error")
            driver.quit()
        else:
            time.sleep(random.randint(40, 70))
            driver.quit()
for i in range(5):
    t = threading.Thread(target=e)
    t.start()

(祝大家新年快乐，万事如意)

(Wish everyone has a happy and lucky new year)

解决方案

Dominik Lašo 正确捕获了它 - 每个线程从头开始处理文件.这可能是它的样子:

Dominik Lašo captured it correctly - each threads processes the file from the beginning. Here's probably how it should look like:

from selenium import webdriver
from selenium import webdriver
import time , random
import threading


def e(ip, port):
    profile = webdriver.FirefoxProfile()
    profile.set_preference("network.proxy.type", 1)
    profile.set_preference("network.proxy.socks", IP)
    profile.set_preference("network.proxy.socks_port", PORT)
    try:
        driver = webdriver.Firefox(firefox_profile=profile)
        driver.get("http://www.whatsmyip.org/")
    except:
        print("Proxy Connection Error")
        driver.quit()
    else:
        time.sleep(random.randint(40, 70))
        driver.quit()

my_threads = []
with open("sock2.txt", "r") as fd:
    for line in fd.readlines():
        line = line.strip()
        if not line:
           continue
        prox = line.split(":")
        ip = prox[0]
        port = int(prox[1])
        print('-> {}:{}'.format(ip, port))
        t = threading.Thread(target=e, args=(ip, port,))
        t.start()
        my_threads.append(t)

for t in my_threads:
    t.join()

相关文章