如何在 Selenium 中的 HTML5 Canvas 上执行鼠标滚轮滚动?

2022-01-17 00:00:00 selenium selenium-webdriver html5-canvas java mousewheel

我正在开发一个 GWT 应用程序(类似于 Paint).在此，我有一个 HTML5 画布，其中有一个功能是上下滚动鼠标滚轮将放大和缩小画布.

I am working on a GWT Application (similar to Paint). In this, I have an HTML5 Canvas in which there is a functionality that scrolling a mousewheel up and down will zoom in and out of the canvas.

我进行了很多搜索，但没有找到解决此问题的解决方法.做了什么:

I have searched a lot but didn't find a workaround to fix this issue. Here's what did:

int PosX = 0;
int PosY = 10;
JavascriptExecutor executor = (JavascriptExecutor) getDriver();
String script = "document.getElementById('frontCanvas').scrollBy("
                                + PosX + "," + PosY + ")";
executor.executeScript(script); 

WebDriverWait wait = new WebDriverWait(getDriver(), 20);
wait.until(ExpectedConditions.javaScriptThrowsNoExceptions(script));

现在，上面的代码适用于另一个 Angular 应用程序，我在其中上下滚动一个 div 元素(它有一个滚动条)，但它不适用于我的画布(它没有滚动条)GWT 应用程序.

Now, this above code is working for another Angular application in which I am scrolling up and down a div element (which has a scrollbar) but it is not working on my canvas (which doesn't have a scrollbar) in the GWT application.

我正在使用 Selenium 3.14.0 并在 Chrome 浏览器上运行此代码.谁能建议可以做些什么来解决这个问题?

I am using Selenium 3.14.0 and run this code on the Chrome browser. Can anyone suggest what can be done to fix this issue?

推荐答案

HTML5 Canvas

HTML 元素用于通过 JavaScript 动态绘制图形.元素只是图形的容器.您必须使用 JavaScript 来实际绘制图形.Canvas 有多种方法用于绘制路径、框、圆、文本和添加图像.

HTML5 Canvas

The HTML element is used to draw graphics, on the fly, via JavaScript. The element is only a container for graphics. You must use JavaScript to actually draw the graphics. Canvas has several methods for drawing paths, boxes, circles, text, and adding images.

一般来说，scroll mousewheel up 和 down 我们可以选择 Actions 类.但根据 使用 Selenium WebDriver 自动测试 HTML5 Canvas 应用程序 看来这个 API 不是那么可靠.在 Firefox 中，每个 mouse down、mouse up 或 mouse click 都发生在元素的中心.所以上面的代码对提供的 (x,y) 产生一个 mouse move 事件，然后是一个 mouse move事件到 Canvas 的中心，然后是 mouse down、mouse up 和 click 都在 Canvas 的中心.这对于按钮来说可能很好，但对于 Canvas 是行不通的，您希望能够在其中 hover、click 等一个特定的位置.在 Safari 中情况更糟，它只会产生一个异常，表明不支持鼠标移动事件.与此同时，Chrome 运行良好.

In general, to scroll the mousewheel up and down we could have opted for the Actions Class. But as per Automated Testing of HTML5 Canvas Applications with Selenium WebDriver it seems this API is not that reliable. In Firefox, every mouse down, mouse up, or mouse click happens at the center of the element. So the code above produces a mouse move event to the provided (x,y), then a mouse move event to the center of the Canvas, then a mouse down, mouse up, and click all at the center of the Canvas. That may be fine for a button, but is unworkable for a Canvas, where you want to be able to hover, click, etc. at a specific location. The situation is even worse in Safari, where it just produces an exception indicating that mouse move events aren't supported. Chrome, meanwhile, works fine.

解决方法是使用 JavascriptExecutor 使用 JavaScript 手动调度合成鼠标事件的接口.

An work around would be to use the JavascriptExecutor Interface manually dispatching synthesized mouse events using JavaScript.

借鉴@FlorentB.的回答，滚动 鼠标滚轮 向上和向下，您可以发出mouseover、mousemove 和 wheel events 通过脚本注入到顶部元素，您可以使用以下解决方案:

Taking a leaf out of @FlorentB.'s answer, to scroll a mousewheel up and down, you can emit the mouseover, mousemove and wheel events to the top element with a script injection and you can use the following solution:

  • 代码块:

  • Code Block:

package demo;

import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class Canvas {

    static WebDriver driver;
    public static void main(String[] args) {

        System.setProperty("webdriver.chrome.driver", "C:\Utility\BrowserDrivers\chromedriver.exe");
        ChromeOptions options = new ChromeOptions();
        options.addArguments("start-maximized");
        options.addArguments("disable-infobars");
        options.addArguments("--disable-extensions");
        driver = new ChromeDriver(options);
        driver.get("https://www.google.co.uk/maps");
        WebElement elm = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("#scene > div.widget-scene > canvas")));
        // Mouse wheel UP or Zoom In 
        wheel_element(elm, -500, 0, 0);
        System.out.println("Mouse wheel UP or Zoom In through Wheel achieved !!!");
        // Mouse wheel DOWN or Zoom Out 
        wheel_element(elm, 120, 0, 0);
        System.out.println("Mouse wheel DOWN or Zoom Out through Wheel achieved !!!");
        System.out.println("Mouse Scroll through Wheel achieved !!!");
    }

    public static void wheel_element(WebElement element, int deltaY, int offsetX, int offsetY)
    {
        try{
              String script = "var element = arguments[0];"
                +"var deltaY = arguments[1];"
                +"var box = element.getBoundingClientRect();"
                +"var clientX = box.left + (arguments[2] || box.width / 2);"
                +"var clientY = box.top + (arguments[3] || box.height / 2);"
                +"var target = element.ownerDocument.elementFromPoint(clientX, clientY);"
                +"for (var e = target; e; e = e.parentElement) {"
                  +"if (e === element) {"
                +"target.dispatchEvent(new MouseEvent('mouseover', {view: window, bubbles: true, cancelable: true, clientX: clientX, clientY: clientY}));"
                +"target.dispatchEvent(new MouseEvent('mousemove', {view: window, bubbles: true, cancelable: true, clientX: clientX, clientY: clientY}));"
                +"target.dispatchEvent(new WheelEvent('wheel',     {view: window, bubbles: true, cancelable: true, clientX: clientX, clientY: clientY, deltaY: deltaY}));"
                +"return;"
                  +"}"
                +"}";  

              WebElement parent = (WebElement) ((JavascriptExecutor) driver).executeScript("return arguments[0].parentNode;", element);
              ((JavascriptExecutor) driver).executeScript(script, parent, deltaY, offsetX, offsetY);
            }catch(WebDriverException e)
            {
            System.out.println("Exception caught in Catch block");
            }
    }

}

  • 控制台输出:

  • Console Output:

    Mouse wheel UP or Zoom In through Wheel achieved !!!
Mouse wheel DOWN or Zoom Out through Wheel achieved !!!
Mouse Scroll through Wheel achieved !!!

    • 您可以在以下位置找到一些相关的详细讨论:

    You can find a couple of relevant detailed discussion in:

    • 如何在画布上找到按钮的坐标，并在使用Java和Selenium之后点击它们?

    相关文章