如何将 POINT 结构传递给 Python 中的 ElementFromPoint 方法?

2022-01-14 00:00:00 python com ui-automation microsoft-ui-automation comtypes

问题描述

我正在尝试使用方法 IUIAutomation::ElementFromPoint 在 Python 中使用 comtypes包裹.有很多示例如何在 C++ 中使用它，但在 Python 中没有.这个简单的代码在 64 位 Windows 10(Python 2.7 32 位)上重现了该问题:

I'm trying to use method IUIAutomation::ElementFromPoint in Python using comtypes package. There are many examples how to use it in C++, but not in Python. This simple code reproduces the problem on 64-bit Windows 10 (Python 2.7 32-bit):

import comtypes.client

UIA_dll = comtypes.client.GetModule('UIAutomationCore.dll')
UIA_dll.IUIAutomation().ElementFromPoint(10, 10)

我收到以下错误:

TypeError: Expected a COM this pointer as first argument

以这种方式创建 POINT 结构也无济于事:

Creating the POINT structure this way doesn't help as well:

from ctypes import Structure, c_long

class POINT(Structure):
    _pack_ = 4
    _fields_ = [
        ('x', c_long),
        ('y', c_long),
    ]

point = POINT(10, 10)
UIA_dll.IUIAutomation().ElementFromPoint(point) # raises the same exception

解决方案

可以直接复用已有的 POINT 结构定义，像这样:

You can reuse existing POINT structure definition directly, like this:

import comtypes
from comtypes import *
from comtypes.client import *

comtypes.client.GetModule('UIAutomationCore.dll')
from comtypes.gen.UIAutomationClient import *

# get IUIAutomation interface
uia = CreateObject(CUIAutomation._reg_clsid_, interface=IUIAutomation)

# import tagPOINT from wintypes
from ctypes.wintypes import tagPOINT
point = tagPOINT(10, 10)
element = uia.ElementFromPoint(point)

rc = element.currentBoundingRectangle # of type ctypes.wintypes.RECT
print("Element bounds left:", rc.left, "right:", rc.right, "top:", rc.top, "bottom:", rc.bottom)

要确定 ElementFromPoint 的预期类型是什么，您只需转到 python 设置目录(对我来说它是 C:Users<user>AppDataLocalProgramsPythonPython36Libsite-packagescomtypesgen) 并检查其中的文件.它应该包含由 comtypes 自动生成的文件，包括 UIAutomationCore.dll 的文件.有趣的文件名以 _944DE083_8FB8_45CF_BCB7_C477ACB2F897(COM 类型库的 GUID)开头.

To determine what's the expected type for ElementFromPoint, you can just go to your python setup directory (for me it was C:Users<user>AppDataLocalProgramsPythonPython36Libsite-packagescomtypesgen) and check the files in there. It should contains files automatically generated by comtypes, including the one for UIAutomationCore.dll. The interesting file name starts with _944DE083_8FB8_45CF_BCB7_C477ACB2F897 (the COM type lib's GUID).

该文件包含以下内容:

COMMETHOD([], HRESULT, 'ElementFromPoint',
          ( ['in'], tagPOINT, 'pt' ),

这告诉你它需要一个 tagPOINT 类型.这种类型被定义为文件的开头，如下所示:

This tells you that it expects a tagPOINT type. And this type is defined a the beginning of the file like this:

from ctypes.wintypes import tagPOINT

之所以命名为 tagPOINT，是因为它在原始 Windows 中是这样定义的标题.

It's named tagPOINT because that's how it's defined in original Windows header.

相关文章