使用无服务器框架将Ploly/Dash应用部署到AWS

问题描述

我正在尝试使用无服务器框架将Ploly Dash应用程序部署为AWS Lambda。该应用程序在本地运行正常，我可以使用serverless wsgi serve命令启动它。serverless deploy报告成功。但是，调用lambda时会失败，并显示以下错误：

Traceback (most recent call last):
File "/var/task/wsgi_handler.py", line 44, in import_app
wsgi_module = importlib.import_module(wsgi_fqn_parts[-1])
File "/var/lang/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/var/task/app.py", line 4, in <module>
import dash
File "/tmp/sls-py-req/dash/__init__.py", line 5, in <module>
from .dash import Dash, no_update  # noqa: F401,E402
File "/tmp/sls-py-req/dash/dash.py", line 21, in <module>
from flask_compress import Compress
File "/tmp/sls-py-req/flask_compress.py", line 14, in <module>
import brotli
File "/tmp/sls-py-req/brotli.py", line 8, in <module>
import _brotli
ModuleNotFoundError: No module named '_brotli'
[ERROR] Exception: Unable to import app.server
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/var/lang/lib/python3.8/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/task/wsgi_handler.py", line 119, in <module>
    wsgi_app = import_app(config)
  File "/var/task/wsgi_handler.py", line 49, in import_app
    raise Exception("Unable to import 
{}
".format(config["app"]))

app.py

import dash
import dash_html_components as html
from flask import Flask

server = Flask(__name__)

@server.route("/")
def index():
    return "Hello Flask app"

app = dash.Dash(
    __name__,
    server=server,
    routes_pathname_prefix='/dash/'
)

app.layout = html.Div(html.H1("Hello Dash!"))

if __name__ == "__main__":
    server.run(debug=True)

serverless.yml

---
service: dash-serverless

variablesResolutionMode: 20210219
useDotenv: true

provider:
  name: aws
  runtime: python3.8
  stage: test
  region: eu-central-1
  apiGateway:
    shouldStartNameWithService: true
  lambdaHashingVersion: 20201221

functions:
  app:
    handler: wsgi_handler.handler
    events:
      - http: ANY /
      - http: "ANY {proxy+}"

custom:
  wsgi:
    app: app.server
    pythonBin: python3
    packRequirements: false
  pythonRequirements:
    dockerPip: non-linux

plugins:
  - serverless-wsgi
  - serverless-python-requirements

package:
  exclude:
    - node_modules/**
    # and other

Requirments.txt

-i https://pypi.org/simple
brotli==1.0.9
click==7.1.2; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'
dash-core-components==1.15.0
dash-html-components==1.1.2
dash-renderer==1.9.0
dash-table==4.11.2
dash==1.19.0
flask-compress==1.9.0
flask==1.1.2
future==0.18.2; python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'
itsdangerous==1.1.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
jinja2==2.11.3; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'
markupsafe==1.1.1; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
plotly==4.14.3
retrying==1.3.3
six==1.15.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
werkzeug==1.0.1; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'

---

解决方案

ModuleNotFoundError: No module named '_brotli'的原因是依赖项打包不正确。通过使用docker和docker-lambda镜像打包应用程序，修复了该问题。slim: true和strip: false最小化包大小，同时保留在某些情况下需要的二进制文件(在本例中确实需要)。

pythonRequirements:
    dockerizePip: true
    slim: true
    strip: false

解决了_brotli未找到的问题后，我遇到了下一个错误消息pkg_resources.DistributionNotFound: The 'flask-compress' distribution was not found and is required by the application。我能够使用Pyinstaller executable cannot find 'flask-compress' distribution that is included中描述的解决方法解决此问题。

最后，应用程序在/<api-stage-name>/(本例中为/test/)下提供服务。这需要修改Dash应用程序配置，即 在Dash构造函数中提供requests_pathname_prefix="/test/dash/"。

完整工作示例：

app.py

from collections import namedtuple
import pkg_resources

# backup true function
_true_get_distribution = pkg_resources.get_distribution
# create small placeholder for the dash call
# _flask_compress_version = parse_version(get_distribution("flask-compress").version)
_Dist = namedtuple('_Dist', ['version'])


def _get_distribution(dist):
    if dist == 'flask-compress':
        return _Dist('1.9.0'). # your flask-compress version
    else:
        return _true_get_distribution(dist)


# monkey patch the function so it can work once frozen and pkg_resources is of
# no help
pkg_resources.get_distribution = _get_distribution

import dash
import dash_html_components as html
from flask import Flask


server = Flask(__name__)


@server.route("/")
def index():
    return "Hello Flask app"


app = dash.Dash(
    __name__,
    server=server,
    routes_pathname_prefix="/dash/",
    requests_pathname_prefix="/test/dash/"
)

app.layout = html.Div(html.H1("Hello Dash!"))

if __name__ == "__main__":
    server.run(debug=True)

serverless.yml

---
service: dash-serverless

variablesResolutionMode: 20210219
useDotenv: true

provider:
  name: aws
  runtime: python3.8
  stage: test
  region: eu-central-1
  apiGateway:
    shouldStartNameWithService: true
  lambdaHashingVersion: 20201221

functions:
  app:
    handler: wsgi_handler.handler
    events:
      - http: ANY /
      - http: "ANY {proxy+}"

custom:
  wsgi:
    app: app.server
    pythonBin: python3
    packRequirements: false
  pythonRequirements:
    dockerizePip: true
    slim: true
    strip: false

plugins:
  - serverless-wsgi
  - serverless-python-requirements

package:
  exclude:
    - node_modules/**
    # and other