[PATCH] First HTTP functional test of the RPC interface

[Filipe Laíns]

On Sun, 2020-04-12 at 18:55 +0200, Frédéric Mangano-Tarumi wrote:
> Though barely useful in its current state, it shows how to integrate
> pytest with SQLAlchemy and Werkzeug, providing a test framework for the
> potential future Flask port, while actually testing the current PHP
> implementation.
> ---
>  aurweb/test/__init__.py |  0
>  aurweb/test/conftest.py | 51 +++++++++++++++++++++++++++++++++++++++++
>  aurweb/test/test_rpc.py | 21 +++++++++++++++++
>  aurweb/test/wsgihttp.py | 38 ++++++++++++++++++++++++++++++
>  test/README.md          |  3 +++
>  test/rpc.t              |  2 ++
>  6 files changed, 115 insertions(+)
>  create mode 100644 aurweb/test/__init__.py
>  create mode 100644 aurweb/test/conftest.py
>  create mode 100644 aurweb/test/test_rpc.py
>  create mode 100644 aurweb/test/wsgihttp.py
>  create mode 100755 test/rpc.t
> 
> diff --git a/aurweb/test/__init__.py b/aurweb/test/__init__.py
> new file mode 100644
> index 00000000..e69de29b
> diff --git a/aurweb/test/conftest.py b/aurweb/test/conftest.py
> new file mode 100644
> index 00000000..49cc2f6e
> --- /dev/null
> +++ b/aurweb/test/conftest.py
> @@ -0,0 +1,51 @@
> +"""
> +Fixtures for pytest.
> +
> +This module is automatically loaded by pytest.
> +"""
> +
> +import pytest
> +import sqlalchemy
> +import werkzeug.test
> +import werkzeug.wrappers
> +import werkzeug.wrappers.json
> +
> +import aurweb.config
> +import aurweb.db
> +from aurweb.test.wsgihttp import WsgiHttpProxy
> +
> +
> +class Response(werkzeug.wrappers.CommonResponseDescriptorsMixin,
> +               werkzeug.wrappers.json.JSONMixin,
> +               werkzeug.wrappers.BaseResponse):
> +    """
> +    Custom response object to be returned by the test client. More mixins could
> +    be added if need be.
> +
> +    See https://werkzeug.palletsprojects.com/en/1.0.x/wrappers/#mixin-classes
> +    """
> +    pass
> +
> +
> + at pytest.fixture
> +def client():
> +    """
> +    Build a Werkzeug test client for making HTTP requests to AUR. It requires
> +    that the AUR test website is already running at `[options] aur_location`,
> +    specified in the configuration file.
> +
> +    When aurweb becomes a pure Flask application, this should return Flask’s
> +    test_client(), which is a Werkzeug test client too.
> +    https://flask.palletsprojects.com/en/1.1.x/testing/#the-testing-skeleton
> +    """
> +    base_uri = aurweb.config.get("options", "aur_location")
> +    proxy = WsgiHttpProxy(base_uri)
> +    return werkzeug.test.Client(proxy, Response)
> +
> +
> + at pytest.fixture(scope="session")
> +def db_engine():
> +    """
> +    Return an SQLAlchemy engine to the configured database.
> +    """
> +    return sqlalchemy.create_engine(aurweb.db.get_sqlalchemy_url())
> diff --git a/aurweb/test/test_rpc.py b/aurweb/test/test_rpc.py
> new file mode 100644
> index 00000000..7079145c
> --- /dev/null
> +++ b/aurweb/test/test_rpc.py
> @@ -0,0 +1,21 @@
> +"""
> +Test suite for the RPC interface.
> +
> +See also `doc/rpc.txt` for the RPC interface documentation.
> +"""
> +
> +import pytest
> +from sqlalchemy.sql import select
> +
> +from aurweb.schema import Packages
> +
> +
> +def test_search_by_name(client, db_engine):
> +    """Take a package from the database, and find it through the RPC interface."""
> +    with db_engine.connect() as conn:
> +        pkg = conn.execute(select([Packages]).limit(1)).fetchone()
> +        if pkg is None:
> +            pytest.skip("needs at least one package in the database")
> +    resp = client.get("/rpc/", query_string={"v": "5", "type": "search", "arg": pkg["Name"]})
> +    result = resp.json
> +    assert result["resultcount"] >= 1
> diff --git a/aurweb/test/wsgihttp.py b/aurweb/test/wsgihttp.py
> new file mode 100644
> index 00000000..5b9d8040
> --- /dev/null
> +++ b/aurweb/test/wsgihttp.py
> @@ -0,0 +1,38 @@
> +import http.client
> +import urllib.parse
> +
> +
> +class WsgiHttpProxy:
> +    """
> +    WSGI-to-HTTP proxy, that is to say a WSGI application that forwards every
> +    WSGI request to an HTTP server, then the HTTP response back to WSGI.
> +
> +    The base URL the constructor takes is something like
> +    `http://localhost:8080`. It must not contain a path, a query string or a
> +    fragment, as the proxy wouldn’t now what to do with it.
> +
> +    Only the HTTP scheme is supported, but HTTPS could probably be easily added.
> +    """
> +
> +    def __init__(self, base_url):
> +        parts = urllib.parse.urlsplit(base_url)
> +        self.netloc = parts.netloc
> +        # Limitations of this dumb proxy
> +        assert parts.scheme == "http"
> +        assert parts.path in ("", "/")

Why a tuple?

> +        assert parts.query == ""
> +        assert parts.fragment == ""
> +
> +    def __call__(self, environ, start_response):
> +        conn = http.client.HTTPConnection(self.netloc)
> +        conn.request(
> +            method=environ["REQUEST_METHOD"],
> +            url=urllib.parse.urlunsplit((
> +                "http", self.netloc,
> +                urllib.parse.quote(environ["PATH_INFO"]),
> +                environ["QUERY_STRING"], "")),
> +            body=environ["wsgi.input"].read(int(environ.get("CONTENT_LENGTH", 0))),
> +        )
> +        resp = conn.getresponse()
> +        start_response(f"{resp.status} {resp.reason}", resp.getheaders())
> +        return resp
> diff --git a/test/README.md b/test/README.md
> index de7eff18..cc8baf33 100644
> --- a/test/README.md
> +++ b/test/README.md
> @@ -20,8 +20,11 @@ For all the test to run, the following Arch packages should be installed:
>  - python-bleach
>  - python-markdown
>  - python-pygit2
> +- python-pytest
> +- python-pytest-tap
>  - python-sqlalchemy
>  - python-srcinfo
> +- python-werkzeug
>  
>  Writing tests
>  -------------
> diff --git a/test/rpc.t b/test/rpc.t
> new file mode 100755
> index 00000000..f950f7df
> --- /dev/null
> +++ b/test/rpc.t
> @@ -0,0 +1,2 @@
> +#!/bin/sh
> +pytest --tap-stream "$(dirname "$0")/../aurweb/test/test_rpc.py"

I think we should be using pytest-runner here. What do you think Lukas?

Otherwise LGTM.

Cheers,
Filipe Laíns
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <https://lists.archlinux.org/pipermail/aur-dev/attachments/20200416/0ba99799/attachment.sig>