[PATCH] Create aurweb.spawn for spawing the test server
This program makes it easier for developers to spawn the PHP server since it fetches automatically what it needs from the configuration file, rather than having the user explicitly pass arguments to the php executable. When the setup gets more complicated as we introduce Python, aurweb.spawn will keep providing the same interface, while under the hood it is planned to support running multiple sub-processes. Its Python interface provides an way for the test suite to spawn the test server when it needs to perform HTTP requests to the test server. The current implementation is somewhat weak as it doesn’t detect when a child process dies, but this is not supposed to happen often, and it is only meant for aurweb developers. In the long term, aurweb.spawn will eventually become obsolete, and replaced by Docker or Flask’s tools. --- TESTING | 7 +-- aurweb/spawn.py | 107 +++++++++++++++++++++++++++++++++++++++++++ conf/config.defaults | 3 ++ 3 files changed, 114 insertions(+), 3 deletions(-) create mode 100644 aurweb/spawn.py diff --git a/TESTING b/TESTING index 4a1e6f4c..a5e08cb8 100644 --- a/TESTING +++ b/TESTING @@ -17,7 +17,8 @@ INSTALL. Ensure to enable the pdo_sqlite extension in php.ini. 3) Copy conf/config.defaults to conf/config and adjust the configuration - (pay attention to disable_http_login, enable_maintenance and aur_location). + Pay attention to disable_http_login, enable_maintenance, aur_location and + htmldir. Be sure to change backend to sqlite and name to the file location of your created test database. @@ -31,6 +32,6 @@ INSTALL. $ ./gendummydata.py out.sql $ sqlite3 path/to/aurweb.sqlite3 < out.sql -5) Run the PHP built-in web server: +5) Run the test server: - $ AUR_CONFIG='/path/to/aurweb/conf/config' php -S localhost:8080 -t /path/to/aurweb/web/html + $ AUR_CONFIG='/path/to/aurweb/conf/config' python -m aurweb.spawn diff --git a/aurweb/spawn.py b/aurweb/spawn.py new file mode 100644 index 00000000..5fa646b5 --- /dev/null +++ b/aurweb/spawn.py @@ -0,0 +1,107 @@ +""" +Provide an automatic way of spawing an HTTP test server running aurweb. + +It can be called from the command-line or from another Python module. + +This module uses a global state, since you can’t open two servers with the same +configuration anyway. +""" + + +import atexit +import argparse +import subprocess +import sys +import time +import urllib + +import aurweb.config +import aurweb.schema + + +children = [] +verbosity = 0 + + +class ProcessExceptions(Exception): + """ + Compound exception used by stop() to list all the errors that happened when + terminating child processes. + """ + def __init__(self, message, exceptions): + self.message = message + self.exceptions = exceptions + messages = [message] + [str(e) for e in exceptions] + super().__init__("\n- ".join(messages)) + + +def spawn_child(args): + """Open a subprocess and add it to the global state.""" + if verbosity >= 1: + print(f"Spawning {args}", file=sys.stderr) + children.append(subprocess.Popen(args)) + + +def start(): + """ + Spawn the test server. If it is already running, do nothing. + + The server can be stopped with stop(), or is automatically stopped when the + Python process ends using atexit. + """ + if children: + return + atexit.register(stop) + aur_location = aurweb.config.get("options", "aur_location") + aur_location_parts = urllib.parse.urlsplit(aur_location) + htmldir = aurweb.config.get("options", "htmldir") + spawn_child(["php", "-S", aur_location_parts.netloc, "-t", htmldir]) + + +def stop(): + """ + Stop all the child processes. + + If an exception occurs during the process, the process continues anyway + because we don’t want to leave runaway processes around, and all the + exceptions are finally raised as a single ProcessExceptions. + """ + global children + atexit.unregister(stop) + exceptions = [] + for p in children: + try: + p.terminate() + if verbosity >= 1: + print(f"Sent SIGTERM to {p.args}", file=sys.stderr) + except Exception as e: + exceptions.append(e) + for p in children: + try: + rc = p.wait() + if rc != 0 and rc != -15: + # rc = -15 indicates the process was terminated with SIGTERM, + # which is to be expected since we called terminate on them. + raise Exception(f"Process {p.args} exited with {rc}") + except Exception as e: + exceptions.append(e) + children = [] + if exceptions: + raise ProcessExceptions("Errors terminating the child processes:", + exceptions) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser( + prog='python -m aurweb.spawn', + description='Start aurweb\'s test server.') + parser.add_argument('-v', '--verbose', action='count', default=0, + help='increase verbosity') + args = parser.parse_args() + verbosity = args.verbose + start() + try: + while True: + time.sleep(60) + except KeyboardInterrupt: + stop() diff --git a/conf/config.defaults b/conf/config.defaults index 447dacac..86fe765c 100644 --- a/conf/config.defaults +++ b/conf/config.defaults @@ -41,6 +41,9 @@ cache = none cache_pkginfo_ttl = 86400 memcache_servers = 127.0.0.1:11211 +; Directory containing aurweb's PHP code, required by aurweb.spawn. +;htmldir = /path/to/web/html + [ratelimit] request_limit = 4000 window_length = 86400 -- 2.26.0
On Sun, 19 Apr 2020 at 14:11:02, Frédéric Mangano-Tarumi wrote:
This program makes it easier for developers to spawn the PHP server since it fetches automatically what it needs from the configuration file, rather than having the user explicitly pass arguments to the php executable.
When the setup gets more complicated as we introduce Python, aurweb.spawn will keep providing the same interface, while under the hood it is planned to support running multiple sub-processes.
Its Python interface provides an way for the test suite to spawn the test server when it needs to perform HTTP requests to the test server.
The current implementation is somewhat weak as it doesn\u2019t detect when a child process dies, but this is not supposed to happen often, and it is only meant for aurweb developers.
In the long term, aurweb.spawn will eventually become obsolete, and replaced by Docker or Flask\u2019s tools. --- TESTING | 7 +-- aurweb/spawn.py | 107 +++++++++++++++++++++++++++++++++++++++++++ conf/config.defaults | 3 ++ 3 files changed, 114 insertions(+), 3 deletions(-) create mode 100644 aurweb/spawn.py
Merged into pu, thanks!
participants (2)
-
Frédéric Mangano-Tarumi
-
Lukas Fleischer