[arch-projects] [ABS] [RFC] the new abs utility design
Hello everyone, I am matt mooney, a.k.a. mfm, the new ABS developer and eventual maintainer ;) Currently, I am working on a new abs utility, and I would like opinions on the design before I fully implement it. mfm:pts/15[~/src/abs] Jun13 19:25:28 % ./abs usage: abs [-h] [--version] [--abs-root=<path>] [--build-dir=<path>] [--pkg-dir=<path>] [--noconfirm] <command>... [<pkgspec>...] optional arguments: -h, --help Print the synopsis and command list --version Print the ABS version and exit --abs-root <path> Path to the ABS tree --build-dir <path> Path to the build directory --pkg-dir <path> Path to the directory where packages will be output --noconfirm Bypass all confirmation messages generated by pacman commands: build Build packages in the ABS tree clean Clean the build directory install Install packages patch Apply patches to packages in the ABS tree. sync Synchronize the ABS tree with the current snapshot mfm:pts/15[~/src/abs] Jun13 19:25:30 % At first I was going to design it to run single commands, such as `abs build' but I felt that would lead to one of two things: 1) a lot of duplicate options to accomplish the other commands without specifying them, or 2) default command chains running to avoid explicitly having to run every command manually (i.e., an `abs install' would implicitly run an `abs sync' then an `abs build` followed by an `abs install'). So, instead, I decided to support explicit command chains, such as `abs sync build install clean emacs'. Nothing is set in stone yet. And I am really interested in hearing suggestions from other people on what they would like to see from the abs utility. I think one feature noticeably absent is the ability to rebuild all currently installed packages from source without having to explicitly specify them. The manpages have all been attached to this email for easier viewing using `man'; furthermore, each asciidoc manpage is threaded along with this email for inline comments. My git repo can be found at http://projects.archlinux.org/users/mfm/abs.git/. Thanks, matt
ABS(7) ====== :doctype: manpage :man source: ABS :man manual: ABS Manual :man version: %%ABS_VERSION%% Name ---- ABS - the Arch Build System Description ----------- The 'Arch Build System', or *ABS*, is a ports-like system for building binary packages from source code. With a collection of tools for package creation and management, it provides a complete package management system for *Arch Linux*. Typically, the process of building a program from source code would consist of downloading, unpacking, patching, compiling, and installing the program. Of course a few key steps were left out, such as building the required dependencies and configuring the software for your system. The *ABS* does 'all' of this for you! And it also creates a package that can be installed and tracked via *pacman*(8). To build a package from source code, the ABS uses *makepkg*(8) and a *PKGBUILD*(5). '`makepkg`' is a package creation tool that automates the build process, and a '`PKGBUILD`' is a build description file that contains the information necessary for building the package. When run, `makepkg` reads the PKGBUILD in the current directory, or the build script given with the `-p` option, and follows the instructions therein to build the package. This will result in *`<pkgname>-<pkgver>.pkg.tar.xz`*, which can then be installed with *`pacman`*. *abs*(8) is the utility that facilitates building packages from source code. It utilizes the other package management tools to accomplish its function. Overview -------- *abs*:: Fetches and creates the directory tree of PKGBUILDs for the official repositories. This hierarchy of directories is known as the *``ABS tree''* and has the same structure as the package database. It is created under '%%ABS_ROOT%%'. *makepkg*:: Compiles and builds a package that can be installed with `pacman`. Dependency handling is built-in and can be controlled through command-line options. A PKGBUILD is required to provide the build instructions. *pacman*:: The official package manager for 'Arch Linux'. Install or update a package after it is built by running either *`pacman -U <pkg>`* or *`makepkg -i`*. *PKGBUILD*:: A build description file that contains the URL of the source code as well as instructions for compilation and packaging. Files ----- *%%ABS_CONF_FILE%%*:: ABS tree configuration file. *%%PACMAN_CONF_DIR%%/makepkg.conf*:: Build related configuration file. *%%PACMAN_CONF_DIR%%/pacman.conf*:: Package management configuration file. See Also -------- *abs*(8), *makepkg*(8), *pacman*(8), *PKGBUILD*(5), *abs.conf*(5), *makepkg.conf*(5), *pacman.conf*(5) include::footer.txt[]
abs(8) ====== :doctype: manpage :man source: abs :man manual: ABS Manual :man version: %%ABS_VERSION%% Name ---- abs - the Arch Build System utility SYNOPSIS -------- [verse] 'abs' [-h] [--version] [--abs-root=<path>] [--build-dir=<path>] [--pkg-dir=<path>] [--noconfirm] <command>... [<pkgspec>...] Description ----------- *`abs`* is a utility used to interact with the 'Arch Build System', *ABS*(7). It provides a rich set of commands to facilitate building packages from source code. The *``ABS tree''* contains the PKGBUILDs used by Arch Linux for creating packages: it is a snapshot of the official package database and the central structure with which *`abs`* operates. `abs` allows full or partial synchronization of the ABS tree through a `<pkgspec>`. Every `abs` '<command>' takes zero or more `<pkgspec>` arguments. Each `<pkgspec>` defines a nonempty set of packages. The union of `<pkgspec>s` is known as the *working set* for that invocation of `abs`. If a `<pkgspec>` is not given, the *working set* is generated entirely from the configuration variables set in 'abs.conf'. The Package Specification ~~~~~~~~~~~~~~~~~~~~~~~~~ A '<pkgspec>' is composed of three parts: an architecture (``<arch>''), a repository (``<repo>''), and a package name (``<pkg>''). The fully specified `<pkgspec>` is `<arch>/<repo>/<pkg>`; however, *`abs`* can deduce the `<pkgspec>` from variables set in the configuration file. Valid package specifications are *`<pkg>`*, *`<repo>/<pkg>`*, and *`<arch>/<repo>/<pkg>`*. When a `<pkgspec>` is not fully specified, the *`repos`* variable determines the precedence of the repositories to search, and the *`arch`* variable determines which architectures to include. Please see *abs.conf*(5) for more details. Options ------- *-h, --help*:: Print the synopsis and command list. *--version*:: Print the ABS version and exit. *--abs-root*=<path>:: Path to the ABS tree. *--build-dir*=<path>:: Path to the build directory. *--pkg-dir*=<path>:: Path to the directory where packages will be output. *--noconfirm*:: Bypass all confirmation messages generated by *`pacman`*. Commands -------- *build*:: Build packages in the *ABS tree*. The 'ABS files' for each package in the *working set* are copied into a subdirectory and built; moreover, missing dependencies are automatically added to the *working set* and treated as if they were explicitly included. Files created during the build process are stored within a subdirectory named for the package and will *not* be overwritten across builds. The resulting packages will be in *`pkgdir`*. '%%ABS_BUILD_DIR%%' is provided for building packages, but you can use a different build directory. *-b, --builddeps*;; Build missing dependencies. The 'default' setting. *-d, --nodeps*;; Do not perform dependency checks. A build failure will occur if build dependencies are missing (i.e., *`makedepends`* is not empty). *-f, --force*;; Recopy ABS files to the build directory. This overwrites the old package subdirectory. *-s, --syncdeps*;; Install missing dependencies using *`pacman`*. *clean*:: Clean the build directory. The files created during the build process are removed from each package in the build directory. *-p, --package*;; Revert patches to the ABS files and source code. *install*:: Install packages. The packages will be installed using *`pacman`*, which should be used for further management operations. All packages found in the *`pkgdir`* that are newer than an installed version will be reinstalled. *patch*:: Apply patches to packages in the *ABS tree*. Packages that are not already in the build directory will be retrieved, accordingly. The 'ABS files' are copied into a subdirectory and patched; next, the source code will be downloaded and patched. If a package in the *working set* has formerly been patched, it will be skipped. // The following line is needed to get the correct indentation!! + The patches must be in the unified format and located under *`patchdir`*; the default is '%%ABS_PATCH_DIR%%'. This should consist of subdirectories containing the patches. Each subdirectory is to be named after the package to which its patches will be applied. // The following line is needed to get the correct indentation!! + The PKGBUILD patch must be named *PKGBUILD.patch*; the remaining patches must end in *.patch* and be for the packages source code. The patches are applied from the source directory in an unspecified manner. (A patch series may be supported in the future!) *sync*:: Synchronize the *ABS tree* with the current snapshot. It allows full or partial syncing of repositories; however, it overwrites the part of the ABS tree that is newly synced. Without arguments, the *`pkgs`* in *`repos`* are synchronized. If the user running this command does not have write access to the ABS root directory, the command will fail. *-r, --rsync*;; Synchronize the ABS tree using rsync. The *`syncserver`* must be running '`rsyncd`'. This is the 'default' method. *-t, --tarball*;; Synchronize the ABS tree using a tarball. The first functioning server in *`mirrorlist`* that contains the ABS tarball will be used. NOTE: When synchronizing the *ABS tree*, only one of the two methods can be used at a time. The options are mutually exclusive and an error will occur if both options are given. Examples -------- A sample workflow to build and install *`abs`*: *abs sync x86_64/extra/abs*:: Sync the *ABS tree* with only the abs package. *abs patch x86_64/extra/abs*:: Add my innovative patch set. *abs build x86_64/extra/abs*:: Build the package. *abs install x86_64/extra/abs*:: Install the package. *abs clean x86_64/extra/abs*:: Clean the build directory but leave my innovative patch set applied. This is quite a tedious process for a single package. Fortunately, *`abs`* supports chaining the commands together. *abs sync patch build install clean x86_64/extra/abs*:: One caveat though, the order of the commands is important but not enforced. Environment ----------- *ABS_ROOT*:: The path to the *ABS tree*. Overrides the value set in '%%ABS_CONF_FILE%%' and can also be set with `--abs-root`. *ABS_BUILD_DIR*:: The path to the ABS build directory. Overrides the value set in '%%ABS_CONF_FILE%%' and can also be set with `--build-dir`. Files ----- *%%ABS_ROOT%%*:: The default location of the ABS tree. *%%ABS_CONF_FILE%%*:: System-wide configuration file. The configuration variables and file format are described in *abs.conf*(5). See Also -------- *abs.conf*(5), *ABS*(7), *PKGBUILD*(5), include::footer.txt[]
abs.conf(5) =========== :doctype: manpage :man source: abs.conf :man manual: ABS Manual :man version: %%ABS_VERSION%% Name ---- abs.conf - the Arch Build System configuration file SYNOPSIS -------- [verse] '~/.abs.conf' '%%ABS_CONF_FILE%%' Description ----------- The *`abs`* configuration file contains a number of variables that affect the utility's behavior; it uses the ``INI file format.'' There are three major sections: *`abs`*, *`makepkg`*, and *`pacman`*. Each section contains the configuration variables that affect abs's invocation of that utility; however, within the *makepkg* and *pacman* sections, not every variable coincides to the named utility. Some are specific to the functioning of the *`abs <command>`* that utilizes it. Configuration data is obtained from the following sources, ordered by precedence from highest to lowest: . command-line arguments . environment variables . user's configuration file . system-wide configuration file Syntax ~~~~~~ Every variable must be within a section and is case sensitive. A variable can be set as *name = value* or *name : value*, but do *not* quote the values. Leading and trailing whitespace is ignored; however, multi-line values require subsequent lines to be indented with at least one space. Comments begin with either the *#* or *;* character and continue to the end of the line. Array elements need to be separated by whitespace, *not* commas. Boolean values can be set as true/false, 1/0, yes/no, or on/off. Strings are case sensitive and should not contain whitespace or be quoted. Variables --------- Configuration variables separated by section. [abs] ~~~~~~~~~~~~~ The standard variables that affect every *`abs <command>`*. *absroot*:: The location of the *ABS tree*. Either an absolute or relative path can be given. The user running `abs` does not require write access unless an *`abs sync`* is issued. Defaults to '%%ABS_ROOT%%'. *builddir*:: The directory used for building the packages. Either an absolute or relative path can be given, and write access is required for this directory. Defaults to '%%ABS_BUILD_DIR%%'. *pkgdir*:: The directory where packages will be output. Either an absolute or relative path can be given, and write access is required for this directory. Defaults to '%%ABS_PKG_DIR%%'. *patchdir*:: The directory containing patches for the source code or ABS files. Either an absolute or relative path can be given, and write access is required for this directory. Defaults to '%%ABS_PATCH_DIR%%'. // The following line is needed to get the correct indentation!! + Patches must be in the unified format and in a directory with the same name as the package to which it will be applied (i.e., '%%ABS_PATCH_DIR%%/<pkg>'). *arch*:: The architectures to include in the *ABS tree*. This value is an array and all of the supported architectures can be set here. Nevertheless, when a cross compiler is not used, trying to build for an architecture not supported by your machine will fail. Defaults to 'i686'. *repos*:: An ordered array of repositories. When only a package name is provided, the array's order establishes the precedence of the repositories to search. Two situations cause this to occur: 1) the form of the `<pkgspec>` used is `<pkg>`, or 2) a `<pkgspec>` is *not* given and the *`pkgs`* variable is set to something other than *ALL*. Defaults to ('core extra community multilib'). *pkgs*:: The array of packages on which `abs` is to operate. When a `<pkgspec>` is *not* given and *`pkgs = ALL`*, the *working set* consists of all packages found in *`repos`*. Defaults to 'ALL'. *syncserver*:: The server to use for synchronization of the *ABS tree*. '`rsyncd`' must be running and setup properly for the ABS. A sample stanza can be found in the conf directory of the abs source code. Defaults to 'rsync.archlinux.org'. *mirrorlist*:: The mirror list used for syncing via tarball. Must be in the same format as defined in *REPOSITORY SECTIONS* of *pacman.conf*(5). Defaults to '/etc/pacman.d/mirrorlist'. [makepkg] ~~~~~~~~~~~~~~~~~ `makepkg` is used during an *`abs build`*, so these variables affect that command. *nodeps*:: A boolean value to prevent dependency checks. This will cause a build failure if the *makedepends* variable for a package is not empty. Moreover, the program will fail to run if installed without packages listed in *depends*. Defaults to 'false'. *builddeps*:: Build missing dependencies. If set to *true*, any missing dependencies will be added to the *working set* and treated as if they were explicitly included. Defaults to 'true'. *syncdeps*:: Install missing dependencies via *`pacman`*. If set to *true*, any missing dependencies will be downloaded and installed. Defaults to 'false'. NOTE: *builddeps* and *syncdeps* are mutually exclusive options that should never be 'true' at the same time. That said, since *builddeps* is 'true' by default, it will take priority over *syncdeps* if both are 'true'. [pacman] ~~~~~~~~~~~~~~~~ `pacman` is used during an *`abs install`* or *`abs build --syncdeps`*, so these variables affect those command. *noconfirm*:: A boolean value to bypass all confirmation messages. Defaults to 'false'. See Also -------- *abs*(8), *ABS*(7) include::footer.txt[]
participants (1)
-
matt mooney