[pacman-dev] A makepkg test suite
I've recently come across issue FS#15645 ([makepkg] needs a test suite) and I'd like to have a bash at implementing it. It seems like a good idea to get peoples ideas and thoughts first so what do you think? My first reaction is to do something similar to the pacman tests and just write a custom framework in python. Also have the output the same as the pacman tests. I think keeping the idea an environment (like in the pacman tests) seems a bit heavy handed because the only configuration done is in makepkg.conf and we can probably fake a pacman environment using a stub pacman. Or maybe we could utilise the pacman test environments! Unit testing should be easy enough because we can just call specific functions in makepkg and check the output. For functional testing I'm not really sure. As for acceptance testing we can just put different flag combinations in different test environments, and check the return code. Check the .PKGINFO. Maybe that files were put where they should be in pkg and src, as well as in the package itself. Have some PKGBUILDs that fail for various reasons like syntax, a dependency package doesn't exist. Make sure missing {,make,check}dependencies are installed. Any suggestions or ideas? Ashley
On Fri, Jul 12, 2013 at 08:00:37PM +0100, Ashley Whetter wrote:
I've recently come across issue FS#15645 ([makepkg] needs a test suite) and I'd like to have a bash at implementing it. It seems like a good idea to get peoples ideas and thoughts first so what do you think?
My first reaction is to do something similar to the pacman tests and just write a custom framework in python. Also have the output the same as the pacman tests.
I think keeping the idea an environment (like in the pacman tests) seems a bit heavy handed because the only configuration done is in makepkg.conf and we can probably fake a pacman environment using a stub pacman. Or maybe we could utilise the pacman test environments!
Unit testing should be easy enough because we can just call specific functions in makepkg and check the output.
For functional testing I'm not really sure.
As for acceptance testing we can just put different flag combinations in different test environments, and check the return code. Check the .PKGINFO. Maybe that files were put where they should be in pkg and src, as well as in the package itself. Have some PKGBUILDs that fail for various reasons like syntax, a dependency package doesn't exist. Make sure missing {,make,check}dependencies are installed.
Any suggestions or ideas? Ashley
Incoming wall of text and can of worms. I could have gone on, but I feel strongly about this because I've been down this road before and it's a shitfest. -------- I fear that you're unfamiliar with the various pitfalls that make testing shell scripts so incredibly unwieldy and error prone. There's even more pitfalls than simply *writing* shell. The greatest hurdle is this: the only parser fit to parse bash scripts is bash itself. Notice that currently in the repo, the only shell which is under test has tests which are also written in shell. So, when you mention Python as a basis for testing makepkg, I want to simply discard the rest of your proposal because it's clear to me that you haven't given this much thought. Sorry if that's harsh. What you describe isn't unit testing which is what makepkg really needs most, imo. You're describing system tests which include too many variables. Your tests will invariably be brittle if they're too specific, or useless if they're too vague. I need to mention offhand that the output from pactest should be destroyed and the whole test suite rewired to use autotools' to launch the tests. Doing this will let you parallelize the tests. If you want to write tests for makepkg, it needs to go something like this: 1) Break out all functionality that needs testing to a separate file. Everything in this broken out file *must* be in a function. 2) m4_include() the functions file somewhere in the new makepkg "main". 3) Write unit tests for the various functions. This alone is NOT trivial because you'll need to be able to import various and choice bits of environment. There will likely be multiple patches generated as a result of this against makepkg proper before you can even write tests. 4) Now you might be able to write tests. But, it isn't straight forward. How do you deal with external commands? You can't simply run them and expect to behave as you want because they may read from system-specific files which are out of your control. This goes back to the pitfalls I spoke of before... So what's actually possible in this arena? I've written tests for mkinitcpio. They live on a branch in my personal devel repo: http://code.falconindy.com/cgit/mkinitcpio.git/tree/?h=tests These are made possible through a test library and mocking framework that I wrote (apron), which does a fair job of creating mocks and providing simple wrappers for assertions and error accounting. If you've ever used Google's gmock before, you might recognize some of the style. I haven't committed these to master because while the tests do work, I'm not really sure how reliable they are. They depend fairly heavily on mocks and expectations which don't scale well with time. I could explain why, but that's for another rant. So in the broad sense, why doesn't this work well? This is also known as: what did I learn from writing apron and mkinitcpio tests? 1) You have no ability to separate the environment of the test suite itself and the code under test if you do the testing "in-process". But, if you run the code under test in a child process, you're prohibitively limited from monitoring the test itself. Shell functionality like traps are needed for some behavior in apron. This means that you potentially modify behavior of the code under test by blowing away its own traps. Similarly, the code under test might destroy the traps that apron has set in order to get its job done. 2) You have little to no ability to control redirections. This is important as tests are all about watching and validating, preventing, or controlling side effects. Once you start getting into creating wrapper functions for things as simple as redirections in the actual code in order to make it testable, you start impacting readability negatively. 3) You're at the mercy of the bash parser before your test code even runs, because PKGBUILDs *are* code. How do you handle testing various error conditions triggered by the bash parser and distinguish them from some other failure? Obviously, "end-to-end" tests might be a lot easier here, but could still potentially fail for a lot of the reasons already covered here (external dependencies, etc). You can only feed so many PKGBUILDs to makepkg before you run out of things to test at this broad of a stance and you won't have covered very much more ground than what one can manually test. Have fun, Dave
Am 12.07.2013 21:56, schrieb Dave Reisner:
On Fri, Jul 12, 2013 at 08:00:37PM +0100, Ashley Whetter wrote:
... Any suggestions or ideas? Ashley
Incoming wall of text and can of worms. I could have gone on, but I feel strongly about this because I've been down this road before and it's a shitfest.
...
I once did write a test suite for dbscripts which is also entirely written in Bash. And I have to agree with Dave here, the it's probably easier to use Bash for your tests as well. Otherwise you might spend more time on writing a Bash parser and test that instead of makepkg. Here is what I did back then: https://projects.archlinux.org/dbscripts.git/tree/test This is based on shunit2. The actual tests are in test.d; lib/common.inc might also be interesting. It sure has issues, but it was better than nothing and helped a lot in development. I spend a good portion of the time in writing setup and teardown function which set up a clean working environment. This should be as self-contained as possible and not rely on external ressources. This will get interesting if you are testing vcs support, gpg and other funny things. But maybe start with very simple PKGBUILD files and check if they result in valid packages. Greetings, Pierre -- Pierre Schmitz, https://pierre-schmitz.com
On 13/07/13 05:56, Dave Reisner wrote:
1) Break out all functionality that needs testing to a separate file. Everything in this broken out file *must* be in a function. 2) m4_include() the functions file somewhere in the new makepkg "main".
Or... just split parts of makepkg out into a library. See the top two commit here for how I would go about this: https://projects.archlinux.org/users/allan/pacman.git/log/?h=makepkg-split I agree that this is the first thing that needs to be done to simplify makepkg into a state where a testsuite begins making sense. Allan
On 12 July 2013 20:00, Ashley Whetter <awhetter.2011@my.bristol.ac.uk> wrote:
I've recently come across issue FS#15645 ([makepkg] needs a test suite) and I'd like to have a bash at implementing it. It seems like a good idea to get peoples ideas and thoughts first so what do you think?
My first reaction is to do something similar to the pacman tests and just write a custom framework in python. Also have the output the same as the pacman tests.
I think keeping the idea an environment (like in the pacman tests) seems a bit heavy handed because the only configuration done is in makepkg.conf and we can probably fake a pacman environment using a stub pacman. Or maybe we could utilise the pacman test environments!
Unit testing should be easy enough because we can just call specific functions in makepkg and check the output.
For functional testing I'm not really sure.
As for acceptance testing we can just put different flag combinations in different test environments, and check the return code. Check the .PKGINFO. Maybe that files were put where they should be in pkg and src, as well as in the package itself. Have some PKGBUILDs that fail for various reasons like syntax, a dependency package doesn't exist. Make sure missing {,make,check}dependencies are installed.
Any suggestions or ideas? Ashley
Thanks for everyone's suggestions. Sorry if my initial proposal seemed a bit ropey. I've never done bash scripting on this scale before, and I'm not as experienced with testing as I'd like. So I appreciate everyone's patience. I've started taking a look at splitting up makepkg into libraries. I've created a (pretty messy) dependency graph that shows what functions call what, and my initial idea of what functions go in what libraries (http://underrun.org/~gadget/makepkg_resolved_deps.png). Is it worth me creating and emailing a plain text list for the benefit of the mailing list archives? There are 5 main libraries: makepkg: For "makepkg specific" stuff. PKGBUILD: For PKGBUILD specific stuff. sources: For downloading and extracting sources. deps: Handles dependencies and pacman stuff. utils: For general useful functions. On the graph there's also 'misc' and 'Messages'. Misc is just general makepkg code that isn't currently in a function and depends on all other libraries. It also calls any functions that don't currently have incoming dependencies, as well as some others. Messages contains just the 5 functions that print messages and everything depends on it. I'm bit a unsure about what to do with a couple of functions though, and I've marked these as orange on the graph. check_option feels to me like it does both 'makepkg' and 'PKGBUILD' specific stuff, and shouldn't really go in utils. Also run_function depends on it. I think get_full_version is a PKGBUILD specific function, but run_function (in utils) depends on it to create the log file. Maybe logging stuff could be split into another library? Other points: get_downloadclient and get_url could be moved to PKGBUILD. extract_sources to check_build_status is the only dependency between 'sources' and 'makepkg'. I feel like it's worth getting rid of this dependency somehow? I was a bit unclear as to where some functions should go, so if anything seems odd then let me know. Thanks, Ashley
On 28/07/13 06:57, Ashley Whetter wrote:
On 12 July 2013 20:00, Ashley Whetter <awhetter.2011@my.bristol.ac.uk> wrote:
I've recently come across issue FS#15645 ([makepkg] needs a test suite) and I'd like to have a bash at implementing it. It seems like a good idea to get peoples ideas and thoughts first so what do you think?
My first reaction is to do something similar to the pacman tests and just write a custom framework in python. Also have the output the same as the pacman tests.
I think keeping the idea an environment (like in the pacman tests) seems a bit heavy handed because the only configuration done is in makepkg.conf and we can probably fake a pacman environment using a stub pacman. Or maybe we could utilise the pacman test environments!
Unit testing should be easy enough because we can just call specific functions in makepkg and check the output.
For functional testing I'm not really sure.
As for acceptance testing we can just put different flag combinations in different test environments, and check the return code. Check the .PKGINFO. Maybe that files were put where they should be in pkg and src, as well as in the package itself. Have some PKGBUILDs that fail for various reasons like syntax, a dependency package doesn't exist. Make sure missing {,make,check}dependencies are installed.
Any suggestions or ideas? Ashley
Thanks for everyone's suggestions. Sorry if my initial proposal seemed a bit ropey. I've never done bash scripting on this scale before, and I'm not as experienced with testing as I'd like. So I appreciate everyone's patience.
I've started taking a look at splitting up makepkg into libraries. I've created a (pretty messy) dependency graph that shows what functions call what, and my initial idea of what functions go in what libraries (http://underrun.org/~gadget/makepkg_resolved_deps.png). Is it worth me creating and emailing a plain text list for the benefit of the mailing list archives?
There are 5 main libraries: makepkg: For "makepkg specific" stuff. PKGBUILD: For PKGBUILD specific stuff. sources: For downloading and extracting sources. deps: Handles dependencies and pacman stuff. utils: For general useful functions.
On the graph there's also 'misc' and 'Messages'. Misc is just general makepkg code that isn't currently in a function and depends on all other libraries. It also calls any functions that don't currently have incoming dependencies, as well as some others. Messages contains just the 5 functions that print messages and everything depends on it.
I'm bit a unsure about what to do with a couple of functions though, and I've marked these as orange on the graph. check_option feels to me like it does both 'makepkg' and 'PKGBUILD' specific stuff, and shouldn't really go in utils. Also run_function depends on it. I think get_full_version is a PKGBUILD specific function, but run_function (in utils) depends on it to create the log file. Maybe logging stuff could be split into another library?
Other points: get_downloadclient and get_url could be moved to PKGBUILD. extract_sources to check_build_status is the only dependency between 'sources' and 'makepkg'. I feel like it's worth getting rid of this dependency somehow?
I was a bit unclear as to where some functions should go, so if anything seems odd then let me know.
The things I think make the most sense to separate out the source downloading/extracting functions first because they are a clear grouping. The rest can be considered later once we get the initial steps completed. Allan
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk> In preparation for creating a makepkg test suite I've started splitting makepkg up into libmakepkg. Currently I've only split out the downloading and extracting functions. Everything else has gone into utils. I decided to keep get_url and get_downloadclient out of the downloads library because they depend on the format of PKGBUILDs. So the new dependency graph looks like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps_v2.png Whereas the old one looked like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps.png Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable Ashley Whetter (3): Moved makepkg functions into a library Moved makepkg download functions into libmakepkg Moved makepkg extraction functions into libmakepkg scripts/.gitignore | 4 + scripts/Makefile.am | 54 +- scripts/libmakepkg/downloads.sh.in | 287 ++++ scripts/libmakepkg/extractions.sh.in | 293 ++++ scripts/libmakepkg/utils.sh.in | 1946 +++++++++++++++++++++++++++ scripts/makepkg-wrapper.sh.in | 23 + scripts/makepkg.sh.in | 2458 +--------------------------------- 7 files changed, 2611 insertions(+), 2454 deletions(-) create mode 100644 scripts/libmakepkg/downloads.sh.in create mode 100644 scripts/libmakepkg/extractions.sh.in create mode 100644 scripts/libmakepkg/utils.sh.in create mode 100644 scripts/makepkg-wrapper.sh.in -- 1.8.4
From: Allan McRae <allan@archlinux.org> Build makepkg to scripts/.lib/makepkg and add a wrapper script to call it. This is not useful at the moment, but is the first step to allowing makepkg to be split into smaller pieces. Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit 015667ab2c8f0be3b5ff34cc93757e7873a09dbd) Signed-off-by: Ashley Whetter <awhetter.2011@my.bristol.ac.uk> Conflicts: scripts/.gitignore scripts/Makefile.am --- scripts/.gitignore | 1 + scripts/Makefile.am | 25 +++++++++++++++++++++++++ scripts/makepkg-wrapper.sh.in | 23 +++++++++++++++++++++++ 3 files changed, 49 insertions(+) create mode 100644 scripts/makepkg-wrapper.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index 26e088b..8dac503 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -1,5 +1,6 @@ makepkg makepkg-template +makepkg-wrapper pacman-db-upgrade pacman-key pacman-optimize diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 1f3bae2..f45065d 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -6,6 +6,7 @@ SUBDIRS = po bin_SCRIPTS = \ $(OURSCRIPTS) \ makepkg-template \ + makepkg-wrapper \ repo-remove \ repo-elephant @@ -20,6 +21,7 @@ OURSCRIPTS = \ EXTRA_DIST = \ makepkg.sh.in \ makepkg-template.pl.in \ + makepkg-wrapper.sh.in \ pacman-db-upgrade.sh.in \ pacman-key.sh.in \ pacman-optimize.sh.in \ @@ -37,6 +39,9 @@ LIBRARY = \ # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) +clean-local: + $(AM_V_at)$(RM) -r .lib + if USE_GIT_VERSION GIT_VERSION := $(shell sh -c 'git describe --abbrev=4 --dirty | sed s/^v//') REAL_PACKAGE_VERSION = $(GIT_VERSION) @@ -77,6 +82,7 @@ $(OURSCRIPTS): Makefile makepkg: \ $(srcdir)/makepkg.sh.in \ + $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/library/parseopts.sh makepkg-template: \ @@ -117,8 +123,27 @@ repo-elephant: $(srcdir)/repo-add.sh.in $(AM_V_at)$(RM) repo-elephant $(AM_V_at)$(LN_S) repo-add repo-elephant +makepkg-wrapper: \ + Makefile \ + $(srcdir)/makepkg-wrapper.sh.in \ + $(srcdir)/makepkg.sh.in \ + $(srcdir)/library/parseopts.sh \ + | makepkg + $(AM_V_at)$(MKDIR_P) .lib + $(AM_V_at)mv -f makepkg .lib + $(AM_V_at)$(RM) $@ + $(AM_V_GEN)sed \ + -e "s|@PWD[@]|$$(pwd)|" \ + -e '1s|!/bin/bash|!$(BASH_SHELL)|g' \ + $(srcdir)/$@.sh.in > $@ + $(AM_V_at)chmod +x,a-w $@ + $(AM_V_at)$(LN_S) makepkg-wrapper makepkg + install-data-hook: cd $(DESTDIR)$(bindir) && \ + $(RM) makepkg makepkg-wrapper + $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg + cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ ( $(LN_S) repo-add repo-elephant || \ ln repo-add repo-elephant || \ diff --git a/scripts/makepkg-wrapper.sh.in b/scripts/makepkg-wrapper.sh.in new file mode 100644 index 0000000..d703ee4 --- /dev/null +++ b/scripts/makepkg-wrapper.sh.in @@ -0,0 +1,23 @@ +#!/bin/bash +# +# makepkg - a wrapper for running the real makepkg in the source tree +# +# Copyright (c) 2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +DIR="@PWD@" + +LIBRARY="$DIR"/libmakepkg exec "$DIR"/.lib/makepkg "$@" -- 1.8.4
From: Allan McRae <allan@archlinux.org> This points makepkg to where is library is located. Can be overridden by value in the environment. Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit ab65d89fb622252eafd6f31b17d7bbcffa89dda7) Signed-off-by: Ashley Whetter <awhetter.2011@my.bristol.ac.uk> Conflicts: scripts/makepkg.sh.in --- scripts/Makefile.am | 3 +++ scripts/makepkg.sh.in | 2 ++ 2 files changed, 5 insertions(+) diff --git a/scripts/Makefile.am b/scripts/Makefile.am index f45065d..8130704 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -39,6 +39,8 @@ LIBRARY = \ # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) +libmakepkgdir = $(libdir)/makepkg + clean-local: $(AM_V_at)$(RM) -r .lib @@ -54,6 +56,7 @@ edit = sed \ -e 's|@localedir[@]|$(localedir)|g' \ -e 's|@sysconfdir[@]|$(sysconfdir)|g' \ -e 's|@localstatedir[@]|$(localstatedir)|g' \ + -e 's|@libmakepkgdir[@]|$(libmakepkgdir)|g' \ -e 's|@pkgdatadir[@]|$(pkgdatadir)|g' \ -e 's|@prefix[@]|$(prefix)|g' \ -e '1s|!/bin/bash|!$(BASH_SHELL)|g' \ diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 025f756..98cc2aa 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -46,6 +46,8 @@ declare -r confdir='@sysconfdir@' declare -r BUILDSCRIPT='@BUILDSCRIPT@' declare -r startdir="$PWD" +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + packaging_options=('strip' 'docs' 'libtool' 'staticlibs' 'emptydirs' 'zipman' \ 'purge' 'upx' 'debug') other_options=('ccache' 'distcc' 'buildflags' 'makeflags') -- 1.8.4
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk> Signed-off-by: Ashley Whetter <awhetter.2011@my.bristol.ac.uk> --- scripts/.gitignore | 1 + scripts/Makefile.am | 4 + scripts/libmakepkg/downloads.sh.in | 287 +++++++++++++++++++++++++++++++++++++ scripts/libmakepkg/utils.sh.in | 261 --------------------------------- scripts/makepkg.sh.in | 1 + 5 files changed, 293 insertions(+), 261 deletions(-) create mode 100644 scripts/libmakepkg/downloads.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index ae17035..b15c7ac 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -8,4 +8,5 @@ pkgdelta repo-add repo-elephant repo-remove +libmakepkg/downloads.sh libmakepkg/utils.sh diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 6080508..979de54 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -37,6 +37,7 @@ LIBRARY = \ library/term_colors.sh LIBMAKEPKG = \ + libmakepkg/downloads.sh \ libmakepkg/utils.sh # Files that should be removed, but which Automake does not know. @@ -92,6 +93,7 @@ $(LIBMAKEPKG): Makefile @$(BASH_SHELL) -O extglob -n $@ libmakepkg: \ + $(srcdir)/libmakepkg/downloads.sh \ $(srcdir)/libmakepkg/utils.sh \ $(srcdir)/library/parseopts.sh @@ -142,6 +144,7 @@ makepkg-wrapper: \ Makefile \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/makepkg.sh.in \ + $(srcdir)/libmakepkg/downloads.sh \ $(srcdir)/libmakepkg/utils.sh \ $(srcdir)/library/parseopts.sh \ | makepkg @@ -160,6 +163,7 @@ install-data-hook: $(RM) makepkg makepkg-wrapper $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir) + $(INSTALL) libmakepkg/downloads.sh $(DESTDIR)$(libmakepkgdir)/downloads.sh $(INSTALL) libmakepkg/utils.sh $(DESTDIR)$(libmakepkgdir)/utils.sh cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ diff --git a/scripts/libmakepkg/downloads.sh.in b/scripts/libmakepkg/downloads.sh.in new file mode 100644 index 0000000..e05d7da --- /dev/null +++ b/scripts/libmakepkg/downloads.sh.in @@ -0,0 +1,287 @@ +# +# downloads.sh +# +# Copyright (c) 2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOADS_SH" ] && return || LIBMAKEPKG_DOWNLOADS_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source $LIBRARY/utils.sh + +download_local() { + local netfile=$1 + local filepath=$(get_filepath "$netfile") + + if [[ -n "$filepath" ]]; then + msg2 "$(gettext "Found %s")" "${filepath##*/}" + rm -f "$srcdir/${filepath##*/}" + ln -s "$filepath" "$srcdir/" + else + local filename=$(get_filename "$netfile") + error "$(gettext "%s was not found in the build directory and is not a URL.")" "$filename" + exit 1 # $E_MISSING_FILE + fi +} + +download_file() { + local netfile=$1 + + local filepath=$(get_filepath "$netfile") + if [[ -n "$filepath" ]]; then + msg2 "$(gettext "Found %s")" "${filepath##*/}" + rm -f "$srcdir/${filepath##*/}" + ln -s "$filepath" "$srcdir/" + return + fi + + local proto=$(get_protocol "$netfile") + + # find the client we should use for this URL + local dlcmd + dlcmd=$(get_downloadclient "$proto") || exit $? + + local filename=$(get_filename "$netfile") + local url=$(get_url "$netfile") + + if [[ $proto = "scp" ]]; then + # scp downloads should not pass the protocol in the url + url="${url##*://}" + fi + + msg2 "$(gettext "Downloading %s...")" "$filename" + + # temporary download file, default to last component of the URL + local dlfile="${url##*/}" + + # replace %o by the temporary dlfile if it exists + if [[ $dlcmd = *%o* ]]; then + dlcmd=${dlcmd//\%o/\"$filename.part\"} + dlfile="$filename.part" + fi + # add the URL, either in place of %u or at the end + if [[ $dlcmd = *%u* ]]; then + dlcmd=${dlcmd//\%u/\"$url\"} + else + dlcmd="$dlcmd \"$url\"" + fi + + local ret=0 + eval "$dlcmd || ret=\$?" + if (( ret )); then + [[ ! -s $dlfile ]] && rm -f -- "$dlfile" + error "$(gettext "Failure while downloading %s")" "$filename" + plain "$(gettext "Aborting...")" + exit 1 + fi + + # rename the temporary download file to the final destination + if [[ $dlfile != "$filename" ]]; then + mv -f "$SRCDEST/$dlfile" "$SRCDEST/$filename" + fi + + rm -f "$srcdir/$filename" + ln -s "$SRCDEST/$filename" "$srcdir/" +} + +download_bzr() { + local netfile=$1 + + local url=$(get_url "$netfile") + url=${url##*bzr+} + url=${url%%#*} + + local repo=$(get_filename "$netfile") + local displaylocation="$url" + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Branching %s ...")" "${displaylocation}" + if ! bzr branch "$url" "$dir" --no-tree --use-existing-dir; then + error "$(gettext "Failure while branching %s")" "${displaylocation}" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + # Make sure we are fetching the right repo + local distant_url="$(bzr info $url 2> /dev/null | sed -n '/branch root/{s/ branch root: //p;q;}')" + local local_url="$(bzr config parent_location -d $dir)" + if [[ -n $distant_url ]]; then + if [[ $distant_url != "$local_url" ]]; then + error "$(gettext "%s is not a branch of %s")" "$dir" "$url" + plain "$(gettext "Aborting...")" + exit 1 + fi + else + if [[ $url != "$local_url" ]] ; then + error "$(gettext "%s is not a branch of %s")" "$dir" "$url" + error "$(gettext "The local URL is %s")" "$local_url" + plain "$(gettext "Aborting...")" + exit 1 + fi + fi + msg2 "$(gettext "Pulling %s ...")" "${displaylocation}" + cd_safe "$dir" + if ! bzr pull "$url" --overwrite; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while pulling %s")" "${displaylocation}" + fi + fi +} + +download_git() { + local netfile=$1 + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + url=${url##*git+} + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git" + if ! git clone --mirror "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + cd_safe "$dir" + # Make sure we are fetching the right repo + if [[ "$url" != "$(git config --get remote.origin.url)" ]] ; then + error "$(gettext "%s is not a clone of %s")" "$dir" "$url" + plain "$(gettext "Aborting...")" + exit 1 + fi + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "git" + if ! git fetch --all -p; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "git" + fi + fi +} + +download_hg() { + local netfile=$1 + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + url=${url##*hg+} + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "hg" + if ! hg clone -U "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "hg" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "hg" + cd_safe "$dir" + if ! hg pull; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "hg" + fi + fi +} + +download_svn() { + local netfile=$1 + + local fragment=${netfile#*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + if [[ $url != svn+ssh* ]]; then + url=${url##*svn+} + fi + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "svn" + mkdir -p "$dir/.makepkg" + if ! svn checkout --config-dir "$dir/.makepkg" "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "svn" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "svn" + cd_safe "$dir" + if ! svn update; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "svn" + fi + fi +} + +download_sources() { + msg "$(gettext "Retrieving sources...")" + + local GET_VCS=1 + if [[ $1 == "fast" ]]; then + GET_VCS=0 + fi + + local netfile + for netfile in "${source[@]}"; do + pushd "$SRCDEST" &>/dev/null + + local proto=$(get_protocol "$netfile") + case "$proto" in + local) + download_local "$netfile" + ;; + bzr*) + (( GET_VCS )) && download_bzr "$netfile" + ;; + git*) + (( GET_VCS )) && download_git "$netfile" + ;; + hg*) + (( GET_VCS )) && download_hg "$netfile" + ;; + svn*) + (( GET_VCS )) && download_svn "$netfile" + ;; + *) + download_file "$netfile" + ;; + esac + + popd &>/dev/null + done +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/utils.sh.in b/scripts/libmakepkg/utils.sh.in index f7df309..4eba0e6 100644 --- a/scripts/libmakepkg/utils.sh.in +++ b/scripts/libmakepkg/utils.sh.in @@ -236,81 +236,6 @@ get_downloadclient() { printf "%s\n" "$agent" } -download_local() { - local netfile=$1 - local filepath=$(get_filepath "$netfile") - - if [[ -n "$filepath" ]]; then - msg2 "$(gettext "Found %s")" "${filepath##*/}" - rm -f "$srcdir/${filepath##*/}" - ln -s "$filepath" "$srcdir/" - else - local filename=$(get_filename "$netfile") - error "$(gettext "%s was not found in the build directory and is not a URL.")" "$filename" - exit 1 # $E_MISSING_FILE - fi -} - -download_file() { - local netfile=$1 - - local filepath=$(get_filepath "$netfile") - if [[ -n "$filepath" ]]; then - msg2 "$(gettext "Found %s")" "${filepath##*/}" - rm -f "$srcdir/${filepath##*/}" - ln -s "$filepath" "$srcdir/" - return - fi - - local proto=$(get_protocol "$netfile") - - # find the client we should use for this URL - local dlcmd - dlcmd=$(get_downloadclient "$proto") || exit $? - - local filename=$(get_filename "$netfile") - local url=$(get_url "$netfile") - - if [[ $proto = "scp" ]]; then - # scp downloads should not pass the protocol in the url - url="${url##*://}" - fi - - msg2 "$(gettext "Downloading %s...")" "$filename" - - # temporary download file, default to last component of the URL - local dlfile="${url##*/}" - - # replace %o by the temporary dlfile if it exists - if [[ $dlcmd = *%o* ]]; then - dlcmd=${dlcmd//\%o/\"$filename.part\"} - dlfile="$filename.part" - fi - # add the URL, either in place of %u or at the end - if [[ $dlcmd = *%u* ]]; then - dlcmd=${dlcmd//\%u/\"$url\"} - else - dlcmd="$dlcmd \"$url\"" - fi - - local ret=0 - eval "$dlcmd || ret=\$?" - if (( ret )); then - [[ ! -s $dlfile ]] && rm -f -- "$dlfile" - error "$(gettext "Failure while downloading %s")" "$filename" - plain "$(gettext "Aborting...")" - exit 1 - fi - - # rename the temporary download file to the final destination - if [[ $dlfile != "$filename" ]]; then - mv -f "$SRCDEST/$dlfile" "$SRCDEST/$filename" - fi - - rm -f "$srcdir/$filename" - ln -s "$SRCDEST/$filename" "$srcdir/" -} - extract_file() { local file=$1 # do not rely on extension for file type @@ -364,53 +289,6 @@ extract_file() { fi } -download_bzr() { - local netfile=$1 - - local url=$(get_url "$netfile") - url=${url##*bzr+} - url=${url%%#*} - - local repo=$(get_filename "$netfile") - local displaylocation="$url" - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Branching %s ...")" "${displaylocation}" - if ! bzr branch "$url" "$dir" --no-tree --use-existing-dir; then - error "$(gettext "Failure while branching %s")" "${displaylocation}" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - # Make sure we are fetching the right repo - local distant_url="$(bzr info $url 2> /dev/null | sed -n '/branch root/{s/ branch root: //p;q;}')" - local local_url="$(bzr config parent_location -d $dir)" - if [[ -n $distant_url ]]; then - if [[ $distant_url != "$local_url" ]]; then - error "$(gettext "%s is not a branch of %s")" "$dir" "$url" - plain "$(gettext "Aborting...")" - exit 1 - fi - else - if [[ $url != "$local_url" ]] ; then - error "$(gettext "%s is not a branch of %s")" "$dir" "$url" - error "$(gettext "The local URL is %s")" "$local_url" - plain "$(gettext "Aborting...")" - exit 1 - fi - fi - msg2 "$(gettext "Pulling %s ...")" "${displaylocation}" - cd_safe "$dir" - if ! bzr pull "$url" --overwrite; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while pulling %s")" "${displaylocation}" - fi - fi -} - extract_bzr() { local netfile=$1 @@ -450,41 +328,6 @@ extract_bzr() { popd &>/dev/null } -download_git() { - local netfile=$1 - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - url=${url##*git+} - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git" - if ! git clone --mirror "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - cd_safe "$dir" - # Make sure we are fetching the right repo - if [[ "$url" != "$(git config --get remote.origin.url)" ]] ; then - error "$(gettext "%s is not a clone of %s")" "$dir" "$url" - plain "$(gettext "Aborting...")" - exit 1 - fi - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "git" - if ! git fetch --all -p; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "git" - fi - fi -} - extract_git() { local netfile=$1 @@ -539,35 +382,6 @@ extract_git() { popd &>/dev/null } -download_hg() { - local netfile=$1 - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - url=${url##*hg+} - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "hg" - if ! hg clone -U "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "hg" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "hg" - cd_safe "$dir" - if ! hg pull; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "hg" - fi - fi -} - extract_hg() { local netfile=$1 @@ -608,43 +422,6 @@ extract_hg() { popd &>/dev/null } -download_svn() { - local netfile=$1 - - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - if [[ $url != svn+ssh* ]]; then - url=${url##*svn+} - fi - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "svn" - mkdir -p "$dir/.makepkg" - if ! svn checkout --config-dir "$dir/.makepkg" "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "svn" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "svn" - cd_safe "$dir" - if ! svn update; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "svn" - fi - fi -} - extract_svn() { local netfile=$1 @@ -689,44 +466,6 @@ extract_svn() { popd &>/dev/null } -download_sources() { - msg "$(gettext "Retrieving sources...")" - - local GET_VCS=1 - if [[ $1 == "fast" ]]; then - GET_VCS=0 - fi - - local netfile - for netfile in "${source[@]}"; do - pushd "$SRCDEST" &>/dev/null - - local proto=$(get_protocol "$netfile") - case "$proto" in - local) - download_local "$netfile" - ;; - bzr*) - (( GET_VCS )) && download_bzr "$netfile" - ;; - git*) - (( GET_VCS )) && download_git "$netfile" - ;; - hg*) - (( GET_VCS )) && download_hg "$netfile" - ;; - svn*) - (( GET_VCS )) && download_svn "$netfile" - ;; - *) - download_file "$netfile" - ;; - esac - - popd &>/dev/null - done -} - # Automatically update pkgver variable if a pkgver() function is provided # Re-sources the PKGBUILD afterwards to allow for other variables that use $pkgver update_pkgver() { diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 8d982ba..0c22774 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -95,6 +95,7 @@ PACMAN_OPTS= shopt -s extglob +source $LIBRARY/downloads.sh source $LIBRARY/utils.sh # PROGRAM START -- 1.8.4
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk> Signed-off-by: Ashley Whetter <awhetter.2011@my.bristol.ac.uk> --- scripts/.gitignore | 1 + scripts/Makefile.am | 4 + scripts/libmakepkg/extractions.sh.in | 293 +++++++++++++++++++++++++++++++++++ scripts/libmakepkg/utils.sh.in | 267 ------------------------------- scripts/makepkg.sh.in | 1 + 5 files changed, 299 insertions(+), 267 deletions(-) create mode 100644 scripts/libmakepkg/extractions.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index b15c7ac..d7c461e 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -9,4 +9,5 @@ repo-add repo-elephant repo-remove libmakepkg/downloads.sh +libmakepkg/extractions.sh libmakepkg/utils.sh diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 979de54..5278580 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -38,6 +38,7 @@ LIBRARY = \ LIBMAKEPKG = \ libmakepkg/downloads.sh \ + libmakepkg/extractions.sh \ libmakepkg/utils.sh # Files that should be removed, but which Automake does not know. @@ -94,6 +95,7 @@ $(LIBMAKEPKG): Makefile libmakepkg: \ $(srcdir)/libmakepkg/downloads.sh \ + $(srcdir)/libmakepkg/extractions.sh \ $(srcdir)/libmakepkg/utils.sh \ $(srcdir)/library/parseopts.sh @@ -145,6 +147,7 @@ makepkg-wrapper: \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/makepkg.sh.in \ $(srcdir)/libmakepkg/downloads.sh \ + $(srcdir)/libmakepkg/extractions.sh \ $(srcdir)/libmakepkg/utils.sh \ $(srcdir)/library/parseopts.sh \ | makepkg @@ -164,6 +167,7 @@ install-data-hook: $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir) $(INSTALL) libmakepkg/downloads.sh $(DESTDIR)$(libmakepkgdir)/downloads.sh + $(INSTALL) libmakepkg/extractions.sh $(DESTDIR)$(libmakepkgdir)/extractions.sh $(INSTALL) libmakepkg/utils.sh $(DESTDIR)$(libmakepkgdir)/utils.sh cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ diff --git a/scripts/libmakepkg/extractions.sh.in b/scripts/libmakepkg/extractions.sh.in new file mode 100644 index 0000000..0ea7331 --- /dev/null +++ b/scripts/libmakepkg/extractions.sh.in @@ -0,0 +1,293 @@ +# +# extractions.sh +# +# Copyright (c) 2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_EXTRACTIONS_SH" ] && return || LIBMAKEPKG_EXTRACTIONS_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source $LIBRARY/utils.sh + +extract_file() { + local file=$1 + # do not rely on extension for file type + local file_type=$(file -bizL "$file") + local ext=${file##*.} + local cmd='' + case "$file_type" in + *application/x-tar*|*application/zip*|*application/x-zip*|*application/x-cpio*) + cmd="bsdtar" ;; + *application/x-gzip*) + case "$ext" in + gz|z|Z) cmd="gzip" ;; + *) continue;; + esac ;; + *application/x-bzip*) + case "$ext" in + bz2|bz) cmd="bzip2" ;; + *) continue;; + esac ;; + *application/x-xz*) + case "$ext" in + xz) cmd="xz" ;; + *) continue;; + esac ;; + *) + # See if bsdtar can recognize the file + if bsdtar -tf "$file" -q '*' &>/dev/null; then + cmd="bsdtar" + else + continue + fi ;; + esac + + local ret=0 + msg2 "$(gettext "Extracting %s with %s")" "$file" "$cmd" + if [[ $cmd = "bsdtar" ]]; then + $cmd -xf "$file" || ret=$? + else + rm -f -- "${file%.*}" + $cmd -dcf "$file" > "${file%.*}" || ret=$? + fi + if (( ret )); then + error "$(gettext "Failed to extract %s")" "$file" + plain "$(gettext "Aborting...")" + exit 1 + fi + + if (( EUID == 0 )); then + # change perms of all source files to root user & root group + chown -R 0:0 "$srcdir" + fi +} + +extract_bzr() { + local netfile=$1 + + local repo=$(get_filename "$netfile") + local fragment=${netfile#*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + if [[ -n $fragment ]]; then + case ${fragment%%=*} in + revision) + revision=("-r" "${fragment#*=}") + displaylocation="$url -r ${fragment#*=}" + ;; + *) + error "$(gettext "Unrecognized reference: %s")" "${fragment}" + plain "$(gettext "Aborting...")" + exit 1 + esac + fi + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "bzr" + pushd "$srcdir" &>/dev/null + rm -rf "${dir##*/}" + + if ! { bzr checkout "$dir" "${revision[@]}" --lightweight && + ( cd "$repo" && bzr pull "$dir" -q --overwrite "${revision[@]}" ); }; then + error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "bzr" + plain "$(gettext "Aborting...")" + exit 1 + fi + + popd &>/dev/null +} + +extract_git() { + local netfile=$1 + + local fragment=${netfile#*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + local repo=${netfile##*/} + repo=${repo%%#*} + repo=${repo%%.git*} + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "git" + pushd "$srcdir" &>/dev/null + rm -rf "${dir##*/}" + + if ! git clone "$dir"; then + error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" + plain "$(gettext "Aborting...")" + exit 1 + fi + + cd_safe "${dir##*/}" + + local ref + if [[ -n $fragment ]]; then + case ${fragment%%=*} in + commit|tag) + ref=${fragment##*=} + ;; + branch) + ref=origin/${fragment##*=} + ;; + *) + error "$(gettext "Unrecognized reference: %s")" "${fragment}" + plain "$(gettext "Aborting...")" + exit 1 + esac + fi + + if [[ -n $ref ]]; then + if ! git checkout -b makepkg $ref; then + error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" + plain "$(gettext "Aborting...")" + exit 1 + fi + fi + + popd &>/dev/null +} + +extract_hg() { + local netfile=$1 + + local fragment=${netfile#*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=${netfile##*/} + repo=${repo%%#*} + + msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "hg" + pushd "$srcdir" &>/dev/null + rm -rf "${dir##*/}" + + local ref + if [[ -n $fragment ]]; then + case ${fragment%%=*} in + branch|revision|tag) + ref=('-u' "${fragment##*=}") + ;; + *) + error "$(gettext "Unrecognized reference: %s")" "${fragment}" + plain "$(gettext "Aborting...")" + exit 1 + esac + fi + + if ! hg clone "${ref[@]}" "$dir" "${dir##*/}"; then + error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "hg" + plain "$(gettext "Aborting...")" + exit 1 + fi + + popd &>/dev/null +} + +extract_svn() { + local netfile=$1 + + local fragment=${netfile#*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=${netfile##*/} + repo=${repo%%#*} + + msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "svn" + pushd "$srcdir" &>/dev/null + rm -rf "${dir##*/}" + + local ref + if [[ -n $fragment ]]; then + case ${fragment%%=*} in + revision) + ref="${fragment##*=}" + ;; + *) + error "$(gettext "Unrecognized reference: %s")" "${fragment}" + plain "$(gettext "Aborting...")" + exit 1 + esac + fi + + cp -a "$dir" . + + if [[ -n ${ref} ]]; then + cd_safe "$(get_filename "$netfile")" + if ! svn update -r ${ref}; then + error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "svn" + plain "$(gettext "Aborting...")" + fi + fi + + popd &>/dev/null +} + +extract_sources() { + msg "$(gettext "Extracting sources...")" + local netfile + for netfile in "${source[@]}"; do + local file=$(get_filename "$netfile") + if in_array "$file" "${noextract[@]}"; then + #skip source files in the noextract=() array + # these are marked explicitly to NOT be extracted + continue + fi + local proto=$(get_protocol "$netfile") + case "$proto" in + bzr*) + extract_bzr "$netfile" + ;; + git*) + extract_git "$netfile" + ;; + hg*) + extract_hg "$netfile" + ;; + svn*) + extract_svn "$netfile" + ;; + *) + extract_file "$file" + ;; + esac + done + + if (( PKGVERFUNC )); then + update_pkgver + check_pkgver || exit 1 + check_build_status + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/utils.sh.in b/scripts/libmakepkg/utils.sh.in index 4eba0e6..39aa7d9 100644 --- a/scripts/libmakepkg/utils.sh.in +++ b/scripts/libmakepkg/utils.sh.in @@ -236,236 +236,6 @@ get_downloadclient() { printf "%s\n" "$agent" } -extract_file() { - local file=$1 - # do not rely on extension for file type - local file_type=$(file -bizL "$file") - local ext=${file##*.} - local cmd='' - case "$file_type" in - *application/x-tar*|*application/zip*|*application/x-zip*|*application/x-cpio*) - cmd="bsdtar" ;; - *application/x-gzip*) - case "$ext" in - gz|z|Z) cmd="gzip" ;; - *) continue;; - esac ;; - *application/x-bzip*) - case "$ext" in - bz2|bz) cmd="bzip2" ;; - *) continue;; - esac ;; - *application/x-xz*) - case "$ext" in - xz) cmd="xz" ;; - *) continue;; - esac ;; - *) - # See if bsdtar can recognize the file - if bsdtar -tf "$file" -q '*' &>/dev/null; then - cmd="bsdtar" - else - continue - fi ;; - esac - - local ret=0 - msg2 "$(gettext "Extracting %s with %s")" "$file" "$cmd" - if [[ $cmd = "bsdtar" ]]; then - $cmd -xf "$file" || ret=$? - else - rm -f -- "${file%.*}" - $cmd -dcf "$file" > "${file%.*}" || ret=$? - fi - if (( ret )); then - error "$(gettext "Failed to extract %s")" "$file" - plain "$(gettext "Aborting...")" - exit 1 - fi - - if (( EUID == 0 )); then - # change perms of all source files to root user & root group - chown -R 0:0 "$srcdir" - fi -} - -extract_bzr() { - local netfile=$1 - - local repo=$(get_filename "$netfile") - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - - if [[ -n $fragment ]]; then - case ${fragment%%=*} in - revision) - revision=("-r" "${fragment#*=}") - displaylocation="$url -r ${fragment#*=}" - ;; - *) - error "$(gettext "Unrecognized reference: %s")" "${fragment}" - plain "$(gettext "Aborting...")" - exit 1 - esac - fi - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "bzr" - pushd "$srcdir" &>/dev/null - rm -rf "${dir##*/}" - - if ! { bzr checkout "$dir" "${revision[@]}" --lightweight && - ( cd "$repo" && bzr pull "$dir" -q --overwrite "${revision[@]}" ); }; then - error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "bzr" - plain "$(gettext "Aborting...")" - exit 1 - fi - - popd &>/dev/null -} - -extract_git() { - local netfile=$1 - - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - - local repo=${netfile##*/} - repo=${repo%%#*} - repo=${repo%%.git*} - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "git" - pushd "$srcdir" &>/dev/null - rm -rf "${dir##*/}" - - if ! git clone "$dir"; then - error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" - plain "$(gettext "Aborting...")" - exit 1 - fi - - cd_safe "${dir##*/}" - - local ref - if [[ -n $fragment ]]; then - case ${fragment%%=*} in - commit|tag) - ref=${fragment##*=} - ;; - branch) - ref=origin/${fragment##*=} - ;; - *) - error "$(gettext "Unrecognized reference: %s")" "${fragment}" - plain "$(gettext "Aborting...")" - exit 1 - esac - fi - - if [[ -n $ref ]]; then - if ! git checkout -b makepkg $ref; then - error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" - plain "$(gettext "Aborting...")" - exit 1 - fi - fi - - popd &>/dev/null -} - -extract_hg() { - local netfile=$1 - - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=${netfile##*/} - repo=${repo%%#*} - - msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "hg" - pushd "$srcdir" &>/dev/null - rm -rf "${dir##*/}" - - local ref - if [[ -n $fragment ]]; then - case ${fragment%%=*} in - branch|revision|tag) - ref=('-u' "${fragment##*=}") - ;; - *) - error "$(gettext "Unrecognized reference: %s")" "${fragment}" - plain "$(gettext "Aborting...")" - exit 1 - esac - fi - - if ! hg clone "${ref[@]}" "$dir" "${dir##*/}"; then - error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "hg" - plain "$(gettext "Aborting...")" - exit 1 - fi - - popd &>/dev/null -} - -extract_svn() { - local netfile=$1 - - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=${netfile##*/} - repo=${repo%%#*} - - msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "svn" - pushd "$srcdir" &>/dev/null - rm -rf "${dir##*/}" - - local ref - if [[ -n $fragment ]]; then - case ${fragment%%=*} in - revision) - ref="${fragment##*=}" - ;; - *) - error "$(gettext "Unrecognized reference: %s")" "${fragment}" - plain "$(gettext "Aborting...")" - exit 1 - esac - fi - - cp -a "$dir" . - - if [[ -n ${ref} ]]; then - cd_safe "$(get_filename "$netfile")" - if ! svn update -r ${ref}; then - error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "svn" - plain "$(gettext "Aborting...")" - fi - fi - - popd &>/dev/null -} - # Automatically update pkgver variable if a pkgver() function is provided # Re-sources the PKGBUILD afterwards to allow for other variables that use $pkgver update_pkgver() { @@ -997,43 +767,6 @@ check_source_integrity() { fi } -extract_sources() { - msg "$(gettext "Extracting sources...")" - local netfile - for netfile in "${source[@]}"; do - local file=$(get_filename "$netfile") - if in_array "$file" "${noextract[@]}"; then - #skip source files in the noextract=() array - # these are marked explicitly to NOT be extracted - continue - fi - local proto=$(get_protocol "$netfile") - case "$proto" in - bzr*) - extract_bzr "$netfile" - ;; - git*) - extract_git "$netfile" - ;; - hg*) - extract_hg "$netfile" - ;; - svn*) - extract_svn "$netfile" - ;; - *) - extract_file "$file" - ;; - esac - done - - if (( PKGVERFUNC )); then - update_pkgver - check_pkgver || exit 1 - check_build_status - fi -} - error_function() { if [[ -p $logpipe ]]; then rm "$logpipe" diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 0c22774..eab755b 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -96,6 +96,7 @@ PACMAN_OPTS= shopt -s extglob source $LIBRARY/downloads.sh +source $LIBRARY/extractions.sh source $LIBRARY/utils.sh # PROGRAM START -- 1.8.4
On 25/08/13 22:14, awhetter.2011@my.bristol.ac.uk wrote:
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk>
In preparation for creating a makepkg test suite I've started splitting makepkg up into libmakepkg. Currently I've only split out the downloading and extracting functions. Everything else has gone into utils.
I decided to keep get_url and get_downloadclient out of the downloads library because they depend on the format of PKGBUILDs. So the new dependency graph looks like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps_v2.png Whereas the old one looked like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps.png
Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable
Ashley Whetter (3): Moved makepkg functions into a library Moved makepkg download functions into libmakepkg Moved makepkg extraction functions into libmakepkg
Given the third patch is too big to get to the mailing list (and figuring out how to approve things is too much effort for me), can you push your git repo somwhere we can have a look at things? Allan
On 2013-08-31 07:26, Allan McRae wrote:
On 25/08/13 22:14, awhetter.2011@my.bristol.ac.uk wrote:
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk>
In preparation for creating a makepkg test suite I've started splitting makepkg up into libmakepkg. Currently I've only split out the downloading and extracting functions. Everything else has gone into utils.
I decided to keep get_url and get_downloadclient out of the downloads library because they depend on the format of PKGBUILDs. So the new dependency graph looks like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps_v2.png Whereas the old one looked like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps.png
Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable
Ashley Whetter (3): Moved makepkg functions into a library Moved makepkg download functions into libmakepkg Moved makepkg extraction functions into libmakepkg
Given the third patch is too big to get to the mailing list (and figuring out how to approve things is too much effort for me), can you push your git repo somwhere we can have a look at things?
Allan
Sure. You can see the changes here: https://github.com/AWhetter/pacman/compare/makepkg-tests Ashley
On 02/09/13 01:40, Ashley Whetter wrote:
On 2013-08-31 07:26, Allan McRae wrote:
On 25/08/13 22:14, awhetter.2011@my.bristol.ac.uk wrote:
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk>
In preparation for creating a makepkg test suite I've started splitting makepkg up into libmakepkg. Currently I've only split out the downloading and extracting functions. Everything else has gone into utils.
I decided to keep get_url and get_downloadclient out of the downloads library because they depend on the format of PKGBUILDs. So the new dependency graph looks like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps_v2.png Whereas the old one looked like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps.png
Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable
Ashley Whetter (3): Moved makepkg functions into a library Moved makepkg download functions into libmakepkg Moved makepkg extraction functions into libmakepkg
Given the third patch is too big to get to the mailing list (and figuring out how to approve things is too much effort for me), can you push your git repo somwhere we can have a look at things?
Allan
Sure. You can see the changes here: https://github.com/AWhetter/pacman/compare/makepkg-tests
Well... I can see why it was rejected as too big! I'm going to assume the first two patches are awesome (given I wrote them...) and focus on your patches. Here are my opinions - I'd appreciated other comments on this too. 1) I do not like moving everything out of makepkg immediately and the moving it again. I'd much prefer to just move out sections at a time. So it would be great to just focus on the download functions first and once we get that sorted and committed move on to other sections. 2) At the end of this patch series we have ended up with this in makepkg: source $LIBRARY/downloads.sh source $LIBRARY/extractions.sh source $LIBRARY/utils.sh I think it would be best to have a file $LIBRARY/source.sh that is sourced in makepkg and its role it to source all the components of the library. 3) Focusing just on the downloads split, how split should we go? It has download_sources which calls download_{local,file,"vcs"} as needed. I was thinking: $LIBRARY/download.sh /download/local.sh /file.sh /git.sh /... The download.sh file would source all the files in the download directory and carry the download_sources function. @Dave: Your input would be particularly helpful here. 4) Copyright years in new file. I guess we should a git blame on makepkg and have the copyright years of the new files cover the range of the commits to those files. We also need to be careful about the other people listed at the top of the makepkg file, but I guess there is almost zero contribution left by them in makepkg these days. Allan
On 2013-09-03 06:40, Allan McRae wrote:
On 02/09/13 01:40, Ashley Whetter wrote:
On 2013-08-31 07:26, Allan McRae wrote:
On 25/08/13 22:14, awhetter.2011@my.bristol.ac.uk wrote:
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk>
In preparation for creating a makepkg test suite I've started splitting makepkg up into libmakepkg. Currently I've only split out the downloading and extracting functions. Everything else has gone into utils.
I decided to keep get_url and get_downloadclient out of the downloads library because they depend on the format of PKGBUILDs. So the new dependency graph looks like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps_v2.png Whereas the old one looked like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps.png
Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable
Ashley Whetter (3): Moved makepkg functions into a library Moved makepkg download functions into libmakepkg Moved makepkg extraction functions into libmakepkg
Given the third patch is too big to get to the mailing list (and figuring out how to approve things is too much effort for me), can you push your git repo somwhere we can have a look at things?
Allan
Sure. You can see the changes here: https://github.com/AWhetter/pacman/compare/makepkg-tests
Well... I can see why it was rejected as too big!
I'm going to assume the first two patches are awesome (given I wrote them...) and focus on your patches.
Here are my opinions - I'd appreciated other comments on this too.
1) I do not like moving everything out of makepkg immediately and the moving it again. I'd much prefer to just move out sections at a time. So it would be great to just focus on the download functions first and once we get that sorted and committed move on to other sections.
The reason I did this is because there are functions that the download functions depend on that won't end up being in the download library. I could pull these functions out as well but I think it'll end up getting too messy because we'll end up with random functions everywhere. I could instead not pull these extra functions out, but then we'd have unmet dependencies in the libraries. Or I could try and remove the dependencies all together. This is the most work but in this case it seems like the best option to me. Downloading and extracting shouldn't need anything about the PKGBUILD, about makepkg, or need (m)any extra functions. It should simply be given a set of URLs (or files) and then just download (or extract) them.
2) At the end of this patch series we have ended up with this in makepkg:
source $LIBRARY/downloads.sh source $LIBRARY/extractions.sh source $LIBRARY/utils.sh
I think it would be best to have a file $LIBRARY/source.sh that is sourced in makepkg and its role it to source all the components of the library.
What's your reason for wanting to do this?
3) Focusing just on the downloads split, how split should we go? It has download_sources which calls download_{local,file,"vcs"} as needed. I was thinking:
$LIBRARY/download.sh /download/local.sh /file.sh /git.sh /...
The download.sh file would source all the files in the download directory and carry the download_sources function.
@Dave: Your input would be particularly helpful here.
I'll wait to see if Dave has anything to say on this before doing it but it seems like a good idea to me because it means if someone wants to reuse the library, they only have to import the parts they want.
4) Copyright years in new file. I guess we should a git blame on makepkg and have the copyright years of the new files cover the range of the commits to those files. We also need to be careful about the other people listed at the top of the makepkg file, but I guess there is almost zero contribution left by them in makepkg these days.
Allan
Good idea. I'll do this the next time I update the patches. Ashley
Splitting up makepkg seems a little silly. This might qualify as "too simple", but please consider something like this this: [ $1 == "--as-lib" ] && return You place this between the function definitions and the main body of the script. When you do a source makepkg --as-lib then it will load all the functions and bail out before getting to the main code. -Kyle http://kmkeen.com
On 04/09/13 06:15, keenerd wrote:
Splitting up makepkg seems a little silly.
A >3000 line bash script that is virtually impossible to comprehensively test is silly.
This might qualify as "too simple", but please consider something like this this:
[ $1 == "--as-lib" ] && return
You place this between the function definitions and the main body of the script. When you do a
source makepkg --as-lib
then it will load all the functions and bail out before getting to the main code.
Having a library available is not the point of this. I'd guess very little else is actually going to use it. A
On Tue, Sep 03, 2013 at 03:40:24PM +1000, Allan McRae wrote:
On 02/09/13 01:40, Ashley Whetter wrote:
On 2013-08-31 07:26, Allan McRae wrote:
On 25/08/13 22:14, awhetter.2011@my.bristol.ac.uk wrote:
From: Ashley Whetter <awhetter.2011@my.bristol.ac.uk>
In preparation for creating a makepkg test suite I've started splitting makepkg up into libmakepkg. Currently I've only split out the downloading and extracting functions. Everything else has gone into utils.
I decided to keep get_url and get_downloadclient out of the downloads library because they depend on the format of PKGBUILDs. So the new dependency graph looks like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps_v2.png Whereas the old one looked like this: http://files.awhetter.co.uk/permanent/makepkg_resolved_deps.png
Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable
Ashley Whetter (3): Moved makepkg functions into a library Moved makepkg download functions into libmakepkg Moved makepkg extraction functions into libmakepkg
Given the third patch is too big to get to the mailing list (and figuring out how to approve things is too much effort for me), can you push your git repo somwhere we can have a look at things?
Allan
Sure. You can see the changes here: https://github.com/AWhetter/pacman/compare/makepkg-tests
Well... I can see why it was rejected as too big!
I'm going to assume the first two patches are awesome (given I wrote them...) and focus on your patches.
Here are my opinions - I'd appreciated other comments on this too.
1) I do not like moving everything out of makepkg immediately and the moving it again. I'd much prefer to just move out sections at a time. So it would be great to just focus on the download functions first and once we get that sorted and committed move on to other sections.
Agreed.
2) At the end of this patch series we have ended up with this in makepkg:
source $LIBRARY/downloads.sh source $LIBRARY/extractions.sh source $LIBRARY/utils.sh
I think it would be best to have a file $LIBRARY/source.sh that is sourced in makepkg and its role it to source all the components of the library.
I'm not sure how much value there is in doing this. If the idea is to have a "toplevel" file for each directory of sublibs, then it seems like source.sh doesn't seem like it would do much more than: for lib in "$LIBRARY"/*.sh; do . "$lib" done Do you forsee anything more involved that this intermediate file would need/want to do that would make not inlining it meaningful?
3) Focusing just on the downloads split, how split should we go? It has download_sources which calls download_{local,file,"vcs"} as needed. I was thinking:
$LIBRARY/download.sh /download/local.sh /file.sh /git.sh /...
The download.sh file would source all the files in the download directory and carry the download_sources function.
@Dave: Your input would be particularly helpful here.
Well, it *seems* reasonable. My only concern is how to handle common code (get_url, get_filename, get_filepath) which is called in the individual implementations. These fragments would be incomplete if sourced on their own -- this IMO diminishes the value of separating them out. Maybe-crazy idea: create a function that wraps the 'source' builtin. Perhaps it would look like: makepkg_load_module() { local module FUNCNEST=10 if [[ -z ${MAKEPKG_LOADED_MODULES["$1"]} ]]; then unset -v MAKEPKG_MODULE_DEPENDENCIES . "$LIBRARY/$1" MAKEPKG_LOADED_MODULES["$1"]=loaded for module in "${MAKEPKG_MODULE_DEPENDENCIES[@]}"; do makepkg_load_modules "$module" fi fi } MAKEPKG_LOADED_MODULES would be some associative array we declare before calling this function. modules can declare dependencies which is an array of module names it depends on. I'm not sure we want to go down this road, but I'm mentioning it anyways. It would certainly, at a minimum, make testing individual modules more straightforward.
4) Copyright years in new file. I guess we should a git blame on makepkg and have the copyright years of the new files cover the range of the commits to those files. We also need to be careful about the other people listed at the top of the makepkg file, but I guess there is almost zero contribution left by them in makepkg these days.
Allan
On 04/09/13 06:31, Dave Reisner wrote:
On Tue, Sep 03, 2013 at 03:40:24PM +1000, Allan McRae wrote:
2) At the end of this patch series we have ended up with this in makepkg:
source $LIBRARY/downloads.sh source $LIBRARY/extractions.sh source $LIBRARY/utils.sh
I think it would be best to have a file $LIBRARY/source.sh that is sourced in makepkg and its role it to source all the components of the library.
I'm not sure how much value there is in doing this. If the idea is to have a "toplevel" file for each directory of sublibs, then it seems like source.sh doesn't seem like it would do much more than:
for lib in "$LIBRARY"/*.sh; do . "$lib" done
Do you forsee anything more involved that this intermediate file would need/want to do that would make not inlining it meaningful?
Nope. The for loop is fine.
3) Focusing just on the downloads split, how split should we go? It has download_sources which calls download_{local,file,"vcs"} as needed. I was thinking:
$LIBRARY/download.sh /download/local.sh /file.sh /git.sh /...
The download.sh file would source all the files in the download directory and carry the download_sources function.
@Dave: Your input would be particularly helpful here.
Well, it *seems* reasonable. My only concern is how to handle common code (get_url, get_filename, get_filepath) which is called in the individual implementations. These fragments would be incomplete if sourced on their own -- this IMO diminishes the value of separating them out.
Maybe-crazy idea: create a function that wraps the 'source' builtin. Perhaps it would look like:
makepkg_load_module() { local module FUNCNEST=10 if [[ -z ${MAKEPKG_LOADED_MODULES["$1"]} ]]; then unset -v MAKEPKG_MODULE_DEPENDENCIES . "$LIBRARY/$1" MAKEPKG_LOADED_MODULES["$1"]=loaded for module in "${MAKEPKG_MODULE_DEPENDENCIES[@]}"; do makepkg_load_modules "$module" fi fi }
MAKEPKG_LOADED_MODULES would be some associative array we declare before calling this function. modules can declare dependencies which is an array of module names it depends on.
I'm not sure we want to go down this road, but I'm mentioning it anyways. It would certainly, at a minimum, make testing individual modules more straightforward.
This actually is a very good idea - particularly in terms of ease of testing. A module automatically calling all its dependencies would be great. After discussion on IRC, we think an inclusion guard type idea would be a more simple approach. Something like: if (( _PATH_TO_THIS_FILE )); then return; else _PATH_TO_THIS_FILE=1; fi and every file sources all the files containing its needed functions. The first patch I would like to see is: $LIBRARY/util/get_protocol.sh /get_filename.sh ... /download.sh (contains download_sources) /download/file.sh /local.sh /bzr.sh /git.sh /... In fact, make it two patches - the first moving the dependency functions into util, and the second moving the download functions out. Allan
From: Ashley Whetter <ashley@awhetter.co.uk> This time around I've split up amekpkg as per Allan's recommendation, and put each function into it's own file. There's a file at the base of each libmakepkg directory that imports that part of the library. (eg libmakepkg/download.sh imports libmakepkg/download/*.sh). Instead creating a file that imports every part of the library, I've instead put the for loop straight into makepkg.sh.in. Each libmakepkg file has it's own inclusion guard. Each libmakepkg copyright notice was deduced from the full set, dependent on what years git blame said that each line of a function had been written in. I feel like the scripts Makefile is getting a bit cluttered. Even the number of .sh.in files is getting a bit big. This could be reduced by not using LIBRARY in every libmakepkg file and using relative imports instead, but this seems like even less of a good idea. I'm not really sure if/how we want to get rid of this issue. Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable Ashley Whetter (2): Moved dependencies of download functions into a library Moved makepkg download functions into libmakepkg scripts/.gitignore | 17 + scripts/Makefile.am | 121 ++++++ scripts/libmakepkg/download.sh.in | 30 ++ scripts/libmakepkg/download/bzr.sh.in | 77 ++++ scripts/libmakepkg/download/file.sh.in | 89 +++++ scripts/libmakepkg/download/git.sh.in | 65 +++ scripts/libmakepkg/download/hg.sh.in | 59 +++ scripts/libmakepkg/download/local.sh.in | 44 ++ scripts/libmakepkg/download/sources.sh.in | 71 ++++ scripts/libmakepkg/download/svn.sh.in | 67 ++++ scripts/libmakepkg/messages.sh.in | 30 ++ scripts/libmakepkg/messages/error.sh | 32 ++ scripts/libmakepkg/messages/msg.sh | 35 ++ scripts/libmakepkg/messages/msg2.sh | 33 ++ scripts/libmakepkg/messages/plain.sh | 35 ++ scripts/libmakepkg/messages/warning.sh | 35 ++ scripts/libmakepkg/pkgbuild.sh.in | 30 ++ .../libmakepkg/pkgbuild/get_downloadclient.sh.in | 60 +++ scripts/libmakepkg/pkgbuild/get_filename.sh.in | 60 +++ scripts/libmakepkg/pkgbuild/get_filepath.sh.in | 62 +++ scripts/libmakepkg/pkgbuild/get_protocol.sh | 38 ++ scripts/libmakepkg/pkgbuild/get_url.sh | 30 ++ scripts/libmakepkg/util.sh.in | 30 ++ scripts/libmakepkg/util/cd_safe.sh.in | 36 ++ scripts/libmakepkg/util/dir_is_empty.sh | 32 ++ scripts/libmakepkg/util/in_array.sh | 38 ++ scripts/makepkg-wrapper.sh.in | 23 ++ scripts/makepkg.sh.in | 443 +-------------------- 28 files changed, 1286 insertions(+), 436 deletions(-) create mode 100644 scripts/libmakepkg/download.sh.in create mode 100644 scripts/libmakepkg/download/bzr.sh.in create mode 100644 scripts/libmakepkg/download/file.sh.in create mode 100644 scripts/libmakepkg/download/git.sh.in create mode 100644 scripts/libmakepkg/download/hg.sh.in create mode 100644 scripts/libmakepkg/download/local.sh.in create mode 100644 scripts/libmakepkg/download/sources.sh.in create mode 100644 scripts/libmakepkg/download/svn.sh.in create mode 100644 scripts/libmakepkg/messages.sh.in create mode 100644 scripts/libmakepkg/messages/error.sh create mode 100644 scripts/libmakepkg/messages/msg.sh create mode 100644 scripts/libmakepkg/messages/msg2.sh create mode 100644 scripts/libmakepkg/messages/plain.sh create mode 100644 scripts/libmakepkg/messages/warning.sh create mode 100644 scripts/libmakepkg/pkgbuild.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_downloadclient.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_filename.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_filepath.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_protocol.sh create mode 100644 scripts/libmakepkg/pkgbuild/get_url.sh create mode 100644 scripts/libmakepkg/util.sh.in create mode 100644 scripts/libmakepkg/util/cd_safe.sh.in create mode 100644 scripts/libmakepkg/util/dir_is_empty.sh create mode 100644 scripts/libmakepkg/util/in_array.sh create mode 100644 scripts/makepkg-wrapper.sh.in -- 1.8.4
From: Allan McRae <allan@archlinux.org> Build makepkg to scripts/.lib/makepkg and add a wrapper script to call it. This is not useful at the moment, but is the first step to allowing makepkg to be split into smaller pieces. Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit 015667ab2c8f0be3b5ff34cc93757e7873a09dbd) Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> Conflicts: scripts/.gitignore scripts/Makefile.am --- scripts/.gitignore | 1 + scripts/Makefile.am | 25 +++++++++++++++++++++++++ scripts/makepkg-wrapper.sh.in | 23 +++++++++++++++++++++++ 3 files changed, 49 insertions(+) create mode 100644 scripts/makepkg-wrapper.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index 26e088b..8dac503 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -1,5 +1,6 @@ makepkg makepkg-template +makepkg-wrapper pacman-db-upgrade pacman-key pacman-optimize diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 1f3bae2..f45065d 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -6,6 +6,7 @@ SUBDIRS = po bin_SCRIPTS = \ $(OURSCRIPTS) \ makepkg-template \ + makepkg-wrapper \ repo-remove \ repo-elephant @@ -20,6 +21,7 @@ OURSCRIPTS = \ EXTRA_DIST = \ makepkg.sh.in \ makepkg-template.pl.in \ + makepkg-wrapper.sh.in \ pacman-db-upgrade.sh.in \ pacman-key.sh.in \ pacman-optimize.sh.in \ @@ -37,6 +39,9 @@ LIBRARY = \ # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) +clean-local: + $(AM_V_at)$(RM) -r .lib + if USE_GIT_VERSION GIT_VERSION := $(shell sh -c 'git describe --abbrev=4 --dirty | sed s/^v//') REAL_PACKAGE_VERSION = $(GIT_VERSION) @@ -77,6 +82,7 @@ $(OURSCRIPTS): Makefile makepkg: \ $(srcdir)/makepkg.sh.in \ + $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/library/parseopts.sh makepkg-template: \ @@ -117,8 +123,27 @@ repo-elephant: $(srcdir)/repo-add.sh.in $(AM_V_at)$(RM) repo-elephant $(AM_V_at)$(LN_S) repo-add repo-elephant +makepkg-wrapper: \ + Makefile \ + $(srcdir)/makepkg-wrapper.sh.in \ + $(srcdir)/makepkg.sh.in \ + $(srcdir)/library/parseopts.sh \ + | makepkg + $(AM_V_at)$(MKDIR_P) .lib + $(AM_V_at)mv -f makepkg .lib + $(AM_V_at)$(RM) $@ + $(AM_V_GEN)sed \ + -e "s|@PWD[@]|$$(pwd)|" \ + -e '1s|!/bin/bash|!$(BASH_SHELL)|g' \ + $(srcdir)/$@.sh.in > $@ + $(AM_V_at)chmod +x,a-w $@ + $(AM_V_at)$(LN_S) makepkg-wrapper makepkg + install-data-hook: cd $(DESTDIR)$(bindir) && \ + $(RM) makepkg makepkg-wrapper + $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg + cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ ( $(LN_S) repo-add repo-elephant || \ ln repo-add repo-elephant || \ diff --git a/scripts/makepkg-wrapper.sh.in b/scripts/makepkg-wrapper.sh.in new file mode 100644 index 0000000..d703ee4 --- /dev/null +++ b/scripts/makepkg-wrapper.sh.in @@ -0,0 +1,23 @@ +#!/bin/bash +# +# makepkg - a wrapper for running the real makepkg in the source tree +# +# Copyright (c) 2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +DIR="@PWD@" + +LIBRARY="$DIR"/libmakepkg exec "$DIR"/.lib/makepkg "$@" -- 1.8.4
From: Allan McRae <allan@archlinux.org> This points makepkg to where is library is located. Can be overridden by value in the environment. Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit ab65d89fb622252eafd6f31b17d7bbcffa89dda7) Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> Conflicts: scripts/makepkg.sh.in --- scripts/Makefile.am | 3 +++ scripts/makepkg.sh.in | 2 ++ 2 files changed, 5 insertions(+) diff --git a/scripts/Makefile.am b/scripts/Makefile.am index f45065d..8130704 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -39,6 +39,8 @@ LIBRARY = \ # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) +libmakepkgdir = $(libdir)/makepkg + clean-local: $(AM_V_at)$(RM) -r .lib @@ -54,6 +56,7 @@ edit = sed \ -e 's|@localedir[@]|$(localedir)|g' \ -e 's|@sysconfdir[@]|$(sysconfdir)|g' \ -e 's|@localstatedir[@]|$(localstatedir)|g' \ + -e 's|@libmakepkgdir[@]|$(libmakepkgdir)|g' \ -e 's|@pkgdatadir[@]|$(pkgdatadir)|g' \ -e 's|@prefix[@]|$(prefix)|g' \ -e '1s|!/bin/bash|!$(BASH_SHELL)|g' \ diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index d0951df..d122537 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -46,6 +46,8 @@ declare -r confdir='@sysconfdir@' declare -r BUILDSCRIPT='@BUILDSCRIPT@' declare -r startdir="$PWD" +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + packaging_options=('strip' 'docs' 'libtool' 'staticlibs' 'emptydirs' 'zipman' \ 'purge' 'upx' 'debug') other_options=('ccache' 'distcc' 'buildflags' 'makeflags') -- 1.8.4
From: Ashley Whetter <ashley@awhetter.co.uk> Moved functions out in preparation for splitting out download functions. scripts/libmakepkg/*.sh files only import the files from their relevant directory. All libmakepkg files have an inclusion guard. Also added libmakepkg targets to Makefile.am. Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> --- scripts/.gitignore | 8 + scripts/Makefile.am | 73 ++++++++- scripts/libmakepkg/messages.sh.in | 30 ++++ scripts/libmakepkg/messages/error.sh | 32 ++++ scripts/libmakepkg/messages/msg.sh | 35 ++++ scripts/libmakepkg/messages/msg2.sh | 33 ++++ scripts/libmakepkg/messages/plain.sh | 35 ++++ scripts/libmakepkg/messages/warning.sh | 35 ++++ scripts/libmakepkg/pkgbuild.sh.in | 30 ++++ .../libmakepkg/pkgbuild/get_downloadclient.sh.in | 60 +++++++ scripts/libmakepkg/pkgbuild/get_filename.sh.in | 60 +++++++ scripts/libmakepkg/pkgbuild/get_filepath.sh.in | 62 +++++++ scripts/libmakepkg/pkgbuild/get_protocol.sh | 38 +++++ scripts/libmakepkg/pkgbuild/get_url.sh | 30 ++++ scripts/libmakepkg/util.sh.in | 30 ++++ scripts/libmakepkg/util/cd_safe.sh.in | 36 +++++ scripts/libmakepkg/util/dir_is_empty.sh | 32 ++++ scripts/libmakepkg/util/in_array.sh | 38 +++++ scripts/makepkg.sh.in | 180 +-------------------- 19 files changed, 700 insertions(+), 177 deletions(-) create mode 100644 scripts/libmakepkg/messages.sh.in create mode 100644 scripts/libmakepkg/messages/error.sh create mode 100644 scripts/libmakepkg/messages/msg.sh create mode 100644 scripts/libmakepkg/messages/msg2.sh create mode 100644 scripts/libmakepkg/messages/plain.sh create mode 100644 scripts/libmakepkg/messages/warning.sh create mode 100644 scripts/libmakepkg/pkgbuild.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_downloadclient.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_filename.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_filepath.sh.in create mode 100644 scripts/libmakepkg/pkgbuild/get_protocol.sh create mode 100644 scripts/libmakepkg/pkgbuild/get_url.sh create mode 100644 scripts/libmakepkg/util.sh.in create mode 100644 scripts/libmakepkg/util/cd_safe.sh.in create mode 100644 scripts/libmakepkg/util/dir_is_empty.sh create mode 100644 scripts/libmakepkg/util/in_array.sh diff --git a/scripts/.gitignore b/scripts/.gitignore index 8dac503..c8ccc07 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -8,3 +8,11 @@ pkgdelta repo-add repo-elephant repo-remove + +libmakepkg/messages.sh +libmakepkg/pkgbuild.sh +libmakepkg/pkgbuild/get_downloadclient.sh +libmakepkg/pkgbuild/get_filename.sh +libmakepkg/pkgbuild/get_filepath.sh +libmakepkg/util.sh +libmakepkg/util/cd_safe.sh diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 8130704..9d81878 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -36,13 +36,34 @@ LIBRARY = \ library/size_to_human.sh \ library/term_colors.sh +LIBMAKEPKG = \ + $(LIBMAKEPKG_INC) \ + messages/error.sh \ + messages/msg.sh \ + messages/msg2.sh \ + messages/plain.sh \ + messages/warning.sh \ + pkgbuild/get_protocol.sh \ + pkgbuild/get_url.sh \ + util/dir_is_empty.sh \ + util/in_array.sh + +LIBMAKEPKG_INC = \ + messages.sh \ + pkgbuild.sh \ + pkgbuild/get_downloadclient.sh \ + pkgbuild/get_filename.sh \ + pkgbuild/get_filepath.sh \ + util.sh \ + util/cd_safe.sh + # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) libmakepkgdir = $(libdir)/makepkg clean-local: - $(AM_V_at)$(RM) -r .lib + $(AM_V_at)$(RM) -r .lib $(addprefix libmakepkg/,$(LIBMAKEPKG_INC)) if USE_GIT_VERSION GIT_VERSION := $(shell sh -c 'git describe --abbrev=4 --dirty | sed s/^v//') @@ -83,7 +104,31 @@ $(OURSCRIPTS): Makefile $(AM_V_at)chmod +x,a-w $@ @$(BASH_SHELL) -O extglob -n $@ +$(addprefix libmakepkg/,$(LIBMAKEPKG_INC)): Makefile + $(AM_V_at)$(RM) $@ + $(AM_V_GEN)test -f $(srcdir)/$@.in && m4 -P -I $(srcdir) $(srcdir)/$@.in | $(edit) >$@ + @$(BASH_SHELL) -O extglob -n $@ + +libmakepkg: \ + $(srcdir)/libmakepkg/messages.sh \ + $(srcdir)/libmakepkg/messages/error.sh \ + $(srcdir)/libmakepkg/messages/msg.sh \ + $(srcdir)/libmakepkg/messages/msg2.sh \ + $(srcdir)/libmakepkg/messages/plain.sh \ + $(srcdir)/libmakepkg/messages/warning.sh \ + $(srcdir)/libmakepkg/pkgbuild.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_downloadclient.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_filename.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_filepath.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_protocol.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_url.sh \ + $(srcdir)/libmakepkg/util.sh \ + $(srcdir)/libmakepkg/util/cd_safe.sh \ + $(srcdir)/libmakepkg/util/dir_is_empty.sh \ + $(srcdir)/libmakepkg/util/in_array.sh + makepkg: \ + libmakepkg \ $(srcdir)/makepkg.sh.in \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/library/parseopts.sh @@ -131,7 +176,24 @@ makepkg-wrapper: \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/makepkg.sh.in \ $(srcdir)/library/parseopts.sh \ - | makepkg + $(srcdir)/libmakepkg/messages.sh \ + $(srcdir)/libmakepkg/messages/error.sh \ + $(srcdir)/libmakepkg/messages/msg.sh \ + $(srcdir)/libmakepkg/messages/msg2.sh \ + $(srcdir)/libmakepkg/messages/plain.sh \ + $(srcdir)/libmakepkg/messages/warning.sh \ + $(srcdir)/libmakepkg/pkgbuild.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_downloadclient.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_filename.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_filepath.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_protocol.sh \ + $(srcdir)/libmakepkg/pkgbuild/get_url.sh \ + $(srcdir)/libmakepkg/util.sh \ + $(srcdir)/libmakepkg/util/cd_safe.sh \ + $(srcdir)/libmakepkg/util/dir_is_empty.sh \ + $(srcdir)/libmakepkg/util/in_array.sh \ + | libmakepkg \ + makepkg $(AM_V_at)$(MKDIR_P) .lib $(AM_V_at)mv -f makepkg .lib $(AM_V_at)$(RM) $@ @@ -146,6 +208,10 @@ install-data-hook: cd $(DESTDIR)$(bindir) && \ $(RM) makepkg makepkg-wrapper $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg + $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir)/{messages,pkgbuild,util} + for lib in $(LIBMAKEPKG); do \ + $(INSTALL) libmakepkg/$$lib $(DESTDIR)$(libmakepkgdir)/$$lib; \ + done cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ ( $(LN_S) repo-add repo-elephant || \ @@ -160,5 +226,8 @@ install-data-hook: uninstall-hook: cd $(DESTDIR)$(bindir) && \ $(RM) repo-remove repo-elephant + for lib in $(LIBMAKEPKG); do \ + $(RM) -r $(DESTDIR)$(libmakepkgdir)/$$lib; \ + done # vim:set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/messages.sh.in b/scripts/libmakepkg/messages.sh.in new file mode 100644 index 0000000..b87219b --- /dev/null +++ b/scripts/libmakepkg/messages.sh.in @@ -0,0 +1,30 @@ +#!/bin/bash +# +# messages.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_MESSAGES_SH" ] && return +LIBMAKEPKG_MESSAGES_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +for lib in "$LIBRARY"/messages/*.sh; do + source "$lib" +done + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/messages/error.sh b/scripts/libmakepkg/messages/error.sh new file mode 100644 index 0000000..6908700 --- /dev/null +++ b/scripts/libmakepkg/messages/error.sh @@ -0,0 +1,32 @@ +#!/bin/bash +# +# error.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org> +# Copyright (c) 2005 by Aurelien Foret <orelien@chez.com> +# Copyright (c) 2006 by Miklos Vajna <vmiklos@frugalware.org> +# Copyright (c) 2005 by Christian Hamar <krics@linuxforum.hu> +# Copyright (c) 2006 by Alex Smith <alex@alex-smith.me.uk> +# Copyright (c) 2006 by Andras Voroskoi <voroskoi@frugalware.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +error() { + local mesg=$1; shift + printf "${RED}==> $(gettext "ERROR:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/messages/msg.sh b/scripts/libmakepkg/messages/msg.sh new file mode 100644 index 0000000..a2b1489 --- /dev/null +++ b/scripts/libmakepkg/messages/msg.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# +# msg.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org> +# Copyright (c) 2005 by Aurelien Foret <orelien@chez.com> +# Copyright (c) 2006 by Miklos Vajna <vmiklos@frugalware.org> +# Copyright (c) 2005 by Christian Hamar <krics@linuxforum.hu> +# Copyright (c) 2006 by Alex Smith <alex@alex-smith.me.uk> +# Copyright (c) 2006 by Andras Voroskoi <voroskoi@frugalware.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_MESSAGES_MSG_SH" ] && return +LIBMAKEPKG_MESSAGES_MSG_SH=1 + +msg() { + local mesg=$1; shift + printf "${GREEN}==>${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/messages/msg2.sh b/scripts/libmakepkg/messages/msg2.sh new file mode 100644 index 0000000..c798d1c --- /dev/null +++ b/scripts/libmakepkg/messages/msg2.sh @@ -0,0 +1,33 @@ +#!/bin/bash +# +# msg2.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org> +# Copyright (c) 2006 by Miklos Vajna <vmiklos@frugalware.org> +# Copyright (c) 2006 by Alex Smith <alex@alex-smith.me.uk> +# Copyright (c) 2006 by Andras Voroskoi <voroskoi@frugalware.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_MESSAGES_MSG2_SH" ] && return +LIBMAKEPKG_MESSAGES_MSG2_SH=1 + +msg2() { + local mesg=$1; shift + printf "${BLUE} ->${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/messages/plain.sh b/scripts/libmakepkg/messages/plain.sh new file mode 100644 index 0000000..762beb3 --- /dev/null +++ b/scripts/libmakepkg/messages/plain.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# +# plain.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org> +# Copyright (c) 2005 by Aurelien Foret <orelien@chez.com> +# Copyright (c) 2006 by Miklos Vajna <vmiklos@frugalware.org> +# Copyright (c) 2005 by Christian Hamar <krics@linuxforum.hu> +# Copyright (c) 2006 by Alex Smith <alex@alex-smith.me.uk> +# Copyright (c) 2006 by Andras Voroskoi <voroskoi@frugalware.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_MESSAGES_PLAIN_SH" ] && return +LIBMAKEPKG_MESSAGES_PLAIN_SH=1 + +plain() { + local mesg=$1; shift + printf "${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/messages/warning.sh b/scripts/libmakepkg/messages/warning.sh new file mode 100644 index 0000000..2ea86bd --- /dev/null +++ b/scripts/libmakepkg/messages/warning.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# +# warning.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org> +# Copyright (c) 2005 by Aurelien Foret <orelien@chez.com> +# Copyright (c) 2006 by Miklos Vajna <vmiklos@frugalware.org> +# Copyright (c) 2005 by Christian Hamar <krics@linuxforum.hu> +# Copyright (c) 2006 by Alex Smith <alex@alex-smith.me.uk> +# Copyright (c) 2006 by Andras Voroskoi <voroskoi@frugalware.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_MESSAGES_WARNING_SH" ] && return +LIBMAKEPKG_MESSAGES_WARNING_SH=1 + +warning() { + local mesg=$1; shift + printf "${YELLOW}==> $(gettext "WARNING:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/pkgbuild.sh.in b/scripts/libmakepkg/pkgbuild.sh.in new file mode 100644 index 0000000..7986eec --- /dev/null +++ b/scripts/libmakepkg/pkgbuild.sh.in @@ -0,0 +1,30 @@ +#!/bin/bash +# +# pkgbuild.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_PKGBUILD_SH" ] && return +LIBMAKEPKG_PKGBUILD_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +for lib in "$LIBRARY"/pkgbuild/*.sh; do + source "$lib" +done + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/pkgbuild/get_downloadclient.sh.in b/scripts/libmakepkg/pkgbuild/get_downloadclient.sh.in new file mode 100644 index 0000000..50d1ce2 --- /dev/null +++ b/scripts/libmakepkg/pkgbuild/get_downloadclient.sh.in @@ -0,0 +1,60 @@ +#!/bin/bash +# +# get_downloadclient.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_PKGBUILD_GET_DOWNLOADCLIENT_SH" ] && return +LIBMAKEPKG_PKGBUILD_GET_DOWNLOADCLIENT_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" + +get_downloadclient() { + local proto=$1 + + # loop through DOWNLOAD_AGENTS variable looking for protocol + local i + for i in "${DLAGENTS[@]}"; do + local handler="${i%%::*}" + if [[ $proto = "$handler" ]]; then + local agent="${i##*::}" + break + fi + done + + # if we didn't find an agent, return an error + if [[ -z $agent ]]; then + error "$(gettext "Unknown download protocol: %s")" "$proto" + plain "$(gettext "Aborting...")" + exit 1 # $E_CONFIG_ERROR + fi + + # ensure specified program is installed + local program="${agent%% *}" + if [[ ! -x $program ]]; then + local baseprog="${program##*/}" + error "$(gettext "The download program %s is not installed.")" "$baseprog" + plain "$(gettext "Aborting...")" + exit 1 # $E_MISSING_PROGRAM + fi + + printf "%s\n" "$agent" +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/pkgbuild/get_filename.sh.in b/scripts/libmakepkg/pkgbuild/get_filename.sh.in new file mode 100644 index 0000000..3028f1b --- /dev/null +++ b/scripts/libmakepkg/pkgbuild/get_filename.sh.in @@ -0,0 +1,60 @@ +#!/bin/bash +# +# get_filename.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_PKGBUILD_GET_FILENAME_SH" ] && return +LIBMAKEPKG_PKGBUILD_GET_FILENAME_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/pkgbuild/get_protocol.sh" + +# extract the filename from a source entry +get_filename() { + local netfile=$1 + + # if a filename is specified, use it + if [[ $netfile = *::* ]]; then + printf "%s\n" ${netfile%%::*} + return + fi + + local proto=$(get_protocol "$netfile") + + case $proto in + bzr*|git*|hg*|svn*) + filename=${netfile%%#*} + filename=${filename%/} + filename=${filename##*/} + if [[ $proto = bzr* ]]; then + filename=${filename#*lp:} + fi + if [[ $proto = git* ]]; then + filename=${filename%%.git*} + fi + ;; + *) + # if it is just an URL, we only keep the last component + filename="${netfile##*/}" + ;; + esac + printf "%s\n" "${filename}" +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/pkgbuild/get_filepath.sh.in b/scripts/libmakepkg/pkgbuild/get_filepath.sh.in new file mode 100644 index 0000000..860fbef --- /dev/null +++ b/scripts/libmakepkg/pkgbuild/get_filepath.sh.in @@ -0,0 +1,62 @@ +#!/bin/bash +# +# get_filepath.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_PKGBUILD_GET_FILEPATH_SH" ] && return +LIBMAKEPKG_PKGBUILD_GET_FILEPATH_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/pkgbuild/get_filename.sh" +source "$LIBRARY/pkgbuild/get_protocol.sh" + +# a source entry can have two forms : +# 1) "filename::http://path/to/file" +# 2) "http://path/to/file" + +# Return the absolute filename of a source entry +get_filepath() { + local file="$(get_filename "$1")" + local proto="$(get_protocol "$1")" + + case $proto in + bzr*|git*|hg*|svn*) + if [[ -d "$startdir/$file" ]]; then + file="$startdir/$file" + elif [[ -d "$SRCDEST/$file" ]]; then + file="$SRCDEST/$file" + else + return 1 + fi + ;; + *) + if [[ -f "$startdir/$file" ]]; then + file="$startdir/$file" + elif [[ -f "$SRCDEST/$file" ]]; then + file="$SRCDEST/$file" + else + return 1 + fi + ;; + esac + + printf "%s\n" "$file" +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/pkgbuild/get_protocol.sh b/scripts/libmakepkg/pkgbuild/get_protocol.sh new file mode 100644 index 0000000..d0a217b --- /dev/null +++ b/scripts/libmakepkg/pkgbuild/get_protocol.sh @@ -0,0 +1,38 @@ +#!/bin/bash +# +# get_protocol.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_PKGBUILD_GET_PROTOCOL_SH" ] && return +LIBMAKEPKG_PKGBUILD_GET_PROTOCOL_SH=1 + +# extract the protocol from a source entry - return "local" for local sources +get_protocol() { + if [[ $1 = *://* ]]; then + # strip leading filename + local proto="${1##*::}" + printf "%s\n" "${proto%%://*}" + elif [[ $1 = *lp:* ]]; then + local proto="${1##*::}" + printf "%s\n" "${proto%%lp:*}" + else + printf "%s\n" local + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/pkgbuild/get_url.sh b/scripts/libmakepkg/pkgbuild/get_url.sh new file mode 100644 index 0000000..f25698c --- /dev/null +++ b/scripts/libmakepkg/pkgbuild/get_url.sh @@ -0,0 +1,30 @@ +#!/bin/bash +# +# get_url.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$_LIBMAKEPKG_PKGBUILD_GET_URL_SH" ] && return +_LIBMAKEPKG_PKGBUILD_GET_URL_SH=1 + +# extract the URL from a source entry +get_url() { + # strip an eventual filename + printf "%s\n" "${1#*::}" +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util.sh.in b/scripts/libmakepkg/util.sh.in new file mode 100644 index 0000000..470e8f5 --- /dev/null +++ b/scripts/libmakepkg/util.sh.in @@ -0,0 +1,30 @@ +#!/bin/bash +# +# util.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_SH" ] && return +LIBMAKEPKG_UTIL_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +for lib in "$LIBRARY"/util/*.sh; do + source "$lib" +done + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util/cd_safe.sh.in b/scripts/libmakepkg/util/cd_safe.sh.in new file mode 100644 index 0000000..5319d51 --- /dev/null +++ b/scripts/libmakepkg/util/cd_safe.sh.in @@ -0,0 +1,36 @@ +#!/bin/bash +# +# cd_safe.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_CD_SAFE_SH" ] && return +LIBMAKEPKG_UTIL_CD_SAFE_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" + +cd_safe() { + if ! cd "$1"; then + error "$(gettext "Failed to change to directory %s")" "$1" + plain "$(gettext "Aborting...")" + exit 1 + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util/dir_is_empty.sh b/scripts/libmakepkg/util/dir_is_empty.sh new file mode 100644 index 0000000..1fb6ca8 --- /dev/null +++ b/scripts/libmakepkg/util/dir_is_empty.sh @@ -0,0 +1,32 @@ +#!/bin/bash +# +# dir_is_empty.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_DIR_IS_EMPTY_SH" ] && return +LIBMAKEPKG_UTIL_DIR_IS_EMPTY_SH=1 + +dir_is_empty() { + ( + shopt -s dotglob nullglob + files=("$1"/*) + (( ${#files} == 0 )) + ) +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util/in_array.sh b/scripts/libmakepkg/util/in_array.sh new file mode 100644 index 0000000..81b43ac --- /dev/null +++ b/scripts/libmakepkg/util/in_array.sh @@ -0,0 +1,38 @@ +#!/bin/bash +# +# in_array.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_IN_ARRAY_SH" ] && return +LIBMAKEPKG_UTIL_IN_ARRAY_SH=1 + +## +# usage : in_array( $needle, $haystack ) +# return : 0 - found +# 1 - not found +## +in_array() { + local needle=$1; shift + local item + for item in "$@"; do + [[ $item = "$needle" ]] && return 0 # Found + done + return 1 # Not Found +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index d122537..bce5c8d 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -97,33 +97,12 @@ PACMAN_OPTS= shopt -s extglob -### SUBROUTINES ### - -plain() { - local mesg=$1; shift - printf "${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -msg() { - local mesg=$1; shift - printf "${GREEN}==>${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -msg2() { - local mesg=$1; shift - printf "${BLUE} ->${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -warning() { - local mesg=$1; shift - printf "${YELLOW}==> $(gettext "WARNING:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -error() { - local mesg=$1; shift - printf "${RED}==> $(gettext "ERROR:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} +# Import libmakepkg +for lib in "$LIBRARY"/*.sh; do + source "$lib" +done +### SUBROUTINES ### ## # Special exit call for traps, Don't print any error messages when inside, @@ -198,124 +177,6 @@ enter_fakeroot() { fakeroot -- $0 -F "${ARGLIST[@]}" || exit $? } - -# a source entry can have two forms : -# 1) "filename::http://path/to/file" -# 2) "http://path/to/file" - -# Return the absolute filename of a source entry -get_filepath() { - local file="$(get_filename "$1")" - local proto="$(get_protocol "$1")" - - case $proto in - bzr*|git*|hg*|svn*) - if [[ -d "$startdir/$file" ]]; then - file="$startdir/$file" - elif [[ -d "$SRCDEST/$file" ]]; then - file="$SRCDEST/$file" - else - return 1 - fi - ;; - *) - if [[ -f "$startdir/$file" ]]; then - file="$startdir/$file" - elif [[ -f "$SRCDEST/$file" ]]; then - file="$SRCDEST/$file" - else - return 1 - fi - ;; - esac - - printf "%s\n" "$file" -} - -# extract the filename from a source entry -get_filename() { - local netfile=$1 - - # if a filename is specified, use it - if [[ $netfile = *::* ]]; then - printf "%s\n" ${netfile%%::*} - return - fi - - local proto=$(get_protocol "$netfile") - - case $proto in - bzr*|git*|hg*|svn*) - filename=${netfile%%#*} - filename=${filename%/} - filename=${filename##*/} - if [[ $proto = bzr* ]]; then - filename=${filename#*lp:} - fi - if [[ $proto = git* ]]; then - filename=${filename%%.git*} - fi - ;; - *) - # if it is just an URL, we only keep the last component - filename="${netfile##*/}" - ;; - esac - printf "%s\n" "${filename}" -} - -# extract the URL from a source entry -get_url() { - # strip an eventual filename - printf "%s\n" "${1#*::}" -} - -# extract the protocol from a source entry - return "local" for local sources -get_protocol() { - if [[ $1 = *://* ]]; then - # strip leading filename - local proto="${1##*::}" - printf "%s\n" "${proto%%://*}" - elif [[ $1 = *lp:* ]]; then - local proto="${1##*::}" - printf "%s\n" "${proto%%lp:*}" - else - printf "%s\n" local - fi -} - -get_downloadclient() { - local proto=$1 - - # loop through DOWNLOAD_AGENTS variable looking for protocol - local i - for i in "${DLAGENTS[@]}"; do - local handler="${i%%::*}" - if [[ $proto = "$handler" ]]; then - local agent="${i##*::}" - break - fi - done - - # if we didn't find an agent, return an error - if [[ -z $agent ]]; then - error "$(gettext "Unknown download protocol: %s")" "$proto" - plain "$(gettext "Aborting...")" - exit 1 # $E_CONFIG_ERROR - fi - - # ensure specified program is installed - local program="${agent%% *}" - if [[ ! -x $program ]]; then - local baseprog="${program##*/}" - error "$(gettext "The download program %s is not installed.")" "$baseprog" - plain "$(gettext "Aborting...")" - exit 1 # $E_MISSING_PROGRAM - fi - - printf "%s\n" "$agent" -} - download_local() { local netfile=$1 local filepath=$(get_filepath "$netfile") @@ -966,21 +827,6 @@ in_opt_array() { return 127 } - -## -# usage : in_array( $needle, $haystack ) -# return : 0 - found -# 1 - not found -## -in_array() { - local needle=$1; shift - local item - for item in "$@"; do - [[ $item = "$needle" ]] && return 0 # Found - done - return 1 # Not Found -} - source_has_signatures() { local file for file in "${source[@]}"; do @@ -1386,14 +1232,6 @@ error_function() { exit 2 # $E_BUILD_FAILED } -cd_safe() { - if ! cd "$1"; then - error "$(gettext "Failed to change to directory %s")" "$1" - plain "$(gettext "Aborting...")" - exit 1 - fi -} - source_safe() { shopt -u extglob if ! source "$@"; then @@ -2481,14 +2319,6 @@ canonicalize_path() { fi } -dir_is_empty() { - ( - shopt -s dotglob nullglob - files=("$1"/*) - (( ${#files} == 0 )) - ) -} - m4_include(library/parseopts.sh) usage() { -- 1.8.4
From: Ashley Whetter <ashley@awhetter.co.uk> Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> --- scripts/.gitignore | 8 + scripts/Makefile.am | 26 ++- scripts/libmakepkg/download.sh.in | 30 ++++ scripts/libmakepkg/download/bzr.sh.in | 77 +++++++++ scripts/libmakepkg/download/file.sh.in | 89 ++++++++++ scripts/libmakepkg/download/git.sh.in | 65 ++++++++ scripts/libmakepkg/download/hg.sh.in | 59 +++++++ scripts/libmakepkg/download/local.sh.in | 44 +++++ scripts/libmakepkg/download/sources.sh.in | 71 ++++++++ scripts/libmakepkg/download/svn.sh.in | 67 ++++++++ scripts/makepkg.sh.in | 261 ------------------------------ 11 files changed, 535 insertions(+), 262 deletions(-) create mode 100644 scripts/libmakepkg/download.sh.in create mode 100644 scripts/libmakepkg/download/bzr.sh.in create mode 100644 scripts/libmakepkg/download/file.sh.in create mode 100644 scripts/libmakepkg/download/git.sh.in create mode 100644 scripts/libmakepkg/download/hg.sh.in create mode 100644 scripts/libmakepkg/download/local.sh.in create mode 100644 scripts/libmakepkg/download/sources.sh.in create mode 100644 scripts/libmakepkg/download/svn.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index c8ccc07..e7ffc4c 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -9,6 +9,14 @@ repo-add repo-elephant repo-remove +libmakepkg/download.sh +libmakepkg/download/bzr.sh +libmakepkg/download/file.sh +libmakepkg/download/git.sh +libmakepkg/download/hg.sh +libmakepkg/download/local.sh +libmakepkg/download/sources.sh +libmakepkg/download/svn.sh libmakepkg/messages.sh libmakepkg/pkgbuild.sh libmakepkg/pkgbuild/get_downloadclient.sh diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 9d81878..35420a2 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -49,6 +49,14 @@ LIBMAKEPKG = \ util/in_array.sh LIBMAKEPKG_INC = \ + download.sh \ + download/bzr.sh \ + download/file.sh \ + download/git.sh \ + download/hg.sh \ + download/local.sh \ + download/sources.sh \ + download/svn.sh \ messages.sh \ pkgbuild.sh \ pkgbuild/get_downloadclient.sh \ @@ -110,6 +118,14 @@ $(addprefix libmakepkg/,$(LIBMAKEPKG_INC)): Makefile @$(BASH_SHELL) -O extglob -n $@ libmakepkg: \ + $(srcdir)/libmakepkg/download.sh \ + $(srcdir)/libmakepkg/download/bzr.sh \ + $(srcdir)/libmakepkg/download/file.sh \ + $(srcdir)/libmakepkg/download/git.sh \ + $(srcdir)/libmakepkg/download/hg.sh \ + $(srcdir)/libmakepkg/download/local.sh \ + $(srcdir)/libmakepkg/download/sources.sh \ + $(srcdir)/libmakepkg/download/svn.sh \ $(srcdir)/libmakepkg/messages.sh \ $(srcdir)/libmakepkg/messages/error.sh \ $(srcdir)/libmakepkg/messages/msg.sh \ @@ -176,6 +192,14 @@ makepkg-wrapper: \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/makepkg.sh.in \ $(srcdir)/library/parseopts.sh \ + $(srcdir)/libmakepkg/download.sh \ + $(srcdir)/libmakepkg/download/bzr.sh \ + $(srcdir)/libmakepkg/download/file.sh \ + $(srcdir)/libmakepkg/download/git.sh \ + $(srcdir)/libmakepkg/download/hg.sh \ + $(srcdir)/libmakepkg/download/local.sh \ + $(srcdir)/libmakepkg/download/sources.sh \ + $(srcdir)/libmakepkg/download/svn.sh \ $(srcdir)/libmakepkg/messages.sh \ $(srcdir)/libmakepkg/messages/error.sh \ $(srcdir)/libmakepkg/messages/msg.sh \ @@ -208,7 +232,7 @@ install-data-hook: cd $(DESTDIR)$(bindir) && \ $(RM) makepkg makepkg-wrapper $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg - $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir)/{messages,pkgbuild,util} + $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir)/{download,messages,pkgbuild,util} for lib in $(LIBMAKEPKG); do \ $(INSTALL) libmakepkg/$$lib $(DESTDIR)$(libmakepkgdir)/$$lib; \ done diff --git a/scripts/libmakepkg/download.sh.in b/scripts/libmakepkg/download.sh.in new file mode 100644 index 0000000..fb8ec7e --- /dev/null +++ b/scripts/libmakepkg/download.sh.in @@ -0,0 +1,30 @@ +#!/bin/bash +# +# download.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_SH" ] && return +LIBMAKEPKG_DOWNLOAD_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +for lib in "$LIBRARY"/download/*.sh; do + source "$lib" +done + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/download/bzr.sh.in b/scripts/libmakepkg/download/bzr.sh.in new file mode 100644 index 0000000..e980403 --- /dev/null +++ b/scripts/libmakepkg/download/bzr.sh.in @@ -0,0 +1,77 @@ +#!/bin/bash +# +# bzr.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_BZR_SH" ] && return +LIBMAKEPKG_DOWNLOAD_BZR_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" +source "$LIBRARY/pkgbuild.sh" +source "$LIBRARY/util.sh" + +download_bzr() { + local netfile=$1 + + local url=$(get_url "$netfile") + url=${url##*bzr+} + url=${url%%#*} + + local repo=$(get_filename "$netfile") + local displaylocation="$url" + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Branching %s ...")" "${displaylocation}" + if ! bzr branch "$url" "$dir" --no-tree --use-existing-dir; then + error "$(gettext "Failure while branching %s")" "${displaylocation}" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + # Make sure we are fetching the right repo + local distant_url="$(bzr info $url 2> /dev/null | sed -n '/branch root/{s/ branch root: //p;q;}')" + local local_url="$(bzr config parent_location -d $dir)" + if [[ -n $distant_url ]]; then + if [[ $distant_url != "$local_url" ]]; then + error "$(gettext "%s is not a branch of %s")" "$dir" "$url" + plain "$(gettext "Aborting...")" + exit 1 + fi + else + if [[ $url != "$local_url" ]] ; then + error "$(gettext "%s is not a branch of %s")" "$dir" "$url" + error "$(gettext "The local URL is %s")" "$local_url" + plain "$(gettext "Aborting...")" + exit 1 + fi + fi + msg2 "$(gettext "Pulling %s ...")" "${displaylocation}" + cd_safe "$dir" + if ! bzr pull "$url" --overwrite; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while pulling %s")" "${displaylocation}" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/download/file.sh.in b/scripts/libmakepkg/download/file.sh.in new file mode 100644 index 0000000..7831875 --- /dev/null +++ b/scripts/libmakepkg/download/file.sh.in @@ -0,0 +1,89 @@ +#!/bin/bash +# +# file.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_FILE_SH" ] && return +LIBMAKEPKG_DOWNLOAD_FILE_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" +source "$LIBRARY/pkgbuild.sh" + +download_file() { + local netfile=$1 + + local filepath=$(get_filepath "$netfile") + if [[ -n "$filepath" ]]; then + msg2 "$(gettext "Found %s")" "${filepath##*/}" + rm -f "$srcdir/${filepath##*/}" + ln -s "$filepath" "$srcdir/" + return + fi + + local proto=$(get_protocol "$netfile") + + # find the client we should use for this URL + local dlcmd + dlcmd=$(get_downloadclient "$proto") || exit $? + + local filename=$(get_filename "$netfile") + local url=$(get_url "$netfile") + + if [[ $proto = "scp" ]]; then + # scp downloads should not pass the protocol in the url + url="${url##*://}" + fi + + msg2 "$(gettext "Downloading %s...")" "$filename" + + # temporary download file, default to last component of the URL + local dlfile="${url##*/}" + + # replace %o by the temporary dlfile if it exists + if [[ $dlcmd = *%o* ]]; then + dlcmd=${dlcmd//\%o/\"$filename.part\"} + dlfile="$filename.part" + fi + # add the URL, either in place of %u or at the end + if [[ $dlcmd = *%u* ]]; then + dlcmd=${dlcmd//\%u/\"$url\"} + else + dlcmd="$dlcmd \"$url\"" + fi + + local ret=0 + eval "$dlcmd || ret=\$?" + if (( ret )); then + [[ ! -s $dlfile ]] && rm -f -- "$dlfile" + error "$(gettext "Failure while downloading %s")" "$filename" + plain "$(gettext "Aborting...")" + exit 1 + fi + + # rename the temporary download file to the final destination + if [[ $dlfile != "$filename" ]]; then + mv -f "$SRCDEST/$dlfile" "$SRCDEST/$filename" + fi + + rm -f "$srcdir/$filename" + ln -s "$SRCDEST/$filename" "$srcdir/" +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/download/git.sh.in b/scripts/libmakepkg/download/git.sh.in new file mode 100644 index 0000000..653d3a1 --- /dev/null +++ b/scripts/libmakepkg/download/git.sh.in @@ -0,0 +1,65 @@ +#!/bin/bash +# +# git.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_GIT_SH" ] && return +LIBMAKEPKG_DOWNLOAD_GIT_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" +source "$LIBRARY/pkgbuild.sh" +source "$LIBRARY/util.sh" + +download_git() { + local netfile=$1 + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + url=${url##*git+} + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git" + if ! git clone --mirror "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + cd_safe "$dir" + # Make sure we are fetching the right repo + if [[ "$url" != "$(git config --get remote.origin.url)" ]] ; then + error "$(gettext "%s is not a clone of %s")" "$dir" "$url" + plain "$(gettext "Aborting...")" + exit 1 + fi + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "git" + if ! git fetch --all -p; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "git" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/download/hg.sh.in b/scripts/libmakepkg/download/hg.sh.in new file mode 100644 index 0000000..5494775 --- /dev/null +++ b/scripts/libmakepkg/download/hg.sh.in @@ -0,0 +1,59 @@ +#!/bin/bash +# +# hg.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_HG_SH" ] && return +LIBMAKEPKG_DOWNLOAD_HG_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" +source "$LIBRARY/pkgbuild.sh" +source "$LIBRARY/util.sh" + +download_hg() { + local netfile=$1 + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + url=${url##*hg+} + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "hg" + if ! hg clone -U "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "hg" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "hg" + cd_safe "$dir" + if ! hg pull; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "hg" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/download/local.sh.in b/scripts/libmakepkg/download/local.sh.in new file mode 100644 index 0000000..a9c4ac9 --- /dev/null +++ b/scripts/libmakepkg/download/local.sh.in @@ -0,0 +1,44 @@ +#!/bin/bash +# +# local.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_LOCAL_SH" ] && return +LIBMAKEPKG_DOWNLOAD_LOCAL_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" +source "$LIBRARY/pkgbuild.sh" + +download_local() { + local netfile=$1 + local filepath=$(get_filepath "$netfile") + + if [[ -n "$filepath" ]]; then + msg2 "$(gettext "Found %s")" "${filepath##*/}" + rm -f "$srcdir/${filepath##*/}" + ln -s "$filepath" "$srcdir/" + else + local filename=$(get_filename "$netfile") + error "$(gettext "%s was not found in the build directory and is not a URL.")" "$filename" + exit 1 # $E_MISSING_FILE + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/download/sources.sh.in b/scripts/libmakepkg/download/sources.sh.in new file mode 100644 index 0000000..d118cc5 --- /dev/null +++ b/scripts/libmakepkg/download/sources.sh.in @@ -0,0 +1,71 @@ +#!/bin/bash +# +# sources.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_SOURCES_SH" ] && return +LIBMAKEPKG_DOWNLOAD_SOURCES_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/download/bzr.sh" +source "$LIBRARY/download/file.sh" +source "$LIBRARY/download/git.sh" +source "$LIBRARY/download/hg.sh" +source "$LIBRARY/download/local.sh" +source "$LIBRARY/download/svn.sh" + +download_sources() { + msg "$(gettext "Retrieving sources...")" + + local GET_VCS=1 + if [[ $1 == "fast" ]]; then + GET_VCS=0 + fi + + local netfile + for netfile in "${source[@]}"; do + pushd "$SRCDEST" &>/dev/null + + local proto=$(get_protocol "$netfile") + case "$proto" in + local) + download_local "$netfile" + ;; + bzr*) + (( GET_VCS )) && download_bzr "$netfile" + ;; + git*) + (( GET_VCS )) && download_git "$netfile" + ;; + hg*) + (( GET_VCS )) && download_hg "$netfile" + ;; + svn*) + (( GET_VCS )) && download_svn "$netfile" + ;; + *) + download_file "$netfile" + ;; + esac + + popd &>/dev/null + done +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/download/svn.sh.in b/scripts/libmakepkg/download/svn.sh.in new file mode 100644 index 0000000..51e264e --- /dev/null +++ b/scripts/libmakepkg/download/svn.sh.in @@ -0,0 +1,67 @@ +#!/bin/bash +# +# svn.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_DOWNLOAD_SVN_SH" ] && return +LIBMAKEPKG_DOWNLOAD_SVN_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/messages.sh" +source "$LIBRARY/pkgbuild.sh" +source "$LIBRARY/util.sh" + +download_svn() { + local netfile=$1 + + local fragment=${netfile#*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + if [[ $url != svn+ssh* ]]; then + url=${url##*svn+} + fi + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "svn" + mkdir -p "$dir/.makepkg" + if ! svn checkout --config-dir "$dir/.makepkg" "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "svn" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "svn" + cd_safe "$dir" + if ! svn update; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "svn" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index bce5c8d..b5ec21f 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -177,81 +177,6 @@ enter_fakeroot() { fakeroot -- $0 -F "${ARGLIST[@]}" || exit $? } -download_local() { - local netfile=$1 - local filepath=$(get_filepath "$netfile") - - if [[ -n "$filepath" ]]; then - msg2 "$(gettext "Found %s")" "${filepath##*/}" - rm -f "$srcdir/${filepath##*/}" - ln -s "$filepath" "$srcdir/" - else - local filename=$(get_filename "$netfile") - error "$(gettext "%s was not found in the build directory and is not a URL.")" "$filename" - exit 1 # $E_MISSING_FILE - fi -} - -download_file() { - local netfile=$1 - - local filepath=$(get_filepath "$netfile") - if [[ -n "$filepath" ]]; then - msg2 "$(gettext "Found %s")" "${filepath##*/}" - rm -f "$srcdir/${filepath##*/}" - ln -s "$filepath" "$srcdir/" - return - fi - - local proto=$(get_protocol "$netfile") - - # find the client we should use for this URL - local dlcmd - dlcmd=$(get_downloadclient "$proto") || exit $? - - local filename=$(get_filename "$netfile") - local url=$(get_url "$netfile") - - if [[ $proto = "scp" ]]; then - # scp downloads should not pass the protocol in the url - url="${url##*://}" - fi - - msg2 "$(gettext "Downloading %s...")" "$filename" - - # temporary download file, default to last component of the URL - local dlfile="${url##*/}" - - # replace %o by the temporary dlfile if it exists - if [[ $dlcmd = *%o* ]]; then - dlcmd=${dlcmd//\%o/\"$filename.part\"} - dlfile="$filename.part" - fi - # add the URL, either in place of %u or at the end - if [[ $dlcmd = *%u* ]]; then - dlcmd=${dlcmd//\%u/\"$url\"} - else - dlcmd="$dlcmd \"$url\"" - fi - - local ret=0 - eval "$dlcmd || ret=\$?" - if (( ret )); then - [[ ! -s $dlfile ]] && rm -f -- "$dlfile" - error "$(gettext "Failure while downloading %s")" "$filename" - plain "$(gettext "Aborting...")" - exit 1 - fi - - # rename the temporary download file to the final destination - if [[ $dlfile != "$filename" ]]; then - mv -f "$SRCDEST/$dlfile" "$SRCDEST/$filename" - fi - - rm -f "$srcdir/$filename" - ln -s "$SRCDEST/$filename" "$srcdir/" -} - extract_file() { local file=$1 # do not rely on extension for file type @@ -305,53 +230,6 @@ extract_file() { fi } -download_bzr() { - local netfile=$1 - - local url=$(get_url "$netfile") - url=${url##*bzr+} - url=${url%%#*} - - local repo=$(get_filename "$netfile") - local displaylocation="$url" - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Branching %s ...")" "${displaylocation}" - if ! bzr branch "$url" "$dir" --no-tree --use-existing-dir; then - error "$(gettext "Failure while branching %s")" "${displaylocation}" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - # Make sure we are fetching the right repo - local distant_url="$(bzr info $url 2> /dev/null | sed -n '/branch root/{s/ branch root: //p;q;}')" - local local_url="$(bzr config parent_location -d $dir)" - if [[ -n $distant_url ]]; then - if [[ $distant_url != "$local_url" ]]; then - error "$(gettext "%s is not a branch of %s")" "$dir" "$url" - plain "$(gettext "Aborting...")" - exit 1 - fi - else - if [[ $url != "$local_url" ]] ; then - error "$(gettext "%s is not a branch of %s")" "$dir" "$url" - error "$(gettext "The local URL is %s")" "$local_url" - plain "$(gettext "Aborting...")" - exit 1 - fi - fi - msg2 "$(gettext "Pulling %s ...")" "${displaylocation}" - cd_safe "$dir" - if ! bzr pull "$url" --overwrite; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while pulling %s")" "${displaylocation}" - fi - fi -} - extract_bzr() { local netfile=$1 @@ -391,41 +269,6 @@ extract_bzr() { popd &>/dev/null } -download_git() { - local netfile=$1 - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - url=${url##*git+} - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git" - if ! git clone --mirror "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - cd_safe "$dir" - # Make sure we are fetching the right repo - if [[ "$url" != "$(git config --get remote.origin.url)" ]] ; then - error "$(gettext "%s is not a clone of %s")" "$dir" "$url" - plain "$(gettext "Aborting...")" - exit 1 - fi - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "git" - if ! git fetch --all -p; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "git" - fi - fi -} - extract_git() { local netfile=$1 @@ -480,35 +323,6 @@ extract_git() { popd &>/dev/null } -download_hg() { - local netfile=$1 - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - url=${url##*hg+} - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "hg" - if ! hg clone -U "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "hg" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "hg" - cd_safe "$dir" - if ! hg pull; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "hg" - fi - fi -} - extract_hg() { local netfile=$1 @@ -549,43 +363,6 @@ extract_hg() { popd &>/dev/null } -download_svn() { - local netfile=$1 - - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - if [[ $url != svn+ssh* ]]; then - url=${url##*svn+} - fi - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "svn" - mkdir -p "$dir/.makepkg" - if ! svn checkout --config-dir "$dir/.makepkg" "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "svn" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "svn" - cd_safe "$dir" - if ! svn update; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "svn" - fi - fi -} - extract_svn() { local netfile=$1 @@ -630,44 +407,6 @@ extract_svn() { popd &>/dev/null } -download_sources() { - msg "$(gettext "Retrieving sources...")" - - local GET_VCS=1 - if [[ $1 == "fast" ]]; then - GET_VCS=0 - fi - - local netfile - for netfile in "${source[@]}"; do - pushd "$SRCDEST" &>/dev/null - - local proto=$(get_protocol "$netfile") - case "$proto" in - local) - download_local "$netfile" - ;; - bzr*) - (( GET_VCS )) && download_bzr "$netfile" - ;; - git*) - (( GET_VCS )) && download_git "$netfile" - ;; - hg*) - (( GET_VCS )) && download_hg "$netfile" - ;; - svn*) - (( GET_VCS )) && download_svn "$netfile" - ;; - *) - download_file "$netfile" - ;; - esac - - popd &>/dev/null - done -} - # Automatically update pkgver variable if a pkgver() function is provided # Re-sources the PKGBUILD afterwards to allow for other variables that use $pkgver update_pkgver() { -- 1.8.4
On 06/09/13 08:13, ashley@awhetter.co.uk wrote:
From: Ashley Whetter <ashley@awhetter.co.uk>
This time around I've split up amekpkg as per Allan's recommendation, and put each function into it's own file. There's a file at the base of each libmakepkg directory that imports that part of the library. (eg libmakepkg/download.sh imports libmakepkg/download/*.sh). Instead creating a file that imports every part of the library, I've instead put the for loop straight into makepkg.sh.in. Each libmakepkg file has it's own inclusion guard. Each libmakepkg copyright notice was deduced from the full set, dependent on what years git blame said that each line of a function had been written in.
I feel like the scripts Makefile is getting a bit cluttered. Even the number of .sh.in files is getting a bit big. This could be reduced by not using LIBRARY in every libmakepkg file and using relative imports instead, but this seems like even less of a good idea. I'm not really sure if/how we want to get rid of this issue.
OK - definitely too much splitting... I wanted the download_foo functions to be in individual files are they are fairly substantial functions. But thinks like the message functions that are four lines do not need split. So lets work on a layout before we go further. Here is a start: https://wiki.archlinux.org/index.php/User:Allan/Makepkg_Split Allan
On 2013-09-07 07:40, Allan McRae wrote:
On 06/09/13 08:13, ashley@awhetter.co.uk wrote:
From: Ashley Whetter <ashley@awhetter.co.uk>
This time around I've split up amekpkg as per Allan's recommendation, and put each function into it's own file. There's a file at the base of each libmakepkg directory that imports that part of the library. (eg libmakepkg/download.sh imports libmakepkg/download/*.sh). Instead creating a file that imports every part of the library, I've instead put the for loop straight into makepkg.sh.in. Each libmakepkg file has it's own inclusion guard. Each libmakepkg copyright notice was deduced from the full set, dependent on what years git blame said that each line of a function had been written in.
I feel like the scripts Makefile is getting a bit cluttered. Even the number of .sh.in files is getting a bit big. This could be reduced by not using LIBRARY in every libmakepkg file and using relative imports instead, but this seems like even less of a good idea. I'm not really sure if/how we want to get rid of this issue.
OK - definitely too much splitting...
I mentioned this on IRC while I was doing it and I totally agree.
I wanted the download_foo functions to be in individual files are they are fairly substantial functions. But thinks like the message functions that are four lines do not need split.
So lets work on a layout before we go further. Here is a start: https://wiki.archlinux.org/index.php/User:Allan/Makepkg_Split
Allan
I disagree with util/source.sh going into util. The source array is part of a PKGBUILD so I think it might be worth having a separate libmakepkg/pkgbuild folder, and putting source.sh and pkgbuild.sh in it. I definitely agree with grouping the download and extract functions together. It makes more sense, and adding a new VCS target will mean creating only one new file. To move out the extraction functions we'll also need to address extract_sources calling the check_build_status function. I originally grouped check_build_status into a makepkg specific part of the library. We could put check_build_status into libmakepkg/makepkg.sh (or is this what libmakepkg/packaging.sh is for?). I don't think makepkg.sh should go into the util folder because it's not a "generally useful" function, but specific to the functionality of makepkg. Also to quote something I said in my original grouping proposal "extract_sources to check_build_status is the only dependency between 'sources' and 'makepkg'. I feel like it's worth getting rid of this dependency somehow?". check_build_status also depends on get_full_version. "I think get_full_version is a PKGBUILD specific function, but run_function (in utils) depends on it to create the log file. Maybe logging stuff could be split into another [part of the] library?". check_build_status also depends on install_package. I originally put install_package in makepkg.sh/packaging.sh because although it was calling pacman, it was generating the location of the generated package file (something which is specific to makepkg) to pass onto pacman. So I didn't think it made sense to put it into dependencies.sh unless we passed it a list of package files to install. install_package depends on run_pacman. I still think it makes sense to put this into libmakepkg/util/dependencies.sh. Minor side note: Are we calling util "utils" or "util"? I keep writing utils then correcting myself, but it should be plural as it's a set of utilities. Ashley
On 2013-09-07 11:34, Ashley Whetter wrote:
On 2013-09-07 07:40, Allan McRae wrote:
On 06/09/13 08:13, ashley@awhetter.co.uk wrote:
From: Ashley Whetter <ashley@awhetter.co.uk>
This time around I've split up amekpkg as per Allan's recommendation, and put each function into it's own file. There's a file at the base of each libmakepkg directory that imports that part of the library. (eg libmakepkg/download.sh imports libmakepkg/download/*.sh). Instead creating a file that imports every part of the library, I've instead put the for loop straight into makepkg.sh.in. Each libmakepkg file has it's own inclusion guard. Each libmakepkg copyright notice was deduced from the full set, dependent on what years git blame said that each line of a function had been written in.
I feel like the scripts Makefile is getting a bit cluttered. Even the number of .sh.in files is getting a bit big. This could be reduced by not using LIBRARY in every libmakepkg file and using relative imports instead, but this seems like even less of a good idea. I'm not really sure if/how we want to get rid of this issue.
OK - definitely too much splitting...
I mentioned this on IRC while I was doing it and I totally agree.
I wanted the download_foo functions to be in individual files are they are fairly substantial functions. But thinks like the message functions that are four lines do not need split.
So lets work on a layout before we go further. Here is a start: https://wiki.archlinux.org/index.php/User:Allan/Makepkg_Split
Allan
I disagree with util/source.sh going into util. The source array is part of a PKGBUILD so I think it might be worth having a separate libmakepkg/pkgbuild folder, and putting source.sh and pkgbuild.sh in it.
I definitely agree with grouping the download and extract functions together. It makes more sense, and adding a new VCS target will mean creating only one new file.
To move out the extraction functions we'll also need to address extract_sources calling the check_build_status function. I originally grouped check_build_status into a makepkg specific part of the library. We could put check_build_status into libmakepkg/makepkg.sh (or is this what libmakepkg/packaging.sh is for?). I don't think makepkg.sh should go into the util folder because it's not a "generally useful" function, but specific to the functionality of makepkg. Also to quote something I said in my original grouping proposal "extract_sources to check_build_status is the only dependency between 'sources' and 'makepkg'. I feel like it's worth getting rid of this dependency somehow?".
check_build_status also depends on get_full_version. "I think get_full_version is a PKGBUILD specific function, but run_function (in utils) depends on it to create the log file. Maybe logging stuff could be split into another [part of the] library?". check_build_status also depends on install_package. I originally put install_package in makepkg.sh/packaging.sh because although it was calling pacman, it was generating the location of the generated package file (something which is specific to makepkg) to pass onto pacman. So I didn't think it made sense to put it into dependencies.sh unless we passed it a list of package files to install. install_package depends on run_pacman. I still think it makes sense to put this into libmakepkg/util/dependencies.sh.
Minor side note: Are we calling util "utils" or "util"? I keep writing utils then correcting myself, but it should be plural as it's a set of utilities.
Ashley
Does anyone have any comments on this? It's going to be difficult for me to do much work on it until we at least agree on a resolution to the check_build_status problem, if not the layout of the library files as well. Ashley
On 16/09/13 20:58, Ashley Whetter wrote:
On 2013-09-07 11:34, Ashley Whetter wrote:
On 2013-09-07 07:40, Allan McRae wrote:
On 06/09/13 08:13, ashley@awhetter.co.uk wrote:
From: Ashley Whetter <ashley@awhetter.co.uk>
This time around I've split up amekpkg as per Allan's recommendation, and put each function into it's own file. There's a file at the base of each libmakepkg directory that imports that part of the library. (eg libmakepkg/download.sh imports libmakepkg/download/*.sh). Instead creating a file that imports every part of the library, I've instead put the for loop straight into makepkg.sh.in. Each libmakepkg file has it's own inclusion guard. Each libmakepkg copyright notice was deduced from the full set, dependent on what years git blame said that each line of a function had been written in.
I feel like the scripts Makefile is getting a bit cluttered. Even the number of .sh.in files is getting a bit big. This could be reduced by not using LIBRARY in every libmakepkg file and using relative imports instead, but this seems like even less of a good idea. I'm not really sure if/how we want to get rid of this issue.
OK - definitely too much splitting...
I mentioned this on IRC while I was doing it and I totally agree.
I wanted the download_foo functions to be in individual files are they are fairly substantial functions. But thinks like the message functions that are four lines do not need split.
So lets work on a layout before we go further. Here is a start: https://wiki.archlinux.org/index.php/User:Allan/Makepkg_Split
Allan
I disagree with util/source.sh going into util. The source array is part of a PKGBUILD so I think it might be worth having a separate libmakepkg/pkgbuild folder, and putting source.sh and pkgbuild.sh in it.
I definitely agree with grouping the download and extract functions together. It makes more sense, and adding a new VCS target will mean creating only one new file.
To move out the extraction functions we'll also need to address extract_sources calling the check_build_status function. I originally grouped check_build_status into a makepkg specific part of the library. We could put check_build_status into libmakepkg/makepkg.sh (or is this what libmakepkg/packaging.sh is for?). I don't think makepkg.sh should go into the util folder because it's not a "generally useful" function, but specific to the functionality of makepkg. Also to quote something I said in my original grouping proposal "extract_sources to check_build_status is the only dependency between 'sources' and 'makepkg'. I feel like it's worth getting rid of this dependency somehow?".
check_build_status also depends on get_full_version. "I think get_full_version is a PKGBUILD specific function, but run_function (in utils) depends on it to create the log file. Maybe logging stuff could be split into another [part of the] library?". check_build_status also depends on install_package. I originally put install_package in makepkg.sh/packaging.sh because although it was calling pacman, it was generating the location of the generated package file (something which is specific to makepkg) to pass onto pacman. So I didn't think it made sense to put it into dependencies.sh unless we passed it a list of package files to install. install_package depends on run_pacman. I still think it makes sense to put this into libmakepkg/util/dependencies.sh.
Minor side note: Are we calling util "utils" or "util"? I keep writing utils then correcting myself, but it should be plural as it's a set of utilities.
Ashley
Does anyone have any comments on this? It's going to be difficult for me to do much work on it until we at least agree on a resolution to the check_build_status problem, if not the layout of the library files as well.
Ashley
Give me a few more days - been busy for the last couple of weeks and I am catching up. Allan
On 07/09/13 20:34, Ashley Whetter wrote:
On 2013-09-07 07:40, Allan McRae wrote:
On 06/09/13 08:13, ashley@awhetter.co.uk wrote:
From: Ashley Whetter <ashley@awhetter.co.uk>
This time around I've split up amekpkg as per Allan's recommendation, and put each function into it's own file. There's a file at the base of each libmakepkg directory that imports that part of the library. (eg libmakepkg/download.sh imports libmakepkg/download/*.sh). Instead creating a file that imports every part of the library, I've instead put the for loop straight into makepkg.sh.in. Each libmakepkg file has it's own inclusion guard. Each libmakepkg copyright notice was deduced from the full set, dependent on what years git blame said that each line of a function had been written in.
I feel like the scripts Makefile is getting a bit cluttered. Even the number of .sh.in files is getting a bit big. This could be reduced by not using LIBRARY in every libmakepkg file and using relative imports instead, but this seems like even less of a good idea. I'm not really sure if/how we want to get rid of this issue.
OK - definitely too much splitting...
I mentioned this on IRC while I was doing it and I totally agree.
I wanted the download_foo functions to be in individual files are they are fairly substantial functions. But thinks like the message functions that are four lines do not need split.
So lets work on a layout before we go further. Here is a start: https://wiki.archlinux.org/index.php/User:Allan/Makepkg_Split
Allan
I disagree with util/source.sh going into util. The source array is part of a PKGBUILD so I think it might be worth having a separate libmakepkg/pkgbuild folder, and putting source.sh and pkgbuild.sh in it.
To clarify, this is what I put in util/source.sh: utils/source.sh - get_filepath() - get_filename() - get_url() - get_protocol() I consider these utility functions for extracting information from URLs. We could name the file utils/url.sh instead?
I definitely agree with grouping the download and extract functions together. It makes more sense, and adding a new VCS target will mean creating only one new file.
To move out the extraction functions we'll also need to address extract_sources calling the check_build_status function. I originally grouped check_build_status into a makepkg specific part of the library. We could put check_build_status into libmakepkg/makepkg.sh (or is this what libmakepkg/packaging.sh is for?).
I put packaging.sh there as a place to deal with the run_build, run_package etc functions.
I don't think makepkg.sh should go into the util folder because it's not a "generally useful" function, but specific to the functionality of makepkg. Also to quote something I said in my original grouping proposal "extract_sources to check_build_status is the only dependency between 'sources' and 'makepkg'. I feel like it's worth getting rid of this dependency somehow?".
The few lines dealing with updating the pkgver and checking whether the build needs to continue should be removed from the extract sources function. They really should just be put into the makepkg script itself right after the extract_sources function is called.
check_build_status also depends on get_full_version. "I think get_full_version is a PKGBUILD specific function, but run_function (in utils) depends on it to create the log file. Maybe logging stuff could be split into another [part of the] library?". check_build_status also depends on install_package. I originally put install_package in makepkg.sh/packaging.sh because although it was calling pacman, it was generating the location of the generated package file (something which is specific to makepkg) to pass onto pacman. So I didn't think it made sense to put it into dependencies.sh unless we passed it a list of package files to install. install_package depends on run_pacman. I still think it makes sense to put this into libmakepkg/util/dependencies.sh.
I would not get too carried away with what is makepkg specific. I highly doubt anything else will make use of this "library". As far as I am concerned, the whole reason for the split is to make the script more maintainable and allow for testing individual components. I see run_pacman as a utility function.
Minor side note: Are we calling util "utils" or "util"? I keep writing utils then correcting myself, but it should be plural as it's a set of utilities.
I us "util" because we have util.c in the pacman source. But I really do not care. BTW, feel free to edit that wiki page with your proposed layout. You can even put a different section for your version. It is a lot easier to visualise than describing with text. Allan
From PATCH v2: There's still a file at the base of each libmakepkg directory that imports that
libmakepkg is now split up as per the format on the wiki. The only method I had to add to the layout was get_downloadclient, which I added to libmakepkg/util/url.sh. libmakepkg/util/url.sh was renamed from libmakepkg/util/source.sh. part of the library. (eg libmakepkg/source.sh imports libmakepkg/source/*.sh). Instead creating a file that imports every part of the library, I've instead put the for loop straight into makepkg.sh.in. Each libmakepkg file has it's own inclusion guard. Each libmakepkg copyright notice was deduced from the full set, dependent on what years git blame said that each line of a function had been written in. Allan McRae (2): makepkg: run locally with libtool style wrapper makepkg: add LIBRARY variable Ashley Whetter (2): Moved dependencies of download functions into a library Moved makepkg download functions into libmakepkg scripts/.gitignore | 12 + scripts/Makefile.am | 83 +++++++ scripts/libmakepkg/source.sh.in | 71 ++++++ scripts/libmakepkg/source/bzr.sh.in | 77 ++++++ scripts/libmakepkg/source/file.sh.in | 89 +++++++ scripts/libmakepkg/source/git.sh.in | 65 +++++ scripts/libmakepkg/source/hg.sh.in | 59 +++++ scripts/libmakepkg/source/local.sh.in | 44 ++++ scripts/libmakepkg/source/svn.sh.in | 67 +++++ scripts/libmakepkg/util.sh.in | 30 +++ scripts/libmakepkg/util/message.sh | 55 +++++ scripts/libmakepkg/util/url.sh.in | 145 +++++++++++ scripts/libmakepkg/util/util.sh.in | 58 +++++ scripts/makepkg-wrapper.sh.in | 23 ++ scripts/makepkg.sh.in | 443 +--------------------------------- 15 files changed, 885 insertions(+), 436 deletions(-) create mode 100644 scripts/libmakepkg/source.sh.in create mode 100644 scripts/libmakepkg/source/bzr.sh.in create mode 100644 scripts/libmakepkg/source/file.sh.in create mode 100644 scripts/libmakepkg/source/git.sh.in create mode 100644 scripts/libmakepkg/source/hg.sh.in create mode 100644 scripts/libmakepkg/source/local.sh.in create mode 100644 scripts/libmakepkg/source/svn.sh.in create mode 100644 scripts/libmakepkg/util.sh.in create mode 100644 scripts/libmakepkg/util/message.sh create mode 100644 scripts/libmakepkg/util/url.sh.in create mode 100644 scripts/libmakepkg/util/util.sh.in create mode 100644 scripts/makepkg-wrapper.sh.in -- 1.8.4
From: Allan McRae <allan@archlinux.org> Build makepkg to scripts/.lib/makepkg and add a wrapper script to call it. This is not useful at the moment, but is the first step to allowing makepkg to be split into smaller pieces. Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit 015667ab2c8f0be3b5ff34cc93757e7873a09dbd) Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> Conflicts: scripts/.gitignore scripts/Makefile.am --- scripts/.gitignore | 1 + scripts/Makefile.am | 25 +++++++++++++++++++++++++ scripts/makepkg-wrapper.sh.in | 23 +++++++++++++++++++++++ 3 files changed, 49 insertions(+) create mode 100644 scripts/makepkg-wrapper.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index 26e088b..8dac503 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -1,5 +1,6 @@ makepkg makepkg-template +makepkg-wrapper pacman-db-upgrade pacman-key pacman-optimize diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 1f3bae2..f45065d 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -6,6 +6,7 @@ SUBDIRS = po bin_SCRIPTS = \ $(OURSCRIPTS) \ makepkg-template \ + makepkg-wrapper \ repo-remove \ repo-elephant @@ -20,6 +21,7 @@ OURSCRIPTS = \ EXTRA_DIST = \ makepkg.sh.in \ makepkg-template.pl.in \ + makepkg-wrapper.sh.in \ pacman-db-upgrade.sh.in \ pacman-key.sh.in \ pacman-optimize.sh.in \ @@ -37,6 +39,9 @@ LIBRARY = \ # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) +clean-local: + $(AM_V_at)$(RM) -r .lib + if USE_GIT_VERSION GIT_VERSION := $(shell sh -c 'git describe --abbrev=4 --dirty | sed s/^v//') REAL_PACKAGE_VERSION = $(GIT_VERSION) @@ -77,6 +82,7 @@ $(OURSCRIPTS): Makefile makepkg: \ $(srcdir)/makepkg.sh.in \ + $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/library/parseopts.sh makepkg-template: \ @@ -117,8 +123,27 @@ repo-elephant: $(srcdir)/repo-add.sh.in $(AM_V_at)$(RM) repo-elephant $(AM_V_at)$(LN_S) repo-add repo-elephant +makepkg-wrapper: \ + Makefile \ + $(srcdir)/makepkg-wrapper.sh.in \ + $(srcdir)/makepkg.sh.in \ + $(srcdir)/library/parseopts.sh \ + | makepkg + $(AM_V_at)$(MKDIR_P) .lib + $(AM_V_at)mv -f makepkg .lib + $(AM_V_at)$(RM) $@ + $(AM_V_GEN)sed \ + -e "s|@PWD[@]|$$(pwd)|" \ + -e '1s|!/bin/bash|!$(BASH_SHELL)|g' \ + $(srcdir)/$@.sh.in > $@ + $(AM_V_at)chmod +x,a-w $@ + $(AM_V_at)$(LN_S) makepkg-wrapper makepkg + install-data-hook: cd $(DESTDIR)$(bindir) && \ + $(RM) makepkg makepkg-wrapper + $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg + cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ ( $(LN_S) repo-add repo-elephant || \ ln repo-add repo-elephant || \ diff --git a/scripts/makepkg-wrapper.sh.in b/scripts/makepkg-wrapper.sh.in new file mode 100644 index 0000000..d703ee4 --- /dev/null +++ b/scripts/makepkg-wrapper.sh.in @@ -0,0 +1,23 @@ +#!/bin/bash +# +# makepkg - a wrapper for running the real makepkg in the source tree +# +# Copyright (c) 2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +DIR="@PWD@" + +LIBRARY="$DIR"/libmakepkg exec "$DIR"/.lib/makepkg "$@" -- 1.8.4
On 22/09/13 20:25, Ashley Whetter wrote:
From: Allan McRae <allan@archlinux.org>
Build makepkg to scripts/.lib/makepkg and add a wrapper script to call it. This is not useful at the moment, but is the first step to allowing makepkg to be split into smaller pieces.
Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit 015667ab2c8f0be3b5ff34cc93757e7873a09dbd) Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk>
Conflicts: scripts/.gitignore scripts/Makefile.am
These two patches from me are fine. No need to send them yet again. I will remove the "(cherry picked..." line and the conflicts lines when I pull the patch as that is from a working branch and the commit id will be meaningless once that branch is gone. A
From: Allan McRae <allan@archlinux.org> This points makepkg to where is library is located. Can be overridden by value in the environment. Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit ab65d89fb622252eafd6f31b17d7bbcffa89dda7) Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> Conflicts: scripts/makepkg.sh.in --- scripts/Makefile.am | 3 +++ scripts/makepkg.sh.in | 2 ++ 2 files changed, 5 insertions(+) diff --git a/scripts/Makefile.am b/scripts/Makefile.am index f45065d..8130704 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -39,6 +39,8 @@ LIBRARY = \ # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) +libmakepkgdir = $(libdir)/makepkg + clean-local: $(AM_V_at)$(RM) -r .lib @@ -54,6 +56,7 @@ edit = sed \ -e 's|@localedir[@]|$(localedir)|g' \ -e 's|@sysconfdir[@]|$(sysconfdir)|g' \ -e 's|@localstatedir[@]|$(localstatedir)|g' \ + -e 's|@libmakepkgdir[@]|$(libmakepkgdir)|g' \ -e 's|@pkgdatadir[@]|$(pkgdatadir)|g' \ -e 's|@prefix[@]|$(prefix)|g' \ -e '1s|!/bin/bash|!$(BASH_SHELL)|g' \ diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 67ec240..5ac00a1 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -46,6 +46,8 @@ declare -r confdir='@sysconfdir@' declare -r BUILDSCRIPT='@BUILDSCRIPT@' declare -r startdir="$PWD" +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + packaging_options=('strip' 'docs' 'libtool' 'staticlibs' 'emptydirs' 'zipman' \ 'purge' 'upx' 'debug') other_options=('ccache' 'distcc' 'buildflags' 'makeflags') -- 1.8.4
Moved functions out in preparation for splitting out download functions. scripts/libmakepkg/*.sh files only import the files from their relevant directory. All libmakepkg files have an inclusion guard. Also added libmakepkg targets to Makefile.am. Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> --- scripts/.gitignore | 4 + scripts/Makefile.am | 38 +++++++- scripts/libmakepkg/util.sh.in | 30 +++++++ scripts/libmakepkg/util/message.sh | 55 ++++++++++++ scripts/libmakepkg/util/url.sh.in | 145 ++++++++++++++++++++++++++++++ scripts/libmakepkg/util/util.sh.in | 58 ++++++++++++ scripts/makepkg.sh.in | 180 ++----------------------------------- 7 files changed, 333 insertions(+), 177 deletions(-) create mode 100644 scripts/libmakepkg/util.sh.in create mode 100644 scripts/libmakepkg/util/message.sh create mode 100644 scripts/libmakepkg/util/url.sh.in create mode 100644 scripts/libmakepkg/util/util.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index 8dac503..6f2e84d 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -8,3 +8,7 @@ pkgdelta repo-add repo-elephant repo-remove + +libmakepkg/util.sh +libmakepkg/util/url.sh +libmakepkg/util/util.sh diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 8130704..4c52bd4 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -36,13 +36,22 @@ LIBRARY = \ library/size_to_human.sh \ library/term_colors.sh +LIBMAKEPKG = \ + $(LIBMAKEPKG_INC) \ + util/message.sh + +LIBMAKEPKG_INC = \ + util.sh \ + util/url.sh \ + util/util.sh + # Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS) libmakepkgdir = $(libdir)/makepkg clean-local: - $(AM_V_at)$(RM) -r .lib + $(AM_V_at)$(RM) -r .lib $(addprefix libmakepkg/,$(LIBMAKEPKG_INC)) if USE_GIT_VERSION GIT_VERSION := $(shell sh -c 'git describe --abbrev=4 --dirty | sed s/^v//') @@ -83,7 +92,20 @@ $(OURSCRIPTS): Makefile $(AM_V_at)chmod +x,a-w $@ @$(BASH_SHELL) -O extglob -n $@ +$(addprefix libmakepkg/,$(LIBMAKEPKG_INC)): Makefile + $(AM_V_at)$(RM) $@ + $(AM_V_GEN)test -f $(srcdir)/$@.in && m4 -P -I $(srcdir) $(srcdir)/$@.in | $(edit) >$@ + $(AM_V_at)chmod a-w $@ + @$(BASH_SHELL) -O extglob -n $@ + +libmakepkg: \ + $(srcdir)/libmakepkg/util.sh \ + $(srcdir)/libmakepkg/util/message.sh \ + $(srcdir)/libmakepkg/util/url.sh \ + $(srcdir)/libmakepkg/util/util.sh + makepkg: \ + libmakepkg \ $(srcdir)/makepkg.sh.in \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/library/parseopts.sh @@ -131,7 +153,12 @@ makepkg-wrapper: \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/makepkg.sh.in \ $(srcdir)/library/parseopts.sh \ - | makepkg + $(srcdir)/libmakepkg/util.sh \ + $(srcdir)/libmakepkg/util/message.sh \ + $(srcdir)/libmakepkg/util/url.sh \ + $(srcdir)/libmakepkg/util/util.sh \ + | libmakepkg \ + makepkg $(AM_V_at)$(MKDIR_P) .lib $(AM_V_at)mv -f makepkg .lib $(AM_V_at)$(RM) $@ @@ -146,6 +173,10 @@ install-data-hook: cd $(DESTDIR)$(bindir) && \ $(RM) makepkg makepkg-wrapper $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg + $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir)/util + for lib in $(LIBMAKEPKG); do \ + $(INSTALL) libmakepkg/$$lib $(DESTDIR)$(libmakepkgdir)/$$lib; \ + done cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ ( $(LN_S) repo-add repo-elephant || \ @@ -160,5 +191,8 @@ install-data-hook: uninstall-hook: cd $(DESTDIR)$(bindir) && \ $(RM) repo-remove repo-elephant + for lib in $(LIBMAKEPKG); do \ + $(RM) -r $(DESTDIR)$(libmakepkgdir)/$$lib; \ + done # vim:set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util.sh.in b/scripts/libmakepkg/util.sh.in new file mode 100644 index 0000000..11626bd --- /dev/null +++ b/scripts/libmakepkg/util.sh.in @@ -0,0 +1,30 @@ +#!/bin/bash +# +# util.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_SH" ] && return +LIBMAKEPKG_UTIL_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +for lib in "$LIBRARY/util/"*.sh; do + source "$lib" +done + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util/message.sh b/scripts/libmakepkg/util/message.sh new file mode 100644 index 0000000..ad134fe --- /dev/null +++ b/scripts/libmakepkg/util/message.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# +# message.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org> +# Copyright (c) 2005 by Aurelien Foret <orelien@chez.com> +# Copyright (c) 2006 by Miklos Vajna <vmiklos@frugalware.org> +# Copyright (c) 2005 by Christian Hamar <krics@linuxforum.hu> +# Copyright (c) 2006 by Alex Smith <alex@alex-smith.me.uk> +# Copyright (c) 2006 by Andras Voroskoi <voroskoi@frugalware.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_MESSAGE_SH" ] && return +LIBMAKEPKG_UTIL_MESSAGE_SH=1 + +error() { + local mesg=$1; shift + printf "${RED}==> $(gettext "ERROR:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +msg() { + local mesg=$1; shift + printf "${GREEN}==>${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +msg2() { + local mesg=$1; shift + printf "${BLUE} ->${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +plain() { + local mesg=$1; shift + printf "${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +warning() { + local mesg=$1; shift + printf "${YELLOW}==> $(gettext "WARNING:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util/url.sh.in b/scripts/libmakepkg/util/url.sh.in new file mode 100644 index 0000000..595ce16 --- /dev/null +++ b/scripts/libmakepkg/util/url.sh.in @@ -0,0 +1,145 @@ +#!/bin/bash +# +# url.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_URL_SH" ] && return +LIBMAKEPKG_UTIL_URL_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" + +get_downloadclient() { + local proto=$1 + + # loop through DOWNLOAD_AGENTS variable looking for protocol + local i + for i in "${DLAGENTS[@]}"; do + local handler="${i%%::*}" + if [[ $proto = "$handler" ]]; then + local agent="${i##*::}" + break + fi + done + + # if we didn't find an agent, return an error + if [[ -z $agent ]]; then + error "$(gettext "Unknown download protocol: %s")" "$proto" + plain "$(gettext "Aborting...")" + exit 1 # $E_CONFIG_ERROR + fi + + # ensure specified program is installed + local program="${agent%% *}" + if [[ ! -x $program ]]; then + local baseprog="${program##*/}" + error "$(gettext "The download program %s is not installed.")" "$baseprog" + plain "$(gettext "Aborting...")" + exit 1 # $E_MISSING_PROGRAM + fi + + printf "%s\n" "$agent" +} + +# extract the filename from a source entry +get_filename() { + local netfile=$1 + + # if a filename is specified, use it + if [[ $netfile = *::* ]]; then + printf "%s\n" ${netfile%%::*} + return + fi + + local proto=$(get_protocol "$netfile") + + case $proto in + bzr*|git*|hg*|svn*) + filename=${netfile%%#*} + filename=${filename%/} + filename=${filename##*/} + if [[ $proto = bzr* ]]; then + filename=${filename#*lp:} + fi + if [[ $proto = git* ]]; then + filename=${filename%%.git*} + fi + ;; + *) + # if it is just an URL, we only keep the last component + filename="${netfile##*/}" + ;; + esac + printf "%s\n" "${filename}" +} + +# a source entry can have two forms : +# 1) "filename::http://path/to/file" +# 2) "http://path/to/file" + +# Return the absolute filename of a source entry +get_filepath() { + local file="$(get_filename "$1")" + local proto="$(get_protocol "$1")" + + case $proto in + bzr*|git*|hg*|svn*) + if [[ -d "$startdir/$file" ]]; then + file="$startdir/$file" + elif [[ -d "$SRCDEST/$file" ]]; then + file="$SRCDEST/$file" + else + return 1 + fi + ;; + *) + if [[ -f "$startdir/$file" ]]; then + file="$startdir/$file" + elif [[ -f "$SRCDEST/$file" ]]; then + file="$SRCDEST/$file" + else + return 1 + fi + ;; + esac + + printf "%s\n" "$file" +} + +# extract the protocol from a source entry - return "local" for local sources +get_protocol() { + if [[ $1 = *://* ]]; then + # strip leading filename + local proto="${1##*::}" + printf "%s\n" "${proto%%://*}" + elif [[ $1 = *lp:* ]]; then + local proto="${1##*::}" + printf "%s\n" "${proto%%lp:*}" + else + printf "%s\n" local + fi +} + +# extract the URL from a source entry +get_url() { + # strip an eventual filename + printf "%s\n" "${1#*::}" +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util/util.sh.in b/scripts/libmakepkg/util/util.sh.in new file mode 100644 index 0000000..76418b0 --- /dev/null +++ b/scripts/libmakepkg/util/util.sh.in @@ -0,0 +1,58 @@ +#!/bin/bash +# +# util.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_UTIL_SH" ] && return +LIBMAKEPKG_UTIL_UTIL_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" + +cd_safe() { + if ! cd "$1"; then + error "$(gettext "Failed to change to directory %s")" "$1" + plain "$(gettext "Aborting...")" + exit 1 + fi +} + +dir_is_empty() { + ( + shopt -s dotglob nullglob + files=("$1"/*) + (( ${#files} == 0 )) + ) +} + +## +# usage : in_array( $needle, $haystack ) +# return : 0 - found +# 1 - not found +## +in_array() { + local needle=$1; shift + local item + for item in "$@"; do + [[ $item = "$needle" ]] && return 0 # Found + done + return 1 # Not Found +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 5ac00a1..d1dd964 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -97,33 +97,12 @@ PACMAN_OPTS= shopt -s extglob -### SUBROUTINES ### - -plain() { - local mesg=$1; shift - printf "${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -msg() { - local mesg=$1; shift - printf "${GREEN}==>${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -msg2() { - local mesg=$1; shift - printf "${BLUE} ->${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -warning() { - local mesg=$1; shift - printf "${YELLOW}==> $(gettext "WARNING:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -error() { - local mesg=$1; shift - printf "${RED}==> $(gettext "ERROR:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} +# Import libmakepkg +for lib in "$LIBRARY"/*.sh; do + source "$lib" +done +### SUBROUTINES ### ## # Special exit call for traps, Don't print any error messages when inside, @@ -198,124 +177,6 @@ enter_fakeroot() { fakeroot -- $0 -F "${ARGLIST[@]}" || exit $? } - -# a source entry can have two forms : -# 1) "filename::http://path/to/file" -# 2) "http://path/to/file" - -# Return the absolute filename of a source entry -get_filepath() { - local file="$(get_filename "$1")" - local proto="$(get_protocol "$1")" - - case $proto in - bzr*|git*|hg*|svn*) - if [[ -d "$startdir/$file" ]]; then - file="$startdir/$file" - elif [[ -d "$SRCDEST/$file" ]]; then - file="$SRCDEST/$file" - else - return 1 - fi - ;; - *) - if [[ -f "$startdir/$file" ]]; then - file="$startdir/$file" - elif [[ -f "$SRCDEST/$file" ]]; then - file="$SRCDEST/$file" - else - return 1 - fi - ;; - esac - - printf "%s\n" "$file" -} - -# extract the filename from a source entry -get_filename() { - local netfile=$1 - - # if a filename is specified, use it - if [[ $netfile = *::* ]]; then - printf "%s\n" ${netfile%%::*} - return - fi - - local proto=$(get_protocol "$netfile") - - case $proto in - bzr*|git*|hg*|svn*) - filename=${netfile%%#*} - filename=${filename%/} - filename=${filename##*/} - if [[ $proto = bzr* ]]; then - filename=${filename#*lp:} - fi - if [[ $proto = git* ]]; then - filename=${filename%%.git*} - fi - ;; - *) - # if it is just an URL, we only keep the last component - filename="${netfile##*/}" - ;; - esac - printf "%s\n" "${filename}" -} - -# extract the URL from a source entry -get_url() { - # strip an eventual filename - printf "%s\n" "${1#*::}" -} - -# extract the protocol from a source entry - return "local" for local sources -get_protocol() { - if [[ $1 = *://* ]]; then - # strip leading filename - local proto="${1##*::}" - printf "%s\n" "${proto%%://*}" - elif [[ $1 = *lp:* ]]; then - local proto="${1##*::}" - printf "%s\n" "${proto%%lp:*}" - else - printf "%s\n" local - fi -} - -get_downloadclient() { - local proto=$1 - - # loop through DOWNLOAD_AGENTS variable looking for protocol - local i - for i in "${DLAGENTS[@]}"; do - local handler="${i%%::*}" - if [[ $proto = "$handler" ]]; then - local agent="${i##*::}" - break - fi - done - - # if we didn't find an agent, return an error - if [[ -z $agent ]]; then - error "$(gettext "Unknown download protocol: %s")" "$proto" - plain "$(gettext "Aborting...")" - exit 1 # $E_CONFIG_ERROR - fi - - # ensure specified program is installed - local program="${agent%% *}" - if [[ ! -x $program ]]; then - local baseprog="${program##*/}" - error "$(gettext "The download program %s is not installed.")" "$baseprog" - plain "$(gettext "Aborting...")" - exit 1 # $E_MISSING_PROGRAM - fi - - printf "%s\n" "$agent" -} - download_local() { local netfile=$1 local filepath=$(get_filepath "$netfile") @@ -966,21 +827,6 @@ in_opt_array() { return 127 } - -## -# usage : in_array( $needle, $haystack ) -# return : 0 - found -# 1 - not found -## -in_array() { - local needle=$1; shift - local item - for item in "$@"; do - [[ $item = "$needle" ]] && return 0 # Found - done - return 1 # Not Found -} - source_has_signatures() { local file for file in "${source[@]}"; do @@ -1372,14 +1218,6 @@ error_function() { exit 2 # $E_BUILD_FAILED } -cd_safe() { - if ! cd "$1"; then - error "$(gettext "Failed to change to directory %s")" "$1" - plain "$(gettext "Aborting...")" - exit 1 - fi -} - source_safe() { shopt -u extglob if ! source "$@"; then @@ -2473,14 +2311,6 @@ canonicalize_path() { fi } -dir_is_empty() { - ( - shopt -s dotglob nullglob - files=("$1"/*) - (( ${#files} == 0 )) - ) -} - m4_include(library/parseopts.sh) usage() { -- 1.8.4
On 22/09/13 20:25, Ashley Whetter wrote:
Moved functions out in preparation for splitting out download functions. scripts/libmakepkg/*.sh files only import the files from their relevant directory. All libmakepkg files have an inclusion guard. Also added libmakepkg targets to Makefile.am.
Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> ---
Patch is fine but copyright years are excessive in some cases. As far as I can tell, no line has been left untouch by contributors not covered by the "Pacman Development Team" banner (apart form function names, brackets and whitespace. So we don not need to propegate those to the new files. Changes I will make while pulling this are noted below.
scripts/.gitignore | 4 + scripts/Makefile.am | 38 +++++++- scripts/libmakepkg/util.sh.in | 30 +++++++ scripts/libmakepkg/util/message.sh | 55 ++++++++++++ scripts/libmakepkg/util/url.sh.in | 145 ++++++++++++++++++++++++++++++ scripts/libmakepkg/util/util.sh.in | 58 ++++++++++++ scripts/makepkg.sh.in | 180 ++----------------------------------- 7 files changed, 333 insertions(+), 177 deletions(-) create mode 100644 scripts/libmakepkg/util.sh.in create mode 100644 scripts/libmakepkg/util/message.sh create mode 100644 scripts/libmakepkg/util/url.sh.in create mode 100644 scripts/libmakepkg/util/util.sh.in
<snip>
# vim:set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util.sh.in b/scripts/libmakepkg/util.sh.in new file mode 100644 index 0000000..11626bd --- /dev/null +++ b/scripts/libmakepkg/util.sh.in @@ -0,0 +1,30 @@ +#!/bin/bash +# +# util.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org>
Copyright (c) 2013
+# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_UTIL_SH" ] && return +LIBMAKEPKG_UTIL_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +for lib in "$LIBRARY/util/"*.sh; do + source "$lib" +done + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/util/message.sh b/scripts/libmakepkg/util/message.sh new file mode 100644 index 0000000..ad134fe --- /dev/null +++ b/scripts/libmakepkg/util/message.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# +# message.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org>
Just this line. The rest are unnneeded.
+# Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org> +# Copyright (c) 2005 by Aurelien Foret <orelien@chez.com> +# Copyright (c) 2006 by Miklos Vajna <vmiklos@frugalware.org> +# Copyright (c) 2005 by Christian Hamar <krics@linuxforum.hu> +# Copyright (c) 2006 by Alex Smith <alex@alex-smith.me.uk> +# Copyright (c) 2006 by Andras Voroskoi <voroskoi@frugalware.org> +#
The rest I will keep the same. We could probably trim from the 2006-2013 range for libmakepkg/util/url.sh.in, but it is good enough. Allan
On 09/10/13 14:40, Allan McRae wrote:
On 22/09/13 20:25, Ashley Whetter wrote:
Moved functions out in preparation for splitting out download functions. scripts/libmakepkg/*.sh files only import the files from their relevant directory. All libmakepkg files have an inclusion guard. Also added libmakepkg targets to Makefile.am.
Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> ---
Patch is fine but copyright years are excessive in some cases. As far as I can tell, no line has been left untouch by contributors not covered by the "Pacman Development Team" banner (apart form function names, brackets and whitespace. So we don not need to propegate those to the new files. Changes I will make while pulling this are noted below.
scripts/.gitignore | 4 + scripts/Makefile.am | 38 +++++++- scripts/libmakepkg/util.sh.in | 30 +++++++ scripts/libmakepkg/util/message.sh | 55 ++++++++++++ scripts/libmakepkg/util/url.sh.in | 145 ++++++++++++++++++++++++++++++ scripts/libmakepkg/util/util.sh.in | 58 ++++++++++++ scripts/makepkg.sh.in | 180 ++----------------------------------- 7 files changed, 333 insertions(+), 177 deletions(-) create mode 100644 scripts/libmakepkg/util.sh.in create mode 100644 scripts/libmakepkg/util/message.sh create mode 100644 scripts/libmakepkg/util/url.sh.in create mode 100644 scripts/libmakepkg/util/util.sh.in
Final note. New files needed to be added to scripts/po/POTFILES.in. Done on my working branch. Allan
On 09/10/13 14:46, Allan McRae wrote:
On 09/10/13 14:40, Allan McRae wrote:
On 22/09/13 20:25, Ashley Whetter wrote:
Moved functions out in preparation for splitting out download functions. scripts/libmakepkg/*.sh files only import the files from their relevant directory. All libmakepkg files have an inclusion guard. Also added libmakepkg targets to Makefile.am.
Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> ---
Patch is fine but copyright years are excessive in some cases. As far as I can tell, no line has been left untouch by contributors not covered by the "Pacman Development Team" banner (apart form function names, brackets and whitespace. So we don not need to propegate those to the new files. Changes I will make while pulling this are noted below.
scripts/.gitignore | 4 + scripts/Makefile.am | 38 +++++++- scripts/libmakepkg/util.sh.in | 30 +++++++ scripts/libmakepkg/util/message.sh | 55 ++++++++++++ scripts/libmakepkg/util/url.sh.in | 145 ++++++++++++++++++++++++++++++ scripts/libmakepkg/util/util.sh.in | 58 ++++++++++++ scripts/makepkg.sh.in | 180 ++----------------------------------- 7 files changed, 333 insertions(+), 177 deletions(-) create mode 100644 scripts/libmakepkg/util.sh.in create mode 100644 scripts/libmakepkg/util/message.sh create mode 100644 scripts/libmakepkg/util/url.sh.in create mode 100644 scripts/libmakepkg/util/util.sh.in
Final note. New files needed to be added to scripts/po/POTFILES.in. Done on my working branch.
Final, final note. There are issues in the Makefile that need fixed before I can pull this. Fix that, and address the comments in my previous replies, and I will merge.
diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 8130704..4c52bd4 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -36,13 +36,22 @@ LIBRARY = \ library/size_to_human.sh \ library/term_colors.sh
+LIBMAKEPKG = \ + $(LIBMAKEPKG_INC) \ + util/message.sh + +LIBMAKEPKG_INC = \ + util.sh \ + util/url.sh \ + util/util.sh +
message.sh does not have the library include field, but still needs modified via the $(edit) below. Join these together. Also, I guess this needs added to EXTRA_DIST to stop "make dist" breaking. Please check.
# Files that should be removed, but which Automake does not know. MOSTLYCLEANFILES = $(bin_SCRIPTS)
libmakepkgdir = $(libdir)/makepkg
clean-local: - $(AM_V_at)$(RM) -r .lib + $(AM_V_at)$(RM) -r .lib $(addprefix libmakepkg/,$(LIBMAKEPKG_INC))
if USE_GIT_VERSION GIT_VERSION := $(shell sh -c 'git describe --abbrev=4 --dirty | sed s/^v//') @@ -83,7 +92,20 @@ $(OURSCRIPTS): Makefile $(AM_V_at)chmod +x,a-w $@ @$(BASH_SHELL) -O extglob -n $@
+$(addprefix libmakepkg/,$(LIBMAKEPKG_INC)): Makefile + $(AM_V_at)$(RM) $@ + $(AM_V_GEN)test -f $(srcdir)/$@.in && m4 -P -I $(srcdir) $(srcdir)/$@.in | $(edit) >$@
Currently, we do not need to run these through m4. So: $(AM_V_GEN)test -f $(srcdir)/$@.in && $(edit) $(srcdir)/$@.in >$@
+ $(AM_V_at)chmod a-w $@ + @$(BASH_SHELL) -O extglob -n $@ + +libmakepkg: \ + $(srcdir)/libmakepkg/util.sh \ + $(srcdir)/libmakepkg/util/message.sh \ + $(srcdir)/libmakepkg/util/url.sh \ + $(srcdir)/libmakepkg/util/util.sh + makepkg: \ + libmakepkg \ $(srcdir)/makepkg.sh.in \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/library/parseopts.sh @@ -131,7 +153,12 @@ makepkg-wrapper: \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/makepkg.sh.in \ $(srcdir)/library/parseopts.sh \ - | makepkg + $(srcdir)/libmakepkg/util.sh \ + $(srcdir)/libmakepkg/util/message.sh \ + $(srcdir)/libmakepkg/util/url.sh \ + $(srcdir)/libmakepkg/util/util.sh \ + | libmakepkg \ + makepkg
You have added libmakepkg as a dependency and an order-only dependency here. Only the latter is needed. And there is no need to add the files from libmakepkg, as they are covered already.
$(AM_V_at)$(MKDIR_P) .lib $(AM_V_at)mv -f makepkg .lib $(AM_V_at)$(RM) $@ @@ -146,6 +173,10 @@ install-data-hook: cd $(DESTDIR)$(bindir) && \ $(RM) makepkg makepkg-wrapper $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg + $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir)/util + for lib in $(LIBMAKEPKG); do \ + $(INSTALL) libmakepkg/$$lib $(DESTDIR)$(libmakepkgdir)/$$lib; \ + done cd $(DESTDIR)$(bindir) && \ $(RM) repo-elephant && \ ( $(LN_S) repo-add repo-elephant || \ @@ -160,5 +191,8 @@ install-data-hook: uninstall-hook: cd $(DESTDIR)$(bindir) && \ $(RM) repo-remove repo-elephant + for lib in $(LIBMAKEPKG); do \ + $(RM) -r $(DESTDIR)$(libmakepkgdir)/$$lib; \
What is the -r doing. The directories are not in $(LIBMAKEPKG) so they will not be removed.
+ done
# vim:set ts=2 sw=2 noet:
Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> --- scripts/.gitignore | 7 + scripts/Makefile.am | 23 ++- scripts/libmakepkg/source.sh.in | 71 +++++++++ scripts/libmakepkg/source/bzr.sh.in | 77 ++++++++++ scripts/libmakepkg/source/file.sh.in | 89 ++++++++++++ scripts/libmakepkg/source/git.sh.in | 65 +++++++++ scripts/libmakepkg/source/hg.sh.in | 59 ++++++++ scripts/libmakepkg/source/local.sh.in | 44 ++++++ scripts/libmakepkg/source/svn.sh.in | 67 +++++++++ scripts/makepkg.sh.in | 261 ---------------------------------- 10 files changed, 501 insertions(+), 262 deletions(-) create mode 100644 scripts/libmakepkg/source.sh.in create mode 100644 scripts/libmakepkg/source/bzr.sh.in create mode 100644 scripts/libmakepkg/source/file.sh.in create mode 100644 scripts/libmakepkg/source/git.sh.in create mode 100644 scripts/libmakepkg/source/hg.sh.in create mode 100644 scripts/libmakepkg/source/local.sh.in create mode 100644 scripts/libmakepkg/source/svn.sh.in diff --git a/scripts/.gitignore b/scripts/.gitignore index 6f2e84d..c129207 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -9,6 +9,13 @@ repo-add repo-elephant repo-remove +libmakepkg/source.sh +libmakepkg/source/bzr.sh +libmakepkg/source/file.sh +libmakepkg/source/git.sh +libmakepkg/source/hg.sh +libmakepkg/source/local.sh +libmakepkg/source/svn.sh libmakepkg/util.sh libmakepkg/util/url.sh libmakepkg/util/util.sh diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 4c52bd4..994a168 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -41,6 +41,13 @@ LIBMAKEPKG = \ util/message.sh LIBMAKEPKG_INC = \ + source.sh \ + source/bzr.sh \ + source/file.sh \ + source/git.sh \ + source/hg.sh \ + source/local.sh \ + source/svn.sh \ util.sh \ util/url.sh \ util/util.sh @@ -99,6 +106,13 @@ $(addprefix libmakepkg/,$(LIBMAKEPKG_INC)): Makefile @$(BASH_SHELL) -O extglob -n $@ libmakepkg: \ + $(srcdir)/libmakepkg/source.sh \ + $(srcdir)/libmakepkg/source/bzr.sh \ + $(srcdir)/libmakepkg/source/file.sh \ + $(srcdir)/libmakepkg/source/git.sh \ + $(srcdir)/libmakepkg/source/hg.sh \ + $(srcdir)/libmakepkg/source/local.sh \ + $(srcdir)/libmakepkg/source/svn.sh \ $(srcdir)/libmakepkg/util.sh \ $(srcdir)/libmakepkg/util/message.sh \ $(srcdir)/libmakepkg/util/url.sh \ @@ -153,6 +167,13 @@ makepkg-wrapper: \ $(srcdir)/makepkg-wrapper.sh.in \ $(srcdir)/makepkg.sh.in \ $(srcdir)/library/parseopts.sh \ + $(srcdir)/libmakepkg/source.sh \ + $(srcdir)/libmakepkg/source/bzr.sh \ + $(srcdir)/libmakepkg/source/file.sh \ + $(srcdir)/libmakepkg/source/git.sh \ + $(srcdir)/libmakepkg/source/hg.sh \ + $(srcdir)/libmakepkg/source/local.sh \ + $(srcdir)/libmakepkg/source/svn.sh \ $(srcdir)/libmakepkg/util.sh \ $(srcdir)/libmakepkg/util/message.sh \ $(srcdir)/libmakepkg/util/url.sh \ @@ -173,7 +194,7 @@ install-data-hook: cd $(DESTDIR)$(bindir) && \ $(RM) makepkg makepkg-wrapper $(INSTALL) .lib/makepkg $(DESTDIR)$(bindir)/makepkg - $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir)/util + $(AM_V_at)$(MKDIR_P) $(DESTDIR)$(libmakepkgdir)/{source,util} for lib in $(LIBMAKEPKG); do \ $(INSTALL) libmakepkg/$$lib $(DESTDIR)$(libmakepkgdir)/$$lib; \ done diff --git a/scripts/libmakepkg/source.sh.in b/scripts/libmakepkg/source.sh.in new file mode 100644 index 0000000..3e69aa4 --- /dev/null +++ b/scripts/libmakepkg/source.sh.in @@ -0,0 +1,71 @@ +#!/bin/bash +# +# download.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_SOURCE_SH" ] && return +LIBMAKEPKG_SOURCE_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +for lib in "$LIBRARY/source/"*.sh; do + source "$lib" +done + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/url.sh" + +download_sources() { + msg "$(gettext "Retrieving sources...")" + + local GET_VCS=1 + if [[ $1 == "fast" ]]; then + GET_VCS=0 + fi + + local netfile + for netfile in "${source[@]}"; do + pushd "$SRCDEST" &>/dev/null + + local proto=$(get_protocol "$netfile") + case "$proto" in + local) + download_local "$netfile" + ;; + bzr*) + (( GET_VCS )) && download_bzr "$netfile" + ;; + git*) + (( GET_VCS )) && download_git "$netfile" + ;; + hg*) + (( GET_VCS )) && download_hg "$netfile" + ;; + svn*) + (( GET_VCS )) && download_svn "$netfile" + ;; + *) + download_file "$netfile" + ;; + esac + + popd &>/dev/null + done +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/source/bzr.sh.in b/scripts/libmakepkg/source/bzr.sh.in new file mode 100644 index 0000000..85e6436 --- /dev/null +++ b/scripts/libmakepkg/source/bzr.sh.in @@ -0,0 +1,77 @@ +#!/bin/bash +# +# bzr.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_SOURCE_BZR_SH" ] && return +LIBMAKEPKG_SOURCE_BZR_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/url.sh" +source "$LIBRARY/util/util.sh" + +download_bzr() { + local netfile=$1 + + local url=$(get_url "$netfile") + url=${url##*bzr+} + url=${url%%#*} + + local repo=$(get_filename "$netfile") + local displaylocation="$url" + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Branching %s ...")" "${displaylocation}" + if ! bzr branch "$url" "$dir" --no-tree --use-existing-dir; then + error "$(gettext "Failure while branching %s")" "${displaylocation}" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + # Make sure we are fetching the right repo + local distant_url="$(bzr info $url 2> /dev/null | sed -n '/branch root/{s/ branch root: //p;q;}')" + local local_url="$(bzr config parent_location -d $dir)" + if [[ -n $distant_url ]]; then + if [[ $distant_url != "$local_url" ]]; then + error "$(gettext "%s is not a branch of %s")" "$dir" "$url" + plain "$(gettext "Aborting...")" + exit 1 + fi + else + if [[ $url != "$local_url" ]] ; then + error "$(gettext "%s is not a branch of %s")" "$dir" "$url" + error "$(gettext "The local URL is %s")" "$local_url" + plain "$(gettext "Aborting...")" + exit 1 + fi + fi + msg2 "$(gettext "Pulling %s ...")" "${displaylocation}" + cd_safe "$dir" + if ! bzr pull "$url" --overwrite; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while pulling %s")" "${displaylocation}" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/source/file.sh.in b/scripts/libmakepkg/source/file.sh.in new file mode 100644 index 0000000..14a122e --- /dev/null +++ b/scripts/libmakepkg/source/file.sh.in @@ -0,0 +1,89 @@ +#!/bin/bash +# +# file.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_SOURCE_FILE_SH" ] && return +LIBMAKEPKG_SOURCE_FILE_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/url.sh" + +download_file() { + local netfile=$1 + + local filepath=$(get_filepath "$netfile") + if [[ -n "$filepath" ]]; then + msg2 "$(gettext "Found %s")" "${filepath##*/}" + rm -f "$srcdir/${filepath##*/}" + ln -s "$filepath" "$srcdir/" + return + fi + + local proto=$(get_protocol "$netfile") + + # find the client we should use for this URL + local dlcmd + dlcmd=$(get_downloadclient "$proto") || exit $? + + local filename=$(get_filename "$netfile") + local url=$(get_url "$netfile") + + if [[ $proto = "scp" ]]; then + # scp downloads should not pass the protocol in the url + url="${url##*://}" + fi + + msg2 "$(gettext "Downloading %s...")" "$filename" + + # temporary download file, default to last component of the URL + local dlfile="${url##*/}" + + # replace %o by the temporary dlfile if it exists + if [[ $dlcmd = *%o* ]]; then + dlcmd=${dlcmd//\%o/\"$filename.part\"} + dlfile="$filename.part" + fi + # add the URL, either in place of %u or at the end + if [[ $dlcmd = *%u* ]]; then + dlcmd=${dlcmd//\%u/\"$url\"} + else + dlcmd="$dlcmd \"$url\"" + fi + + local ret=0 + eval "$dlcmd >&2 || ret=\$?" + if (( ret )); then + [[ ! -s $dlfile ]] && rm -f -- "$dlfile" + error "$(gettext "Failure while downloading %s")" "$filename" + plain "$(gettext "Aborting...")" + exit 1 + fi + + # rename the temporary download file to the final destination + if [[ $dlfile != "$filename" ]]; then + mv -f "$SRCDEST/$dlfile" "$SRCDEST/$filename" + fi + + rm -f "$srcdir/$filename" + ln -s "$SRCDEST/$filename" "$srcdir/" +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/source/git.sh.in b/scripts/libmakepkg/source/git.sh.in new file mode 100644 index 0000000..ab7fdce --- /dev/null +++ b/scripts/libmakepkg/source/git.sh.in @@ -0,0 +1,65 @@ +#!/bin/bash +# +# git.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_SOURCE_GIT_SH" ] && return +LIBMAKEPKG_SOURCE_GIT_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/url.sh" +source "$LIBRARY/util/util.sh" + +download_git() { + local netfile=$1 + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + url=${url##*git+} + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git" + if ! git clone --mirror "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + cd_safe "$dir" + # Make sure we are fetching the right repo + if [[ "$url" != "$(git config --get remote.origin.url)" ]] ; then + error "$(gettext "%s is not a clone of %s")" "$dir" "$url" + plain "$(gettext "Aborting...")" + exit 1 + fi + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "git" + if ! git fetch --all -p; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "git" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/source/hg.sh.in b/scripts/libmakepkg/source/hg.sh.in new file mode 100644 index 0000000..e3cd115 --- /dev/null +++ b/scripts/libmakepkg/source/hg.sh.in @@ -0,0 +1,59 @@ +#!/bin/bash +# +# hg.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_SOURCE_HG_SH" ] && return +LIBMAKEPKG_SOURCE_HG_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/url.sh" +source "$LIBRARY/util/util.sh" + +download_hg() { + local netfile=$1 + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + url=${url##*hg+} + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "hg" + if ! hg clone -U "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "hg" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "hg" + cd_safe "$dir" + if ! hg pull; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "hg" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/source/local.sh.in b/scripts/libmakepkg/source/local.sh.in new file mode 100644 index 0000000..d51317b --- /dev/null +++ b/scripts/libmakepkg/source/local.sh.in @@ -0,0 +1,44 @@ +#!/bin/bash +# +# local.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_SOURCE_LOCAL_SH" ] && return +LIBMAKEPKG_SOURCE_LOCAL_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/url.sh" + +download_local() { + local netfile=$1 + local filepath=$(get_filepath "$netfile") + + if [[ -n "$filepath" ]]; then + msg2 "$(gettext "Found %s")" "${filepath##*/}" + rm -f "$srcdir/${filepath##*/}" + ln -s "$filepath" "$srcdir/" + else + local filename=$(get_filename "$netfile") + error "$(gettext "%s was not found in the build directory and is not a URL.")" "$filename" + exit 1 # $E_MISSING_FILE + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/libmakepkg/source/svn.sh.in b/scripts/libmakepkg/source/svn.sh.in new file mode 100644 index 0000000..5b7b557 --- /dev/null +++ b/scripts/libmakepkg/source/svn.sh.in @@ -0,0 +1,67 @@ +#!/bin/bash +# +# svn.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[ -n "$LIBMAKEPKG_SOURCE_SVN_SH" ] && return +LIBMAKEPKG_SOURCE_SVN_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/url.sh" +source "$LIBRARY/util/util.sh" + +download_svn() { + local netfile=$1 + + local fragment=${netfile#*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=$(get_filename "$netfile") + + local url=$(get_url "$netfile") + if [[ $url != svn+ssh* ]]; then + url=${url##*svn+} + fi + url=${url%%#*} + + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "svn" + mkdir -p "$dir/.makepkg" + if ! svn checkout --config-dir "$dir/.makepkg" "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "svn" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "svn" + cd_safe "$dir" + if ! svn update; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "svn" + fi + fi +} + +# vim: set ts=2 sw=2 noet: diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index d1dd964..42d45af 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -177,81 +177,6 @@ enter_fakeroot() { fakeroot -- $0 -F "${ARGLIST[@]}" || exit $? } -download_local() { - local netfile=$1 - local filepath=$(get_filepath "$netfile") - - if [[ -n "$filepath" ]]; then - msg2 "$(gettext "Found %s")" "${filepath##*/}" - rm -f "$srcdir/${filepath##*/}" - ln -s "$filepath" "$srcdir/" - else - local filename=$(get_filename "$netfile") - error "$(gettext "%s was not found in the build directory and is not a URL.")" "$filename" - exit 1 # $E_MISSING_FILE - fi -} - -download_file() { - local netfile=$1 - - local filepath=$(get_filepath "$netfile") - if [[ -n "$filepath" ]]; then - msg2 "$(gettext "Found %s")" "${filepath##*/}" - rm -f "$srcdir/${filepath##*/}" - ln -s "$filepath" "$srcdir/" - return - fi - - local proto=$(get_protocol "$netfile") - - # find the client we should use for this URL - local dlcmd - dlcmd=$(get_downloadclient "$proto") || exit $? - - local filename=$(get_filename "$netfile") - local url=$(get_url "$netfile") - - if [[ $proto = "scp" ]]; then - # scp downloads should not pass the protocol in the url - url="${url##*://}" - fi - - msg2 "$(gettext "Downloading %s...")" "$filename" - - # temporary download file, default to last component of the URL - local dlfile="${url##*/}" - - # replace %o by the temporary dlfile if it exists - if [[ $dlcmd = *%o* ]]; then - dlcmd=${dlcmd//\%o/\"$filename.part\"} - dlfile="$filename.part" - fi - # add the URL, either in place of %u or at the end - if [[ $dlcmd = *%u* ]]; then - dlcmd=${dlcmd//\%u/\"$url\"} - else - dlcmd="$dlcmd \"$url\"" - fi - - local ret=0 - eval "$dlcmd >&2 || ret=\$?" - if (( ret )); then - [[ ! -s $dlfile ]] && rm -f -- "$dlfile" - error "$(gettext "Failure while downloading %s")" "$filename" - plain "$(gettext "Aborting...")" - exit 1 - fi - - # rename the temporary download file to the final destination - if [[ $dlfile != "$filename" ]]; then - mv -f "$SRCDEST/$dlfile" "$SRCDEST/$filename" - fi - - rm -f "$srcdir/$filename" - ln -s "$SRCDEST/$filename" "$srcdir/" -} - extract_file() { local file=$1 # do not rely on extension for file type @@ -305,53 +230,6 @@ extract_file() { fi } -download_bzr() { - local netfile=$1 - - local url=$(get_url "$netfile") - url=${url##*bzr+} - url=${url%%#*} - - local repo=$(get_filename "$netfile") - local displaylocation="$url" - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Branching %s ...")" "${displaylocation}" - if ! bzr branch "$url" "$dir" --no-tree --use-existing-dir; then - error "$(gettext "Failure while branching %s")" "${displaylocation}" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - # Make sure we are fetching the right repo - local distant_url="$(bzr info $url 2> /dev/null | sed -n '/branch root/{s/ branch root: //p;q;}')" - local local_url="$(bzr config parent_location -d $dir)" - if [[ -n $distant_url ]]; then - if [[ $distant_url != "$local_url" ]]; then - error "$(gettext "%s is not a branch of %s")" "$dir" "$url" - plain "$(gettext "Aborting...")" - exit 1 - fi - else - if [[ $url != "$local_url" ]] ; then - error "$(gettext "%s is not a branch of %s")" "$dir" "$url" - error "$(gettext "The local URL is %s")" "$local_url" - plain "$(gettext "Aborting...")" - exit 1 - fi - fi - msg2 "$(gettext "Pulling %s ...")" "${displaylocation}" - cd_safe "$dir" - if ! bzr pull "$url" --overwrite; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while pulling %s")" "${displaylocation}" - fi - fi -} - extract_bzr() { local netfile=$1 @@ -391,41 +269,6 @@ extract_bzr() { popd &>/dev/null } -download_git() { - local netfile=$1 - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - url=${url##*git+} - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git" - if ! git clone --mirror "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - cd_safe "$dir" - # Make sure we are fetching the right repo - if [[ "$url" != "$(git config --get remote.origin.url)" ]] ; then - error "$(gettext "%s is not a clone of %s")" "$dir" "$url" - plain "$(gettext "Aborting...")" - exit 1 - fi - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "git" - if ! git fetch --all -p; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "git" - fi - fi -} - extract_git() { local netfile=$1 @@ -480,35 +323,6 @@ extract_git() { popd &>/dev/null } -download_hg() { - local netfile=$1 - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - url=${url##*hg+} - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "hg" - if ! hg clone -U "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "hg" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "hg" - cd_safe "$dir" - if ! hg pull; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "hg" - fi - fi -} - extract_hg() { local netfile=$1 @@ -549,43 +363,6 @@ extract_hg() { popd &>/dev/null } -download_svn() { - local netfile=$1 - - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - - local dir=$(get_filepath "$netfile") - [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" - - local repo=$(get_filename "$netfile") - - local url=$(get_url "$netfile") - if [[ $url != svn+ssh* ]]; then - url=${url##*svn+} - fi - url=${url%%#*} - - if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then - msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "svn" - mkdir -p "$dir/.makepkg" - if ! svn checkout --config-dir "$dir/.makepkg" "$url" "$dir"; then - error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "svn" - plain "$(gettext "Aborting...")" - exit 1 - fi - elif (( ! HOLDVER )); then - msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "svn" - cd_safe "$dir" - if ! svn update; then - # only warn on failure to allow offline builds - warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "svn" - fi - fi -} - extract_svn() { local netfile=$1 @@ -630,44 +407,6 @@ extract_svn() { popd &>/dev/null } -download_sources() { - msg "$(gettext "Retrieving sources...")" - - local GET_VCS=1 - if [[ $1 == "fast" ]]; then - GET_VCS=0 - fi - - local netfile - for netfile in "${source[@]}"; do - pushd "$SRCDEST" &>/dev/null - - local proto=$(get_protocol "$netfile") - case "$proto" in - local) - download_local "$netfile" - ;; - bzr*) - (( GET_VCS )) && download_bzr "$netfile" - ;; - git*) - (( GET_VCS )) && download_git "$netfile" - ;; - hg*) - (( GET_VCS )) && download_hg "$netfile" - ;; - svn*) - (( GET_VCS )) && download_svn "$netfile" - ;; - *) - download_file "$netfile" - ;; - esac - - popd &>/dev/null - done -} - # Automatically update pkgver variable if a pkgver() function is provided # Re-sources the PKGBUILD afterwards to allow for other variables that use $pkgver update_pkgver() { -- 1.8.4
On 22/09/13 20:25, Ashley Whetter wrote:
Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> ---
Minor comment below for which I need opinions: <snip>
diff --git a/scripts/libmakepkg/source.sh.in b/scripts/libmakepkg/source.sh.in new file mode 100644 index 0000000..3e69aa4 --- /dev/null +++ b/scripts/libmakepkg/source.sh.in @@ -0,0 +1,71 @@ +#!/bin/bash +# +# download.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org>
Looking at that code, every line from from that function was changed in 2012 or 2103. The VCS functions were all written in 2012 and 2013 too. So what year should we be using for the copyright notices in the split files? Allan
On 22/09/13 20:25, Ashley Whetter wrote:
Signed-off-by: Ashley Whetter <ashley@awhetter.co.uk> --- scripts/.gitignore | 7 + scripts/Makefile.am | 23 ++- scripts/libmakepkg/source.sh.in | 71 +++++++++ scripts/libmakepkg/source/bzr.sh.in | 77 ++++++++++ scripts/libmakepkg/source/file.sh.in | 89 ++++++++++++ scripts/libmakepkg/source/git.sh.in | 65 +++++++++ scripts/libmakepkg/source/hg.sh.in | 59 ++++++++ scripts/libmakepkg/source/local.sh.in | 44 ++++++ scripts/libmakepkg/source/svn.sh.in | 67 +++++++++ scripts/makepkg.sh.in | 261 ---------------------------------- 10 files changed, 501 insertions(+), 262 deletions(-) create mode 100644 scripts/libmakepkg/source.sh.in create mode 100644 scripts/libmakepkg/source/bzr.sh.in create mode 100644 scripts/libmakepkg/source/file.sh.in create mode 100644 scripts/libmakepkg/source/git.sh.in create mode 100644 scripts/libmakepkg/source/hg.sh.in create mode 100644 scripts/libmakepkg/source/local.sh.in create mode 100644 scripts/libmakepkg/source/svn.sh.in
Similar to previous patch. Pulling with minor copyright year changes (noted below) and adding relevant new files to POTFILES.in. <snip>
diff --git a/scripts/libmakepkg/source/bzr.sh.in b/scripts/libmakepkg/source/bzr.sh.in new file mode 100644 index 0000000..85e6436 --- /dev/null +++ b/scripts/libmakepkg/source/bzr.sh.in @@ -0,0 +1,77 @@ +#!/bin/bash +# +# bzr.sh +# +# Copyright (c) 2006-2013 Pacman Development Team <pacman-dev@archlinux.org>
All the VCS files are 2012-2013.
participants (8)
-
Allan McRae
-
Ashley Whetter
-
Ashley Whetter
-
ashley＠awhetter.co.uk
-
awhetter.2011＠my.bristol.ac.uk
-
Dave Reisner
-
keenerd
-
Pierre Schmitz