[arch-projects] [ABS] [PATCH v3 0/7] vcs prototype cleanups and some git-specific changes

Linus Arver

9 Nov 2011 9 Nov '11

6:56 a.m.

Hello all, So this is the third iteration of my initial patch series. Sorry about the weird, non-uniform mailings before (I learned about git send-email today). The biggest change from the earlier series is that for the git prototype, I actually threw out the temporary build directory altogether. Jesse Young's suggestion of just using the SCM builtins to get a pristine working directory started making more sense to me as I thought about the problem. My initial resistance to the idea doesn't make sense --- sorry Jesse! Unfortunately, I am really only familiar with git, so I couldn't make a uniform change across all VCS prototypes. I also used rsync to efficiently copy the temporary directory for cvs/svn prototypes (instead of doing a "cp -r" followed with a find command for recursive deletion of unwanted directories). The other notable change is that I dropped the idea of deleting the temporary build directories inside package() altogether (PATCH v2 1/7), as Lukas Fleischer suggested. Linus Arver (7): vcs prototypes: consistent $PWD after checkout vcs prototypes: typo/stylistic fixes git prototype: on initial clones, perform a shallow clone git prototype: remove temp build directory vcs prototypes: simplify code vcs prototypes: consistent coding style vcs prototypes: more efficient temp build directories prototypes/PKGBUILD-bzr.proto | 14 ++++++++------ prototypes/PKGBUILD-cvs.proto | 16 ++++++++-------- prototypes/PKGBUILD-darcs.proto | 20 ++++++++++---------- prototypes/PKGBUILD-git.proto | 22 +++++++++++----------- prototypes/PKGBUILD-hg.proto | 17 +++++++++-------- prototypes/PKGBUILD-svn.proto | 14 ++++++++------ 6 files changed, 54 insertions(+), 49 deletions(-) -- 1.7.7.2

Show replies by date

Linus Arver

9 Nov 9 Nov

6:56 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 1/7] vcs prototypes: consistent $PWD after checkout

Before, some vcs prototypes (bzr, git, hg, svn) cd'ed into the repo directory after a checkout, but not on an initial clone. The cvs/darcs prototypes made it uniform by always going into the repo after the checkout, whether on an initial clone or an update. With this change, we make all vcs prototypes simply not remain in the repo directory after a checkout (whether initial clone or update). Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 1 + prototypes/PKGBUILD-cvs.proto | 2 +- prototypes/PKGBUILD-darcs.proto | 2 +- prototypes/PKGBUILD-git.proto | 1 + prototypes/PKGBUILD-hg.proto | 1 + prototypes/PKGBUILD-svn.proto | 1 + 6 files changed, 6 insertions(+), 2 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index 81f145f..bca92f1 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -37,6 +37,7 @@ build() { if [[ -d "$_bzrmod" ]]; then cd "$_bzrmod" && bzr --no-plugins pull "$_bzrtrunk" -r "$pkgver" msg "The local files are updated." + cd .. else bzr --no-plugins branch "$_bzrtrunk" "$_bzrmod" -q -r "$pkgver" fi diff --git a/prototypes/PKGBUILD-cvs.proto b/prototypes/PKGBUILD-cvs.proto index 982c149..5c883bc 100644 --- a/prototypes/PKGBUILD-cvs.proto +++ b/prototypes/PKGBUILD-cvs.proto @@ -37,9 +37,9 @@ build() { if [[ -d "$_cvsmod/CVS" ]]; then cd "$_cvsmod" cvs -z3 update -d + cd .. else cvs -z3 -d "$_cvsroot" co -D "$pkgver" -f "$_cvsmod" - cd "$_cvsmod" fi msg "CVS checkout done or server timeout" diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index 979e9b5..28b0c32 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -38,10 +38,10 @@ build() { msg "Retrieving missing patches" cd "$_darcsmod" darcs pull -a "$_darcstrunk/$_darcsmod" + cd .. else msg "Retrieving complete sources" darcs get --partial --set-scripts-executable "$_darcstrunk/$_darcsmod" - cd "$_darcsmod" fi rm -rf "$srcdir/$_darcsmod-build" diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index 05b721b..78f3992 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -37,6 +37,7 @@ build() { if [[ -d "$_gitname" ]]; then cd "$_gitname" && git pull origin msg "The local files are updated." + cd .. else git clone "$_gitroot" "$_gitname" fi diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index 8474533..e583142 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -38,6 +38,7 @@ build() { cd "$_hgrepo" hg pull -u msg "The local files are updated." + cd .. else hg clone "$_hgroot" "$_hgrepo" fi diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index a08e0e3..2afb533 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -36,6 +36,7 @@ build() { if [[ -d "$_svnmod/.svn" ]]; then (cd "$_svnmod" && svn up -r "$pkgver") + cd .. else svn co "$_svntrunk" --config-dir ./ -r "$pkgver" "$_svnmod" fi -- 1.7.7.2

Linus Arver

6:56 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 2/7] vcs prototypes: typo/stylistic fixes

Make messages uniform. Also, stop calling repositories "server", and display the repository for more transparency. Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 4 ++-- prototypes/PKGBUILD-cvs.proto | 5 +++-- prototypes/PKGBUILD-darcs.proto | 10 +++++----- prototypes/PKGBUILD-git.proto | 6 +++--- prototypes/PKGBUILD-hg.proto | 4 ++-- prototypes/PKGBUILD-svn.proto | 5 +++-- 6 files changed, 18 insertions(+), 16 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index bca92f1..b412247 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -32,7 +32,7 @@ _bzrmod=MODNAME build() { cd "$srcdir" - msg "Connecting to Bazaar server...." + msg "Connecting to $_bzrtrunk ..." if [[ -d "$_bzrmod" ]]; then cd "$_bzrmod" && bzr --no-plugins pull "$_bzrtrunk" -r "$pkgver" @@ -42,7 +42,7 @@ build() { bzr --no-plugins branch "$_bzrtrunk" "$_bzrmod" -q -r "$pkgver" fi - msg "Bazaar checkout done or server timeout" + msg "Bazaar checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_bzrmod-build" diff --git a/prototypes/PKGBUILD-cvs.proto b/prototypes/PKGBUILD-cvs.proto index 5c883bc..a987437 100644 --- a/prototypes/PKGBUILD-cvs.proto +++ b/prototypes/PKGBUILD-cvs.proto @@ -32,17 +32,18 @@ _cvsmod=MODNAME build() { cd "$srcdir" - msg "Connecting to $_cvsmod.sourceforge.net CVS server...." + msg "Connecting to $_cvsmod ..." if [[ -d "$_cvsmod/CVS" ]]; then cd "$_cvsmod" cvs -z3 update -d + msg "The local files are updated." cd .. else cvs -z3 -d "$_cvsroot" co -D "$pkgver" -f "$_cvsmod" fi - msg "CVS checkout done or server timeout" + msg "CVS checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_cvsmod-build" diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index 28b0c32..8dd0b48 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -32,24 +32,24 @@ _darcsmod=MODNAME build() { cd "$srcdir" - msg "Checking for previous build...." + msg "Connecting to $_darcstrunk ..." if [[ -d "$_darcsmod/_darcs" ]]; then - msg "Retrieving missing patches" cd "$_darcsmod" darcs pull -a "$_darcstrunk/$_darcsmod" + msg "The local files are updated." cd .. else - msg "Retrieving complete sources" darcs get --partial --set-scripts-executable "$_darcstrunk/$_darcsmod" fi + msg "Darcs checkout done or connection timeout" + msg "Starting build..." + rm -rf "$srcdir/$_darcsmod-build" cp -r "$srcdir/$_darcsmod" "$srcdir/$_darcsmod-build" cd "$srcdir/$_darcsmod-build" - msg "Starting build..." - # # BUILD HERE # diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index 78f3992..dd3f76e 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -28,11 +28,11 @@ noextract=() md5sums=() #generate with 'makepkg -g' _gitroot=GITURL -_gitname=MODENAME +_gitname=REPONAME build() { cd "$srcdir" - msg "Connecting to GIT server...." + msg "Connecting to $_gitroot ..." if [[ -d "$_gitname" ]]; then cd "$_gitname" && git pull origin @@ -42,7 +42,7 @@ build() { git clone "$_gitroot" "$_gitname" fi - msg "GIT checkout done or server timeout" + msg "GIT checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_gitname-build" diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index e583142..38713eb 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -32,7 +32,7 @@ _hgrepo=REPONAME build() { cd "$srcdir" - msg "Connecting to Mercurial server...." + msg "Connecting to $_hgroot ..." if [[ -d "$_hgrepo" ]]; then cd "$_hgrepo" @@ -43,7 +43,7 @@ build() { hg clone "$_hgroot" "$_hgrepo" fi - msg "Mercurial checkout done or server timeout" + msg "Mercurial checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_hgrepo-build" diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index 2afb533..3271361 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -32,16 +32,17 @@ _svnmod=MODNAME build() { cd "$srcdir" - msg "Connecting to SVN server...." + msg "Connecting to $_svntrunk ..." if [[ -d "$_svnmod/.svn" ]]; then (cd "$_svnmod" && svn up -r "$pkgver") + msg "The local files are updated." cd .. else svn co "$_svntrunk" --config-dir ./ -r "$pkgver" "$_svnmod" fi - msg "SVN checkout done or server timeout" + msg "SVN checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_svnmod-build" -- 1.7.7.2

Linus Arver

6:56 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 3/7] git prototype: on initial clones, perform a shallow clone

Shallow git clones are just like regular clones, but do not contain any of the past commit history. It is virtually the same thing as doing a regular clone, then doing a rebase to squash all commits into a single commit. Many people who do not understand git dismiss shallow clones because they wrongly believe that shallow clones are incapable of pulling in changes going forward from the remote. This is not the case! You can still do pulls from the master remote repo in the future to update the shallow clone, just like a regular clone! On an inital clone, we should *always* encourage PKGBUILD authors to do a shallow clone. This will save time (less downloading) and disk space (e.g., yaourt users). The savings can be hundreds of MiB for large git repos. This will also help AUR packagers out there who do not understand git at all to make this change themselves. Git PKGBUILD authors who need to pull in an older version of the remote repo (an extremely rare case) will already have the knowledge to remove the "--depth 1" to suit their needs. Signed-off-by: Linus Arver --- prototypes/PKGBUILD-git.proto | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index dd3f76e..7d9bb90 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -39,7 +39,7 @@ build() { msg "The local files are updated." cd .. else - git clone "$_gitroot" "$_gitname" + git clone --depth 1 "$_gitroot" "$_gitname" fi msg "GIT checkout done or connection timeout" -- 1.7.7.2

C Anthony Risinger

7:28 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 3/7] git prototype: on initial clones, perform a shallow clone

On Tue, Nov 8, 2011 at 11:56 PM, Linus Arver wrote:

...

Shallow git clones are just like regular clones, but do not contain any of the past commit history. It is virtually the same thing as doing a regular clone, then doing a rebase to squash all commits into a single commit. Many people who do not understand git dismiss shallow clones because they wrongly believe that shallow clones are incapable of pulling in changes going forward from the remote. This is not the case! You can still do pulls from the master remote repo in the future to update the shallow clone, just like a regular clone!

i had a few other improvements that may be of interest, outlined here: http://mailman.archlinux.org/pipermail/arch-general/2011-July/021078.html ... [sort of] condensed: ) `_gitname` is not used consistently ... though i now forget the various uses ive seen :-( will have to follow up on that ) allow _git* variables to be set by the environment ) introduce `_gitspec` variable which supersedes `_gitname` at *checkout* stage (fallback to `_gitname`) ) use a targeted fetch command instead of a clone -- this can achieve even greater savings than shallow clone, even though the fetch is "deep". the idea is to pull only $_gitname and _nothing_ else. this can however be combined with shallow if done correctly for even greater savings (this method results in 50%+ reduction to kernel pull [dont know shallow variant offhand], and i've seen savings as high as 90%+) ) store repositories in a known list of locations ... people WILL blow the repo away if it's in the build dir. my PKGBUILDs searches these (in order of precedence): /var/abs/PKGBUILD.devel/${pkgname}.git /var/abs/local/PKGBUILD.devel/${pkgname}.git ${SRCDEST}/PKGBUILD.devel/${pkgname}.git ${startdir}/PKGBUILD.devel/${pkgname}.git ~/PKGBUILD.devel/${pkgname}.git ) use a proxy mechanism in the event a repository is found but it is read-only. this lets you read-only bind mount a repo (think mkchrootpkg), and it will simply create a new repository, copy the refs, and setup the object directory as an alternative for the proxy repo. thus the proxy has the ability to download new objects as needed, but starts from the same spot as the bind-mounted repo. ... these techniques are all in use, and primarily derived from experiences developing, this PKGBUILD: http://aur.archlinux.org/packages/pyjamas-engine-pythonwebkit/PKGBUILD ... which is a massive 1GB+ download and lengthy compile. these modifications also make it very simple to rapidly build git packages within a chroot (one of the primary motivations) *without* any copying/etc. probably a little out of scope from what you've done here, and possibly in need of further discussion, but you're message sparked memory and i still believe they are all good changes -- it saved me oodles of time and prevents constant removal of humongous repos (esp. when in chroot). -- C Anthony

Linus Arver

10 Nov 10 Nov

4:46 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 3/7] git prototype: on initial clones, perform a shallow clone

On Wed, Nov 09, 2011 at 12:28:23AM -0600, C Anthony Risinger wrote:

...

On Tue, Nov 8, 2011 at 11:56 PM, Linus Arver wrote:

...
Shallow git clones are just like regular clones, but do not contain any of the past commit history. It is virtually the same thing as doing a regular clone, then doing a rebase to squash all commits into a single commit. Many people who do not understand git dismiss shallow clones because they wrongly believe that shallow clones are incapable of pulling in changes going forward from the remote. This is not the case! You can still do pulls from the master remote repo in the future to update the shallow clone, just like a regular clone!

i had a few other improvements that may be of interest, outlined here:

http://mailman.archlinux.org/pipermail/arch-general/2011-July/021078.html

... [sort of] condensed:

) `_gitname` is not used consistently ... though i now forget the various uses ive seen :-( will have to follow up on that

) allow _git* variables to be set by the environment

) introduce `_gitspec` variable which supersedes `_gitname` at *checkout* stage (fallback to `_gitname`)

) use a targeted fetch command instead of a clone -- this can achieve even greater savings than shallow clone, even though the fetch is "deep". the idea is to pull only $_gitname and _nothing_ else. this can however be combined with shallow if done correctly for even greater savings (this method results in 50%+ reduction to kernel pull [dont know shallow variant offhand], and i've seen savings as high as 90%+)

) store repositories in a known list of locations ... people WILL blow the repo away if it's in the build dir. my PKGBUILDs searches these (in order of precedence):

/var/abs/PKGBUILD.devel/${pkgname}.git /var/abs/local/PKGBUILD.devel/${pkgname}.git ${SRCDEST}/PKGBUILD.devel/${pkgname}.git ${startdir}/PKGBUILD.devel/${pkgname}.git ~/PKGBUILD.devel/${pkgname}.git

) use a proxy mechanism in the event a repository is found but it is read-only. this lets you read-only bind mount a repo (think mkchrootpkg), and it will simply create a new repository, copy the refs, and setup the object directory as an alternative for the proxy repo. thus the proxy has the ability to download new objects as needed, but starts from the same spot as the bind-mounted repo.

... these techniques are all in use, and primarily derived from experiences developing, this PKGBUILD:

http://aur.archlinux.org/packages/pyjamas-engine-pythonwebkit/PKGBUILD

... which is a massive 1GB+ download and lengthy compile. these modifications also make it very simple to rapidly build git packages within a chroot (one of the primary motivations) *without* any copying/etc.

probably a little out of scope from what you've done here, and possibly in need of further discussion, but you're message sparked memory and i still believe they are all good changes -- it saved me oodles of time and prevents constant removal of humongous repos (esp. when in chroot).

--

C Anthony

The problem I have with your sample PKGBUILD is that it is extremely complicated. Anything extremely complicated goes entirely against the KISS philosophy that we Arch devs/contributors cherish. See https://wiki.archlinux.org/index.php/The_Arch_Way But of course, you are free to write up a separate patch series for git. At this time, however, I am unwilling to delay this patch series to incorporate such extensive changes. -Linus

C Anthony Risinger

7:56 p.m.

New subject: [arch-projects] [ABS] [PATCH v3 3/7] git prototype: on initial clones, perform a shallow clone

On Wed, Nov 9, 2011 at 9:46 PM, Linus Arver wrote:

...

The problem I have with your sample PKGBUILD is that it is extremely complicated. Anything extremely complicated goes entirely against the KISS philosophy that we Arch devs/contributors cherish. See https://wiki.archlinux.org/index.php/The_Arch_Way

well, the message i linked had the bits i was referring to factored out, and they amount to about 15-20 lines -- the PKGBUILD i linked *is* complex, but not complicated ... there is alot going on, and it's a less than trivial build. the bits relating to git are pretty clear, imo at least. "the arch way" is a great guideline -- i believe i've made the process as simple as it *can* be made ;-)

...

But of course, you are free to write up a separate patch series for git. At this time, however, I am unwilling to delay this patch series to incorporate such extensive changes.

that is fine, i was not suggesting any alteration or amendments to your series. it simply reminded me of a my prior work; work i believe is still more than valid. perhaps i should have created a new thread from it, but i'm still not 100% the expectations of this list. no worries, i may be able to spin some patches, but i would recommend maybe reading my original linked proposal, as i think it labels the goals pretty well. -- C Anthony

Linus Arver

11 Nov 11 Nov

6:08 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 3/7] git prototype: on initial clones, perform a shallow clone

On Thu, Nov 10, 2011 at 12:56:35PM -0600, C Anthony Risinger wrote:

...

On Wed, Nov 9, 2011 at 9:46 PM, Linus Arver wrote:

...
The problem I have with your sample PKGBUILD is that it is extremely complicated. Anything extremely complicated goes entirely against the KISS philosophy that we Arch devs/contributors cherish. See https://wiki.archlinux.org/index.php/The_Arch_Way

well, the message i linked had the bits i was referring to factored out, and they amount to about 15-20 lines -- the PKGBUILD i linked *is* complex, but not complicated ... there is alot going on, and it's a less than trivial build.

the bits relating to git are pretty clear, imo at least. "the arch way" is a great guideline -- i believe i've made the process as simple as it *can* be made ;-)

I think your 10-20-line excerpts outlined in http://mailman.archlinux.org/pipermail/arch-general/2011-July/021078.html are still extremely complicated. But maybe I'm the only one who thinks this way (your technical competence with git is certainly above my own). But I think that the PKGBUILD prototypes are meant to be a very simple, sane starting point for devs/contributors to create their own. If we end up introducing too many concepts into these prototypes, maybe they may not be helpful in the end. Perhaps there should be two sets of prototypes --- an "beginner" and "advanced" version. Or maybe the ideas you introduced in your email belong to the Arch Wiki, and not the prototype (under a heading like "Advanced Git PKGBUILD Techniques).

...

...
But of course, you are free to write up a separate patch series for git. At this time, however, I am unwilling to delay this patch series to incorporate such extensive changes.

that is fine, i was not suggesting any alteration or amendments to your series. it simply reminded me of a my prior work; work i believe is still more than valid. perhaps i should have created a new thread from it, but i'm still not 100% the expectations of this list.

no worries, i may be able to spin some patches, but i would recommend maybe reading my original linked proposal, as i think it labels the goals pretty well.

Yes, please do create a new thread/patch series. Hopefully my series will be merged soonish. I think you should make very small, incremental changes at a time. That way, you won't scare off all the people on the list who are not as technically competent in git as yourself. ;) -Linus P.S. When writing a list of paragraphs, use '*' or '-', not ')'.

C Anthony Risinger

7:01 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 3/7] git prototype: on initial clones, perform a shallow clone

On Thu, Nov 10, 2011 at 11:08 PM, Linus Arver wrote:

...

But I think that the PKGBUILD prototypes are meant to be a very simple, sane starting point for devs/contributors to create their own. If we end up introducing too many concepts into these prototypes, maybe they may not be helpful in the end. Perhaps there should be two sets of prototypes --- an "beginner" and "advanced" version. Or maybe the ideas you introduced in your email belong to the Arch Wiki, and not the prototype (under a heading like "Advanced Git PKGBUILD Techniques).

i think the best route is some kind of lower level integration, at makepkg (or ?) level ... i know someone brought up a bug report regarding just that. i'll mull that a bit and probably make noise there instead :-)

...

I think you should make very small, incremental changes at a time. That way, you won't scare off all the people on the list who are not as technically competent in git as yourself. ;)

indeed :-) i'll seek the appropriate channel for these changes.

...

P.S. When writing a list of paragraphs, use '*' or '-', not ')'.

bah! i've done that for quite some time :-) i use `)` specifically because: 1) a) ... looks nice and clear to me, but i tend to skip backfilling the letter/numbers. maybe `*)` or `x)` is better? i'll consider `*`, but `-` is ... nay :-) aaaanyways, i'll look into lower level integration so users can simply set a variable or something, and not even call git directly. -- C Anthony

Linus Arver

9 Nov 9 Nov

6:56 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 4/7] git prototype: remove temp build directory

Git comes with commands that are specifically designed to return the working directory to a "pristine" state. We make use of these commands to avoid the expensive operation of creating a temporary build directory. Having a single directory to pull in upstream code and build from greatly simplifies the code, and as a bonus, saves disk space/time! Signed-off-by: Linus Arver --- prototypes/PKGBUILD-git.proto | 13 ++++++------- 1 files changed, 6 insertions(+), 7 deletions(-) diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index 7d9bb90..a2f2f40 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -35,20 +35,19 @@ build() { msg "Connecting to $_gitroot ..." if [[ -d "$_gitname" ]]; then - cd "$_gitname" && git pull origin + cd "$_gitname" + git clean -dxf + git reset --hard + git pull origin msg "The local files are updated." - cd .. else git clone --depth 1 "$_gitroot" "$_gitname" + cd "$_gitname" fi msg "GIT checkout done or connection timeout" msg "Starting build..." - rm -rf "$srcdir/$_gitname-build" - git clone "$srcdir/$_gitname" "$srcdir/$_gitname-build" - cd "$srcdir/$_gitname-build" - # # BUILD HERE # @@ -58,7 +57,7 @@ build() { } package() { - cd "$srcdir/$_gitname-build" + cd "$srcdir/$_gitname" make DESTDIR="$pkgdir/" install } -- 1.7.7.2

Linus Arver

6:56 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 5/7] vcs prototypes: simplify code

Since the $PWD after the update/initial clone is always "$srcdir", we don't need to reference it later on. Signed-off-by: Linus Arver <linusarver at gmail.com> Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 6 +++--- prototypes/PKGBUILD-cvs.proto | 6 +++--- prototypes/PKGBUILD-darcs.proto | 6 +++--- prototypes/PKGBUILD-hg.proto | 6 +++--- prototypes/PKGBUILD-svn.proto | 6 +++--- 5 files changed, 15 insertions(+), 15 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index b412247..3f6d2e9 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -45,9 +45,9 @@ build() { msg "Bazaar checkout done or connection timeout" msg "Starting build..." - rm -rf "$srcdir/$_bzrmod-build" - cp -r "$srcdir/$_bzrmod" "$srcdir/$_bzrmod-build" - cd "$srcdir/$_bzrmod-build" + rm -rf "$_bzrmod-build" + cp -r "$_bzrmod" "$_bzrmod-build" + cd "$_bzrmod-build" # # BUILD HERE diff --git a/prototypes/PKGBUILD-cvs.proto b/prototypes/PKGBUILD-cvs.proto index a987437..0ddd91d 100644 --- a/prototypes/PKGBUILD-cvs.proto +++ b/prototypes/PKGBUILD-cvs.proto @@ -46,9 +46,9 @@ build() { msg "CVS checkout done or connection timeout" msg "Starting build..." - rm -rf "$srcdir/$_cvsmod-build" - cp -r "$srcdir/$_cvsmod" "$srcdir/$_cvsmod-build" - cd "$srcdir/$_cvsmod-build" + rm -rf "$_cvsmod-build" + cp -r "$_cvsmod" "$_cvsmod-build" + cd "$_cvsmod-build" # # BUILD HERE diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index 8dd0b48..1103588 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -46,9 +46,9 @@ build() { msg "Darcs checkout done or connection timeout" msg "Starting build..." - rm -rf "$srcdir/$_darcsmod-build" - cp -r "$srcdir/$_darcsmod" "$srcdir/$_darcsmod-build" - cd "$srcdir/$_darcsmod-build" + rm -rf "$_darcsmod-build" + cp -r "$_darcsmod" "$_darcsmod-build" + cd "$_darcsmod-build" # # BUILD HERE diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index 38713eb..b6610fa 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -46,9 +46,9 @@ build() { msg "Mercurial checkout done or connection timeout" msg "Starting build..." - rm -rf "$srcdir/$_hgrepo-build" - cp -r "$srcdir/$_hgrepo" "$srcdir/$_hgrepo-build" - cd "$srcdir/$_hgrepo-build" + rm -rf "$_hgrepo-build" + cp -r "$_hgrepo" "$_hgrepo-build" + cd "$_hgrepo-build" # # BUILD HERE diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index 3271361..4a4f6cb 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -45,9 +45,9 @@ build() { msg "SVN checkout done or connection timeout" msg "Starting build..." - rm -rf "$srcdir/$_svnmod-build" - cp -r "$srcdir/$_svnmod" "$srcdir/$_svnmod-build" - cd "$srcdir/$_svnmod-build" + rm -rf "$_svnmod-build" + cp -r "$_svnmod" "$_svnmod-build" + cd "$_svnmod-build" # # BUILD HERE -- 1.7.7.2

Linus Arver

6:56 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 6/7] vcs prototypes: consistent coding style

Some vcs prototypes do cd repo && update while others do it like cd repo update to update an existing repo. It makes sense to have them all do it the first way (there's nothing wrong with it, and it has better form). We also check for the (hidden) version control directory in the if-statement for consisteny. Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 2 +- prototypes/PKGBUILD-cvs.proto | 3 +-- prototypes/PKGBUILD-darcs.proto | 3 +-- prototypes/PKGBUILD-git.proto | 2 +- prototypes/PKGBUILD-hg.proto | 5 ++--- prototypes/PKGBUILD-svn.proto | 2 +- 6 files changed, 7 insertions(+), 10 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index 3f6d2e9..887d97b 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -34,7 +34,7 @@ build() { cd "$srcdir" msg "Connecting to $_bzrtrunk ..." - if [[ -d "$_bzrmod" ]]; then + if [[ -d "$_bzrmod/.bzr" ]]; then cd "$_bzrmod" && bzr --no-plugins pull "$_bzrtrunk" -r "$pkgver" msg "The local files are updated." cd .. diff --git a/prototypes/PKGBUILD-cvs.proto b/prototypes/PKGBUILD-cvs.proto index 0ddd91d..1d37621 100644 --- a/prototypes/PKGBUILD-cvs.proto +++ b/prototypes/PKGBUILD-cvs.proto @@ -35,8 +35,7 @@ build() { msg "Connecting to $_cvsmod ..." if [[ -d "$_cvsmod/CVS" ]]; then - cd "$_cvsmod" - cvs -z3 update -d + cd "$_cvsmod" && cvs -z3 update -d msg "The local files are updated." cd .. else diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index 1103588..5da47e1 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -35,8 +35,7 @@ build() { msg "Connecting to $_darcstrunk ..." if [[ -d "$_darcsmod/_darcs" ]]; then - cd "$_darcsmod" - darcs pull -a "$_darcstrunk/$_darcsmod" + cd "$_darcsmod" && darcs pull -a "$_darcstrunk/$_darcsmod" msg "The local files are updated." cd .. else diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index a2f2f40..b8f5de7 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -34,7 +34,7 @@ build() { cd "$srcdir" msg "Connecting to $_gitroot ..." - if [[ -d "$_gitname" ]]; then + if [[ -d "$_gitname/.git" ]]; then cd "$_gitname" git clean -dxf git reset --hard diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index b6610fa..27883f6 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -34,9 +34,8 @@ build() { cd "$srcdir" msg "Connecting to $_hgroot ..." - if [[ -d "$_hgrepo" ]]; then - cd "$_hgrepo" - hg pull -u + if [[ -d "$_hgrepo/.hg" ]]; then + cd "$_hgrepo" && hg pull -u msg "The local files are updated." cd .. else diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index 4a4f6cb..9cb3426 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -35,7 +35,7 @@ build() { msg "Connecting to $_svntrunk ..." if [[ -d "$_svnmod/.svn" ]]; then - (cd "$_svnmod" && svn up -r "$pkgver") + cd "$_svnmod" && svn up -r "$pkgver" msg "The local files are updated." cd .. else -- 1.7.7.2

Sebastian Schwarz

3:09 p.m.

New subject: [arch-projects] [ABS] [PATCH v3 6/7] vcs prototypes: consistent coding style

On 2011-11-08 at 21:56 -0800, Linus Arver wrote:

...

Some vcs prototypes do

cd repo && update

while others do it like

cd repo update

to update an existing repo. It makes sense to have them all do it the first way (there's nothing wrong with it, and it has better form).

I agree with the consistency issue, but not with using the first way. makepkg is executed with `/bin/bash -e`[1]. Whenever a command returns a non zero exit status makepkg exits immediately. See bash(1) or set(1p). Thus conditional checks on single commands with && and || are seldomly required[2]. Even in the second case the "update" step is never reached. I personally find the second form easier to read, as one does not have to think about why the && was mixed in, and if this makes sense. [1]: http://projects.archlinux.org/pacman.git/commit/?id=b69edc1c3532816576198995... [2]: http://projects.archlinux.org/pacman.git/commit/?id=2710b256cc260db6a0805c83...

Linus Arver

10 Nov 10 Nov

5:10 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 6/7] vcs prototypes: consistent coding style

On Wed, Nov 09, 2011 at 03:09:09PM +0100, Sebastian Schwarz wrote:

...

On 2011-11-08 at 21:56 -0800, Linus Arver wrote:

...
Some vcs prototypes do

cd repo && update

while others do it like

cd repo update

to update an existing repo. It makes sense to have them all do it the first way (there's nothing wrong with it, and it has better form).

I agree with the consistency issue, but not with using the first way. makepkg is executed with `/bin/bash -e`[1]. Whenever a command returns a non zero exit status makepkg exits immediately. See bash(1) or set(1p). Thus conditional checks on single commands with && and || are seldomly required[2]. Even in the second case the "update" step is never reached.

I personally find the second form easier to read, as one does not have to think about why the && was mixed in, and if this makes sense.

[1]: http://projects.archlinux.org/pacman.git/commit/?id=b69edc1c3532816576198995... [2]: http://projects.archlinux.org/pacman.git/commit/?id=2710b256cc260db6a0805c83...

Hmm, I did not realize that makepkg behaved in that manner. This changes the whole underlying assumption about what is "good form". I've verified the behavior with a quick test in a sample PKGBUILD of mine; indeed, if "cd repo" itself fails, makepkg will abort the build() function. I will revert back to the second form, as in the first version of the patch series. -Linus

lolilolicon

5:29 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 6/7] vcs prototypes: consistent coding style

On Wed, Nov 9, 2011 at 10:09 PM, Sebastian Schwarz wrote:

...

On 2011-11-08 at 21:56 -0800, Linus Arver wrote:

...
Some vcs prototypes do

cd repo && update

while others do it like

cd repo update

to update an existing repo. It makes sense to have them all do it the first way (there's nothing wrong with it, and it has better form).

I agree with the consistency issue, but not with using the first way. makepkg is executed with `/bin/bash -e`[1]. Whenever a command returns a non zero exit status makepkg exits immediately. See bash(1) or set(1p). Thus conditional checks on single commands with && and || are seldomly required[2].

Actually, with `set -e', the second form is safer. In the first form, if `cd repo' fails, bash will NOT abort. This is the relevant part in bash(1) on `set -e': The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test following the if or elif reserved words, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted with !. So yes, the `&&' form is totally busted.

...

Even in the second case the "update" step is never reached.

I personally find the second form easier to read, as one does not have to think about why the && was mixed in, and if this makes sense.

[1]: http://projects.archlinux.org/pacman.git/commit/?id=b69edc1c3532816576198995... [2]: http://projects.archlinux.org/pacman.git/commit/?id=2710b256cc260db6a0805c83...

Linus Arver

9 Nov 9 Nov

6:56 a.m.

New subject: [arch-projects] [ABS] [PATCH v3 7/7] vcs prototypes: more efficient temp build directories

The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on. We exclude this folder when creating a temporary build directory to save time and space. Since commit 0e79802c0ac8453376d8c0f99629f5a3b499f571 in pacman includes "shopt -s extglob" in pacman/scripts/makepkg.sh.in, we can use the simple "!(foo)" syntax. CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems that I am aware of. Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 3 ++- prototypes/PKGBUILD-cvs.proto | 2 +- prototypes/PKGBUILD-darcs.proto | 3 ++- prototypes/PKGBUILD-hg.proto | 3 ++- prototypes/PKGBUILD-svn.proto | 2 +- 5 files changed, 8 insertions(+), 5 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index 887d97b..8011314 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -46,7 +46,8 @@ build() { msg "Starting build..." rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" + mkdir "$_bzrmod-build" + cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build" cd "$_bzrmod-build" # diff --git a/prototypes/PKGBUILD-cvs.proto b/prototypes/PKGBUILD-cvs.proto index 1d37621..269dbfc 100644 --- a/prototypes/PKGBUILD-cvs.proto +++ b/prototypes/PKGBUILD-cvs.proto @@ -46,7 +46,7 @@ build() { msg "Starting build..." rm -rf "$_cvsmod-build" - cp -r "$_cvsmod" "$_cvsmod-build" + rsync -a --exclude=CVS "$_cvsmod"/* "$_cvsmod-build" cd "$_cvsmod-build" # diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index 5da47e1..8d6909d 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -46,7 +46,8 @@ build() { msg "Starting build..." rm -rf "$_darcsmod-build" - cp -r "$_darcsmod" "$_darcsmod-build" + mkdir "$_darcsmod-build" + cp -a "$_darcsmod"/!(_darcs) "$_darcsmod-build" cd "$_darcsmod-build" # diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index 27883f6..5aeb4f9 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -46,7 +46,8 @@ build() { msg "Starting build..." rm -rf "$_hgrepo-build" - cp -r "$_hgrepo" "$_hgrepo-build" + mkdir "$_hgrepo-build" + cp -a "$_hgrepo"/!(.hg) "$_hgrepo-build" cd "$_hgrepo-build" # diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index 9cb3426..4f2efde 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -46,7 +46,7 @@ build() { msg "Starting build..." rm -rf "$_svnmod-build" - cp -r "$_svnmod" "$_svnmod-build" + rsync -a --exclude=.svn "$_svnmod"/* "$_svnmod-build" cd "$_svnmod-build" # -- 1.7.7.2

Linus Arver

12 Nov 12 Nov

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 0/7] vcs prototype cleanups and some git-specific changes

Hello all, This is the fourth (and final?) iteration of my patch series. Changes since v3: * The "vcs prototypes: consistent coding style" patch has been changed. See http://mailman.archlinux.org/pipermail/arch-projects/2011-November/002098.ht.... * The "vcs prototypes: more efficient temp build directories" patch had an incorrect commit message re: cvs/svn prototypes. This has been fixed. Linus Arver (7): vcs prototypes: consistent $PWD after checkout vcs prototypes: typo/stylistic fixes git prototype: on initial clones, perform a shallow clone git prototype: remove temp build directory vcs prototypes: simplify code vcs prototypes: consistent coding style vcs prototypes: more efficient temp build directories prototypes/PKGBUILD-bzr.proto | 17 ++++++++++------- prototypes/PKGBUILD-cvs.proto | 13 +++++++------ prototypes/PKGBUILD-darcs.proto | 17 +++++++++-------- prototypes/PKGBUILD-git.proto | 22 +++++++++++----------- prototypes/PKGBUILD-hg.proto | 14 ++++++++------ prototypes/PKGBUILD-svn.proto | 15 +++++++++------ 6 files changed, 54 insertions(+), 44 deletions(-) -- 1.7.7.3

Linus Arver

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 1/7] vcs prototypes: consistent $PWD after checkout

Linus Arver

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 2/7] vcs prototypes: typo/stylistic fixes

Make messages uniform. Also, stop calling repositories "server", and display the repository URL for more transparency. Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 4 ++-- prototypes/PKGBUILD-cvs.proto | 5 +++-- prototypes/PKGBUILD-darcs.proto | 10 +++++----- prototypes/PKGBUILD-git.proto | 6 +++--- prototypes/PKGBUILD-hg.proto | 4 ++-- prototypes/PKGBUILD-svn.proto | 5 +++-- 6 files changed, 18 insertions(+), 16 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index bca92f1..b412247 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -32,7 +32,7 @@ _bzrmod=MODNAME build() { cd "$srcdir" - msg "Connecting to Bazaar server...." + msg "Connecting to $_bzrtrunk ..." if [[ -d "$_bzrmod" ]]; then cd "$_bzrmod" && bzr --no-plugins pull "$_bzrtrunk" -r "$pkgver" @@ -42,7 +42,7 @@ build() { bzr --no-plugins branch "$_bzrtrunk" "$_bzrmod" -q -r "$pkgver" fi - msg "Bazaar checkout done or server timeout" + msg "Bazaar checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_bzrmod-build" diff --git a/prototypes/PKGBUILD-cvs.proto b/prototypes/PKGBUILD-cvs.proto index 5c883bc..a987437 100644 --- a/prototypes/PKGBUILD-cvs.proto +++ b/prototypes/PKGBUILD-cvs.proto @@ -32,17 +32,18 @@ _cvsmod=MODNAME build() { cd "$srcdir" - msg "Connecting to $_cvsmod.sourceforge.net CVS server...." + msg "Connecting to $_cvsmod ..." if [[ -d "$_cvsmod/CVS" ]]; then cd "$_cvsmod" cvs -z3 update -d + msg "The local files are updated." cd .. else cvs -z3 -d "$_cvsroot" co -D "$pkgver" -f "$_cvsmod" fi - msg "CVS checkout done or server timeout" + msg "CVS checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_cvsmod-build" diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index 28b0c32..8dd0b48 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -32,24 +32,24 @@ _darcsmod=MODNAME build() { cd "$srcdir" - msg "Checking for previous build...." + msg "Connecting to $_darcstrunk ..." if [[ -d "$_darcsmod/_darcs" ]]; then - msg "Retrieving missing patches" cd "$_darcsmod" darcs pull -a "$_darcstrunk/$_darcsmod" + msg "The local files are updated." cd .. else - msg "Retrieving complete sources" darcs get --partial --set-scripts-executable "$_darcstrunk/$_darcsmod" fi + msg "Darcs checkout done or connection timeout" + msg "Starting build..." + rm -rf "$srcdir/$_darcsmod-build" cp -r "$srcdir/$_darcsmod" "$srcdir/$_darcsmod-build" cd "$srcdir/$_darcsmod-build" - msg "Starting build..." - # # BUILD HERE # diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index 78f3992..dd3f76e 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -28,11 +28,11 @@ noextract=() md5sums=() #generate with 'makepkg -g' _gitroot=GITURL -_gitname=MODENAME +_gitname=REPONAME build() { cd "$srcdir" - msg "Connecting to GIT server...." + msg "Connecting to $_gitroot ..." if [[ -d "$_gitname" ]]; then cd "$_gitname" && git pull origin @@ -42,7 +42,7 @@ build() { git clone "$_gitroot" "$_gitname" fi - msg "GIT checkout done or server timeout" + msg "GIT checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_gitname-build" diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index e583142..38713eb 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -32,7 +32,7 @@ _hgrepo=REPONAME build() { cd "$srcdir" - msg "Connecting to Mercurial server...." + msg "Connecting to $_hgroot ..." if [[ -d "$_hgrepo" ]]; then cd "$_hgrepo" @@ -43,7 +43,7 @@ build() { hg clone "$_hgroot" "$_hgrepo" fi - msg "Mercurial checkout done or server timeout" + msg "Mercurial checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_hgrepo-build" diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index 2afb533..3271361 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -32,16 +32,17 @@ _svnmod=MODNAME build() { cd "$srcdir" - msg "Connecting to SVN server...." + msg "Connecting to $_svntrunk ..." if [[ -d "$_svnmod/.svn" ]]; then (cd "$_svnmod" && svn up -r "$pkgver") + msg "The local files are updated." cd .. else svn co "$_svntrunk" --config-dir ./ -r "$pkgver" "$_svnmod" fi - msg "SVN checkout done or server timeout" + msg "SVN checkout done or connection timeout" msg "Starting build..." rm -rf "$srcdir/$_svnmod-build" -- 1.7.7.3

Linus Arver

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 3/7] git prototype: on initial clones, perform a shallow clone

Linus Arver

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 4/7] git prototype: remove temp build directory

Git comes with commands that are specifically designed to return the working directory to a "pristine" state. We make use of these commands to avoid the expensive operation of creating a temporary build directory. Having a single directory to pull in upstream code and build from greatly simplifies the code, and as a bonus, saves disk space/time! Thanks to Jesse Young for suggesting this idea. [1] [1]: http://mailman.archlinux.org/pipermail/arch-projects/2011-November/002052.ht... Signed-off-by: Linus Arver --- prototypes/PKGBUILD-git.proto | 13 ++++++------- 1 files changed, 6 insertions(+), 7 deletions(-) diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index 7d9bb90..a2f2f40 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -35,20 +35,19 @@ build() { msg "Connecting to $_gitroot ..." if [[ -d "$_gitname" ]]; then - cd "$_gitname" && git pull origin + cd "$_gitname" + git clean -dxf + git reset --hard + git pull origin msg "The local files are updated." - cd .. else git clone --depth 1 "$_gitroot" "$_gitname" + cd "$_gitname" fi msg "GIT checkout done or connection timeout" msg "Starting build..." - rm -rf "$srcdir/$_gitname-build" - git clone "$srcdir/$_gitname" "$srcdir/$_gitname-build" - cd "$srcdir/$_gitname-build" - # # BUILD HERE # @@ -58,7 +57,7 @@ build() { } package() { - cd "$srcdir/$_gitname-build" + cd "$srcdir/$_gitname" make DESTDIR="$pkgdir/" install } -- 1.7.7.3

Linus Arver

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 5/7] vcs prototypes: simplify code

Linus Arver

3:11 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 5/7] vcs prototypes: simplify code

On Fri, Nov 11, 2011 at 05:50:21PM -0800, Linus Arver wrote:

...

Since the $PWD after the update/initial clone is always "$srcdir", we don't need to reference it later on.

Signed-off-by: Linus Arver <linusarver at gmail.com> Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 6 +++--- prototypes/PKGBUILD-cvs.proto | 6 +++--- prototypes/PKGBUILD-darcs.proto | 6 +++---

Embarrasing --- please fix this duplicate signed-off-by line in the commit message after merging with a rebase. I don't think posting an entirely new patch series just for this typo is a sane thing to do... -Linus

Linus Arver

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 6/7] vcs prototypes: consistent coding style

Some vcs prototypes do cd repo && update while others do it like cd repo update to update an existing repo. It makes sense to have them all do it the second way, because makepkg runs with `/bin/bash -e` [1]. The manpage for bash(1) states regarding the "-e" option: "The shell does not exit if the command that fails is ... part of any command executed in a && or || list except the command following the final && or || ..." I.e., if "cd repo" fails in "cd repo && update", the shell will not exit! The second form avoids this pitfall and is slightly easier to read, especially for the longer "update" commands. Thanks to Sebastian Schwarz [2] and lolilolicon [3] for the pointers. Lastly, we also check for the (hidden) version control directory in the if-statement for consisteny. [1]: http://projects.archlinux.org/pacman.git/commit/?id=b69edc1c3532816576198995... [2]: http://mailman.archlinux.org/pipermail/arch-projects/2011-November/002096.ht... [3]: http://mailman.archlinux.org/pipermail/arch-projects/2011-November/002099.ht... Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 5 +++-- prototypes/PKGBUILD-git.proto | 2 +- prototypes/PKGBUILD-hg.proto | 2 +- prototypes/PKGBUILD-svn.proto | 3 ++- 4 files changed, 7 insertions(+), 5 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index 3f6d2e9..287486d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -34,8 +34,9 @@ build() { cd "$srcdir" msg "Connecting to $_bzrtrunk ..." - if [[ -d "$_bzrmod" ]]; then - cd "$_bzrmod" && bzr --no-plugins pull "$_bzrtrunk" -r "$pkgver" + if [[ -d "$_bzrmod/.bzr" ]]; then + cd "$_bzrmod" + bzr --no-plugins pull "$_bzrtrunk" -r "$pkgver" msg "The local files are updated." cd .. else diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index a2f2f40..b8f5de7 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -34,7 +34,7 @@ build() { cd "$srcdir" msg "Connecting to $_gitroot ..." - if [[ -d "$_gitname" ]]; then + if [[ -d "$_gitname/.git" ]]; then cd "$_gitname" git clean -dxf git reset --hard diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index b6610fa..e2ddca1 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -34,7 +34,7 @@ build() { cd "$srcdir" msg "Connecting to $_hgroot ..." - if [[ -d "$_hgrepo" ]]; then + if [[ -d "$_hgrepo/.hg" ]]; then cd "$_hgrepo" hg pull -u msg "The local files are updated." diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index 4a4f6cb..42f7f60 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -35,7 +35,8 @@ build() { msg "Connecting to $_svntrunk ..." if [[ -d "$_svnmod/.svn" ]]; then - (cd "$_svnmod" && svn up -r "$pkgver") + cd "$_svnmod" + svn up -r "$pkgver" msg "The local files are updated." cd .. else -- 1.7.7.3

Linus Arver

2:50 a.m.

New subject: [arch-projects] [ABS] [PATCH v4 7/7] vcs prototypes: more efficient temp build directories

The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on. We exclude this folder when creating a temporary build directory to save time and space. Since commit 0e79802c0ac8453376d8c0f99629f5a3b499f571 in pacman includes "shopt -s extglob" in pacman/scripts/makepkg.sh.in, we can use the simple "!(foo)" syntax. Thanks to Dave Reisner for pointing this out. [1] CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so we use rsync. Most, if not all, Arch systems should have rsync installed, so this should be safe. [1]: http://mailman.archlinux.org/pipermail/arch-projects/2011-November/002033.ht... Signed-off-by: Linus Arver --- prototypes/PKGBUILD-bzr.proto | 3 ++- prototypes/PKGBUILD-cvs.proto | 2 +- prototypes/PKGBUILD-darcs.proto | 3 ++- prototypes/PKGBUILD-hg.proto | 3 ++- prototypes/PKGBUILD-svn.proto | 2 +- 5 files changed, 8 insertions(+), 5 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index 287486d..745414c 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,7 +47,8 @@ build() { msg "Starting build..." rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" + mkdir "$_bzrmod-build" + cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build" cd "$_bzrmod-build" # diff --git a/prototypes/PKGBUILD-cvs.proto b/prototypes/PKGBUILD-cvs.proto index 0ddd91d..4cf87cc 100644 --- a/prototypes/PKGBUILD-cvs.proto +++ b/prototypes/PKGBUILD-cvs.proto @@ -47,7 +47,7 @@ build() { msg "Starting build..." rm -rf "$_cvsmod-build" - cp -r "$_cvsmod" "$_cvsmod-build" + rsync -a --exclude=CVS "$_cvsmod"/* "$_cvsmod-build" cd "$_cvsmod-build" # diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index 1103588..4ab19f5 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -47,7 +47,8 @@ build() { msg "Starting build..." rm -rf "$_darcsmod-build" - cp -r "$_darcsmod" "$_darcsmod-build" + mkdir "$_darcsmod-build" + cp -a "$_darcsmod"/!(_darcs) "$_darcsmod-build" cd "$_darcsmod-build" # diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index e2ddca1..9168deb 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -47,7 +47,8 @@ build() { msg "Starting build..." rm -rf "$_hgrepo-build" - cp -r "$_hgrepo" "$_hgrepo-build" + mkdir "$_hgrepo-build" + cp -a "$_hgrepo"/!(.hg) "$_hgrepo-build" cd "$_hgrepo-build" # diff --git a/prototypes/PKGBUILD-svn.proto b/prototypes/PKGBUILD-svn.proto index 42f7f60..2f6438a 100644 --- a/prototypes/PKGBUILD-svn.proto +++ b/prototypes/PKGBUILD-svn.proto @@ -47,7 +47,7 @@ build() { msg "Starting build..." rm -rf "$_svnmod-build" - cp -r "$_svnmod" "$_svnmod-build" + rsync -a --exclude=.svn "$_svnmod"/* "$_svnmod-build" cd "$_svnmod-build" # -- 1.7.7.3

4554

Age (days ago)

4557

Last active (days ago)

List overview

Download

24 comments

4 participants

participants (4)

C Anthony Risinger
Linus Arver
lolilolicon
Sebastian Schwarz