[arch-projects] [ABS] [PATCH 5/6] vcs prototypes: more efficient temp build directories
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on. We exclude this folder when creating a temporary build directory to save time and space. CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems. Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-) diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..." rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build" + cd ../"$_bzrmod-build" # # BUILD HERE diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index a55a346..affd2c8 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -45,8 +45,10 @@ build() { fi rm -rf "$_darcsmod-build" - cp -r "$_darcsmod" "$_darcsmod-build" - cd "$_darcsmod-build" + mkdir $_darcsmod-build + cd $_darcsmod + ls -A | grep -v _darcs | xargs -d '\n' cp -r -t ../"$_darcsmod-build" + cd ../"$_darcsmod-build" msg "Starting build..." diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index 88bc76b..75edb8b 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..." rm -rf "$_gitname-build" - git clone "$_gitname" "$_gitname-build" - cd "$_gitname-build" + mkdir $_gitname-build + cd $_gitname + ls -A | grep -v .git | xargs -d '\n' cp -r -t ../"$_gitname-build" + cd ../"$_gitname-build" # # BUILD HERE diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index 6f54af8..8927fbf 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..." rm -rf "$_hgrepo-build" - cp -r "$_hgrepo" "$_hgrepo-build" - cd "$_hgrepo-build" + mkdir $_hgrepo-build + cd $_hgrepo + ls -A | grep -v .hg | xargs -d '\n' cp -r -t ../"$_hgrepo-build" + cd ../"$_hgrepo-build" # # BUILD HERE -- 1.7.7.1
On Wed, Nov 02, 2011 at 05:52:37PM -0700, Linus Arver wrote:
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on.
We exclude this folder when creating a temporary build directory to save time and space.
CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems.
Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build"
Oh hell no. We are not parsing ls here. This can be done without the need for grep and findutils as well, e.g. mkdir "$_bzrmod-build" cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build" Note the proper quoting here, too, that you've stripped.
+ cd ../"$_bzrmod-build"
# # BUILD HERE diff --git a/prototypes/PKGBUILD-darcs.proto b/prototypes/PKGBUILD-darcs.proto index a55a346..affd2c8 100644 --- a/prototypes/PKGBUILD-darcs.proto +++ b/prototypes/PKGBUILD-darcs.proto @@ -45,8 +45,10 @@ build() { fi
rm -rf "$_darcsmod-build" - cp -r "$_darcsmod" "$_darcsmod-build" - cd "$_darcsmod-build" + mkdir $_darcsmod-build + cd $_darcsmod + ls -A | grep -v _darcs | xargs -d '\n' cp -r -t ../"$_darcsmod-build" + cd ../"$_darcsmod-build"
msg "Starting build..."
diff --git a/prototypes/PKGBUILD-git.proto b/prototypes/PKGBUILD-git.proto index 88bc76b..75edb8b 100644 --- a/prototypes/PKGBUILD-git.proto +++ b/prototypes/PKGBUILD-git.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_gitname-build" - git clone "$_gitname" "$_gitname-build" - cd "$_gitname-build" + mkdir $_gitname-build + cd $_gitname + ls -A | grep -v .git | xargs -d '\n' cp -r -t ../"$_gitname-build" + cd ../"$_gitname-build"
# # BUILD HERE diff --git a/prototypes/PKGBUILD-hg.proto b/prototypes/PKGBUILD-hg.proto index 6f54af8..8927fbf 100644 --- a/prototypes/PKGBUILD-hg.proto +++ b/prototypes/PKGBUILD-hg.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_hgrepo-build" - cp -r "$_hgrepo" "$_hgrepo-build" - cd "$_hgrepo-build" + mkdir $_hgrepo-build + cd $_hgrepo + ls -A | grep -v .hg | xargs -d '\n' cp -r -t ../"$_hgrepo-build" + cd ../"$_hgrepo-build"
# # BUILD HERE -- 1.7.7.1
On Wed, Nov 02, 2011 at 09:03:37PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 05:52:37PM -0700, Linus Arver wrote:
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on.
We exclude this folder when creating a temporary build directory to save time and space.
CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems.
Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build"
Oh hell no. We are not parsing ls here. This can be done without the need for grep and findutils as well, e.g.
mkdir "$_bzrmod-build" cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build"
Note the proper quoting here, too, that you've stripped.
Ah, thanks for that! I'll change it to your version.
On Wed, Nov 02, 2011 at 09:03:37PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 05:52:37PM -0700, Linus Arver wrote:
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on.
We exclude this folder when creating a temporary build directory to save time and space.
CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems.
Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build"
Oh hell no. We are not parsing ls here. This can be done without the need for grep and findutils as well, e.g.
mkdir "$_bzrmod-build" cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build"
Note the proper quoting here, too, that you've stripped.
Hmm, I just did some tests, but the "!(.bzr)" syntax that you used gives me a parse error.
On Wed, Nov 02, 2011 at 07:13:48PM -0700, Linus Arver wrote:
On Wed, Nov 02, 2011 at 09:03:37PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 05:52:37PM -0700, Linus Arver wrote:
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on.
We exclude this folder when creating a temporary build directory to save time and space.
CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems.
Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build"
Oh hell no. We are not parsing ls here. This can be done without the need for grep and findutils as well, e.g.
mkdir "$_bzrmod-build" cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build"
Note the proper quoting here, too, that you've stripped.
Hmm, I just did some tests, but the "!(.bzr)" syntax that you used gives me a parse error.
Yes, it's a syntax error when extglob isn't enabled. We provide this by default in makepkg, so PKGBUILDs will inherit it.
On Wed, Nov 02, 2011 at 10:21:04PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 07:13:48PM -0700, Linus Arver wrote:
On Wed, Nov 02, 2011 at 09:03:37PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 05:52:37PM -0700, Linus Arver wrote:
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on.
We exclude this folder when creating a temporary build directory to save time and space.
CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems.
Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build"
Oh hell no. We are not parsing ls here. This can be done without the need for grep and findutils as well, e.g.
mkdir "$_bzrmod-build" cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build"
Note the proper quoting here, too, that you've stripped.
Hmm, I just did some tests, but the "!(.bzr)" syntax that you used gives me a parse error.
Yes, it's a syntax error when extglob isn't enabled. We provide this by default in makepkg, so PKGBUILDs will inherit it.
Strange -- the test I did was from inside a PKGBUILD (and I just did a "pacman --Syu" an hour ago). Perhaps there's a bug in pacman/scripts/makepkg.sh.in. I skimmed the code in there and there's a bit of shell option wizardry going on at different points. I guess I'll file a bug report for pacman... Meanwhile, I'll use the "!(foo)" syntax, which is certainly cleaner and much more elegant.
On Wed, Nov 02, 2011 at 07:35:40PM -0700, Linus Arver wrote:
On Wed, Nov 02, 2011 at 10:21:04PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 07:13:48PM -0700, Linus Arver wrote:
On Wed, Nov 02, 2011 at 09:03:37PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 05:52:37PM -0700, Linus Arver wrote:
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on.
We exclude this folder when creating a temporary build directory to save time and space.
CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems.
Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build"
Oh hell no. We are not parsing ls here. This can be done without the need for grep and findutils as well, e.g.
mkdir "$_bzrmod-build" cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build"
Note the proper quoting here, too, that you've stripped.
Hmm, I just did some tests, but the "!(.bzr)" syntax that you used gives me a parse error.
Yes, it's a syntax error when extglob isn't enabled. We provide this by default in makepkg, so PKGBUILDs will inherit it.
Strange -- the test I did was from inside a PKGBUILD (and I just did a "pacman --Syu" an hour ago).
Sorry, I should have been clearer. It's in testing. Line 84 if you're looking at makepkg.sh.in in the git repo.
Perhaps there's a bug in pacman/scripts/makepkg.sh.in. I skimmed the code in there and there's a bit of shell option wizardry going on at different points.
I guess I'll file a bug report for pacman... Meanwhile, I'll use the
No. There is no bug.
"!(foo)" syntax, which is certainly cleaner and much more elegant.
On Wed, Nov 02, 2011 at 10:45:05PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 07:35:40PM -0700, Linus Arver wrote:
On Wed, Nov 02, 2011 at 10:21:04PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 07:13:48PM -0700, Linus Arver wrote:
On Wed, Nov 02, 2011 at 09:03:37PM -0400, Dave Reisner wrote:
On Wed, Nov 02, 2011 at 05:52:37PM -0700, Linus Arver wrote:
The bzr, darcs, git, and hg version control systems use a single internal folder at the root to store all VCS-related data (commits, history, etc.). This folder can get quite large (hundreds of MiB) for big projects and continues to grow as the project lives on.
We exclude this folder when creating a temporary build directory to save time and space.
CVS and SVN pollute the source repo with "CVS" and ".svn" directories recursively for every single directory, so there is no simple one-liner solution to exclude the VCS data for these systems.
Signed-off-by: Linus Arver <linusarver@gmail.com> --- prototypes/PKGBUILD-bzr.proto | 6 ++++-- prototypes/PKGBUILD-darcs.proto | 6 ++++-- prototypes/PKGBUILD-git.proto | 6 ++++-- prototypes/PKGBUILD-hg.proto | 6 ++++-- 4 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/prototypes/PKGBUILD-bzr.proto b/prototypes/PKGBUILD-bzr.proto index a01eea8..ef7187d 100644 --- a/prototypes/PKGBUILD-bzr.proto +++ b/prototypes/PKGBUILD-bzr.proto @@ -47,8 +47,10 @@ build() { msg "Starting build..."
rm -rf "$_bzrmod-build" - cp -r "$_bzrmod" "$_bzrmod-build" - cd "$_bzrmod-build" + mkdir $_bzrmod-build + cd $_bzrmod + ls -A | grep -v .bzr | xargs -d '\n' cp -r -t ../"$_bzrmod-build"
Oh hell no. We are not parsing ls here. This can be done without the need for grep and findutils as well, e.g.
mkdir "$_bzrmod-build" cp -a "$_bzrmod"/!(.bzr) "$_bzrmod-build"
Note the proper quoting here, too, that you've stripped.
Hmm, I just did some tests, but the "!(.bzr)" syntax that you used gives me a parse error.
Yes, it's a syntax error when extglob isn't enabled. We provide this by default in makepkg, so PKGBUILDs will inherit it.
Strange -- the test I did was from inside a PKGBUILD (and I just did a "pacman --Syu" an hour ago).
Sorry, I should have been clearer. It's in testing. Line 84 if you're looking at makepkg.sh.in in the git repo.
Perhaps there's a bug in pacman/scripts/makepkg.sh.in. I skimmed the code in there and there's a bit of shell option wizardry going on at different points.
I guess I'll file a bug report for pacman... Meanwhile, I'll use the
No. There is no bug.
"!(foo)" syntax, which is certainly cleaner and much more elegant.
Ah, that makes sense. Yes, I was looking at line 84 myself and scratching my head. Thanks. -Linus
participants (2)
-
Dave Reisner
-
Linus Arver