July 2012 - Pacman-dev - lists.archlinux.org

[pacman-dev] Help adding support for BZR, CVZ, DARCS to makepkg
by Allan McRae 23 Jul '12

23 Jul '12

On my VCS branch, makepkg now supports git, svn and hg. That accounts for ~91.5% of all VCS PKGBUILDs in the AUR. Here are rough AUR package counts for those that are interested: git: 4002 svn: 1771 hg: 498 bzr: 364 cvs: 186 darcs: 29 I will write the code to add bzr, cvs and darcs support back, but I need help given I know nothing about these systems. What I will need: 1) How to checkout a repo to a given folder 2) What url "fragments" need supported (e.g. git can build from a branch, tag or commit; svn can build from a given revision). 3) How to create a working copy of the checked out repo (i.e. "copy" the primary checkout folder) and how to get it to the specified branch/tag/commit/whatever. That can be in all one step. makepkg will not support these protocols with pacman-4.1 if I don't get that information (or someone else does not provide a patch). Allan

1 0

[pacman-dev] [PATCH] makepkg: add hg url support
by Allan McRae 23 Jul '12

23 Jul '12

Supported fragments are branch, revision and tag. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- doc/PKGBUILD.5.txt | 7 ++++-- scripts/makepkg.sh.in | 69 ++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 71 insertions(+), 5 deletions(-) diff --git a/doc/PKGBUILD.5.txt b/doc/PKGBUILD.5.txt index db10041..c49007e 100644 --- a/doc/PKGBUILD.5.txt +++ b/doc/PKGBUILD.5.txt @@ -396,8 +396,8 @@ Using VCS Sources[[VCS]] ------------------------ Building a developmental version of a package using sources from a version control system (VCS) is enabled by specifying the source in the form -`source=('folder::url#fragment')`. Currently makepkg supports the `git` and `svn` -protocols. +`source=('folder::url#fragment')`. Currently makepkg supports the `git`, `hg` and +`svn` protocols. The source URL is divided into three components: @@ -421,6 +421,9 @@ The source URL is divided into three components: *git*;; branch, commit, tag + *hg*;; + branch, revision, tag + *svn*;; revision diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 4fe7f06..014ba3c 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -199,7 +199,7 @@ get_filepath() { local proto="$(get_protocol "$1")" case $proto in - git*|svn*) + git*|hg*|svn*) if [[ -d "$startdir/$file" ]]; then file="$startdir/$file" elif [[ -d "$SRCDEST/$file" ]]; then @@ -235,7 +235,7 @@ get_filename() { local proto=$(get_protocol "$netfile") case $proto in - git*|svn*) + git*|hg*|svn*) filename=${netfile##*/} filename=${filename%%#*} # fall-through @@ -450,6 +450,66 @@ download_git() { popd &>/dev/null } +download_hg() { + local netfile=$1 + + local fragment=${netfile##*#} + if [[ $fragment = "$netfile" ]]; then + unset fragment + fi + + local dir=$(get_filepath "$netfile") + [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" + + local repo=${netfile##*/} + repo=${repo%%#*} + + local url=$(get_url "$netfile") + url=${url##*hg+} + url=${url%%#*} + + if [[ ! -d "$dir" ]]; then + msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "hg" + if ! hg clone "$url" "$dir"; then + error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "hg" + plain "$(gettext "Aborting...")" + exit 1 + fi + elif (( ! HOLDVER )); then + msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "hg" + cd_safe "$dir" + if ! hg pull -u; then + # only warn on failure to allow offline builds + warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "hg" + fi + fi + + msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "hg" + pushd "$srcdir" &>/dev/null + rm -rf "${dir##*/}" + + local ref + if [[ -n $fragment ]]; then + case ${fragment%%=*} in + branch|revision|tag) + ref=('-u' "${fragment##*=}") + ;; + *) + error "$(gettext "Unrecognized reference: %s")" "${fragment}" + plain "$(gettext "Aborting...")" + exit 1 + esac + fi + + if ! hg clone $(ref[@]) "$dir" "${dir##*/}"; then + error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "hg" + plain "$(gettext "Aborting...")" + exit 1 + fi + + popd &>/dev/null +} + download_svn() { local netfile=$1 @@ -535,6 +595,9 @@ download_sources() { git*) (( GET_VCS )) && download_git "$netfile" ;; + hg*) + (( GET_VCS )) && download_hg "$netfile" + ;; svn*) (( GET_VCS )) && download_svn "$netfile" ;; @@ -909,7 +972,7 @@ generate_checksums() { proto="$(get_protocol "$netfile")" case $proto in - git*|svn*) + git*|hg*|svn*) sum="SKIP" ;; *) -- 1.7.11.2

1 0

[pacman-dev] [PATCH 1/4] doc: Document using VCS sources in a PKGBUILD
by Allan McRae 23 Jul '12

23 Jul '12

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- doc/PKGBUILD.5.txt | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/doc/PKGBUILD.5.txt b/doc/PKGBUILD.5.txt index 956179a..2ca3c98 100644 --- a/doc/PKGBUILD.5.txt +++ b/doc/PKGBUILD.5.txt @@ -109,6 +109,10 @@ It is also possible to change the name of the downloaded file, which is helpful with weird URLs and for handling multiple source files with the same name. The syntax is: `source=('filename::url')`. + +makepkg also supports building developmental versions of packages using sources +downloaded from version control systems (VCS). For more information, see +<<VCS,Using VCS Sources>> below. ++ Files in the source array with extensions `.sig`, `.sign` or `.asc` are recognized by makepkg as PGP signatures and will be automatically used to verify the integrity of the corresponding source file. @@ -383,6 +387,34 @@ The install script does not need to be specified in the source array. A template install file is available in '{pkgdatadir}' as 'proto.install' for reference with all of the available functions defined. +Using VCS Sources[[VCS]] +------------------------ +Building a developmental version of a package using sources from a version control +system (VCS) is enabled by specifying the source in the form +`source=('folder::url#fragment')`. Currently makepkg supports the `git` protocol. + +The source URL is divided into three components: + +*folder*:: + (optional) Specifies an alternate folder name for makepkg to download the VCS + source into. + +*url*:: + The url to the VCS repo. This must include the the vcs in the URL protocol for + makepkg to recognize this as a VCS source. If the protocol does not include + the VCS name, it can be added by prefixing the URL with `vcs+`. For example, + using a git repository over `http` would have a source URL in the form + `git+http://...`. + +*fragment*:: + (optional) Allows specifying a revision number or branch for makepkg to checkout + from the VCS. For example, to checkout a given revision, the source line would + have the format `source=(url#revision=123)`. The available fragments depends on + the VCS being used: + + *git*;; + branch, commit, tag + Example ------- The following is an example PKGBUILD for the 'patch' package. For more -- 1.7.11.2

1 4

[pacman-dev] [PATCH] Document that -dd is different from -d in a clear and concise way.
by Micahel J. Evans 20 Jul '12

20 Jul '12

From: "Micahel J. Evans" <mjevans1983(a)gmail.com> Signed-off-by: Micahel J. Evans <mjevans1983(a)gmail.com> --- doc/pacman.8.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt index 0ad8188..926d6e1 100644 --- a/doc/pacman.8.txt +++ b/doc/pacman.8.txt @@ -171,6 +171,8 @@ Options Transaction Options (apply to '-S', '-R' and '-U') -------------------------------------------------- +*-dd, --nodeps --nodeps*:: + Skips all dependency checks. *-d, \--nodeps*:: Skips dependency version checks. Package names are still checked. Normally, pacman will always check a package's dependency fields to ensure that all -- 1.7.11.2

3 3

[pacman-dev] [PATCH] Document that -dd is different from -d in a clear and concise way.
by Micahel J. Evans 20 Jul '12

20 Jul '12

From: "Micahel J. Evans" <mjevans1983(a)gmail.com> Signed-off-by: Micahel J. Evans <mjevans1983(a)gmail.com> --- doc/pacman.8.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt index 0ad8188..926d6e1 100644 --- a/doc/pacman.8.txt +++ b/doc/pacman.8.txt @@ -171,6 +171,8 @@ Options Transaction Options (apply to '-S', '-R' and '-U') -------------------------------------------------- +*-dd, --nodeps --nodeps*:: + Skips all dependency checks. *-d, \--nodeps*:: Skips dependency version checks. Package names are still checked. Normally, pacman will always check a package's dependency fields to ensure that all -- 1.7.11.2

1 0

[pacman-dev] [PATCH] util: fix line length calc in _alpm_archive_fgets
by Dave Reisner 20 Jul '12

20 Jul '12

74274b5dc347ba70 which added the real_line_size to the buffer struct didn't properly account for what happens when archive_fgets has to loop more than once to find the end of a line. In most cases, this isn't a problem, but could potentially cause a longer line such as PGP signature to be improperly read. This patch fixes the oversight and focuses on only calculating the line length when we hit the end of line marker. The effective length is then calculated via pointer arithmetic as: (start_of_last_read + read_length) - start_of_line Signed-off-by: Dave Reisner <dreisner(a)archlinux.org> --- Allan noticed that PHP was reporting an invalid sig and bisected it back to me, (of course). This was lucky enough to only affect about .1% of the sync DB, and mostly the PGPSIG field, naturally due to it being the largest field in the DB by a wide margin. lib/libalpm/util.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/libalpm/util.c b/lib/libalpm/util.c index c2b5d44..721125a 100644 --- a/lib/libalpm/util.c +++ b/lib/libalpm/util.c @@ -1011,15 +1011,16 @@ int _alpm_archive_fgets(struct archive *a, struct archive_read_buffer *b) } if(eol) { - size_t len = b->real_line_size = (size_t)(eol - b->block_offset); + size_t len = (size_t)(eol - b->block_offset); memcpy(b->line_offset, b->block_offset, len); b->line_offset[len] = '\0'; b->block_offset = eol + 1; + b->real_line_size = b->line_offset + len - b->line; /* this is the main return point; from here you can read b->line */ return ARCHIVE_OK; } else { /* we've looked through the whole block but no newline, copy it */ - size_t len = b->real_line_size = (size_t)(b->block + b->block_size - b->block_offset); + size_t len = (size_t)(b->block + b->block_size - b->block_offset); memcpy(b->line_offset, b->block_offset, len); b->line_offset += len; b->block_offset = b->block + b->block_size; -- 1.7.11.2

3 4

[pacman-dev] [PATCH 1/3] Make function for sorting filelists available to library
by Allan McRae 18 Jul '12

18 Jul '12

Make files_msort available to the library as sorting a filelist will be useful for packages where the filelist changes after resolving symlinks. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- lib/libalpm/be_package.c | 49 +----------------------------------------------- lib/libalpm/package.c | 48 +++++++++++++++++++++++++++++++++++++++++++++++ lib/libalpm/package.h | 1 + 3 files changed, 50 insertions(+), 48 deletions(-) diff --git a/lib/libalpm/be_package.c b/lib/libalpm/be_package.c index dd027f5..ed71be2 100644 --- a/lib/libalpm/be_package.c +++ b/lib/libalpm/be_package.c @@ -248,53 +248,6 @@ static int parse_descfile(alpm_handle_t *handle, struct archive *a, alpm_pkg_t * return 0; } -static void files_merge(alpm_file_t a[], alpm_file_t b[], alpm_file_t c[], - size_t m, size_t n) -{ - size_t i = 0, j = 0, k = 0; - while(i < m && j < n) { - if(strcmp(a[i].name, b[j].name) < 0) { - c[k++] = a[i++]; - } else { - c[k++] = b[j++]; - } - } - while(i < m) { - c[k++] = a[i++]; - } - while(j < n) { - c[k++] = b[j++]; - } -} - -static alpm_file_t *files_msort(alpm_file_t *files, size_t n) -{ - alpm_file_t *work; - size_t blocksize = 1; - - CALLOC(work, n, sizeof(alpm_file_t), return NULL); - - for(blocksize = 1; blocksize < n; blocksize *= 2) { - size_t i, max_extent = 0; - for(i = 0; i < n - blocksize; i += 2 * blocksize) { - /* this limits our actual merge to the length of the array, since we will - * not likely be a perfect power of two. */ - size_t right_blocksize = blocksize; - if(i + blocksize * 2 > n) { - right_blocksize = n - i - blocksize; - } - files_merge(files + i, files + i + blocksize, work + i, - blocksize, right_blocksize); - max_extent = i + blocksize + right_blocksize; - } - /* ensure we only copy what we actually touched on this merge pass, - * no more, no less */ - memcpy(files, work, max_extent * sizeof(alpm_file_t)); - } - free(work); - return files; -} - /** * Validate a package. * @param handle the context handle @@ -536,7 +489,7 @@ alpm_pkg_t *_alpm_pkg_load_internal(alpm_handle_t *handle, /* "checking for conflicts" requires a sorted list, ensure that here */ _alpm_log(handle, ALPM_LOG_DEBUG, "sorting package filelist for %s\n", pkgfile); - newpkg->files.files = files_msort(newpkg->files.files, + newpkg->files.files = _alpm_files_msort(newpkg->files.files, newpkg->files.count); } newpkg->infolevel |= INFRQ_FILES; diff --git a/lib/libalpm/package.c b/lib/libalpm/package.c index 46d1473..8a7b01f 100644 --- a/lib/libalpm/package.c +++ b/lib/libalpm/package.c @@ -459,6 +459,54 @@ int _alpm_files_cmp(const void *f1, const void *f2) return strcmp(file1->name, file2->name); } + +static void files_merge(alpm_file_t a[], alpm_file_t b[], alpm_file_t c[], + size_t m, size_t n) +{ + size_t i = 0, j = 0, k = 0; + while(i < m && j < n) { + if(strcmp(a[i].name, b[j].name) < 0) { + c[k++] = a[i++]; + } else { + c[k++] = b[j++]; + } + } + while(i < m) { + c[k++] = a[i++]; + } + while(j < n) { + c[k++] = b[j++]; + } +} + +alpm_file_t *_alpm_files_msort(alpm_file_t *files, size_t n) +{ + alpm_file_t *work; + size_t blocksize = 1; + + CALLOC(work, n, sizeof(alpm_file_t), return NULL); + + for(blocksize = 1; blocksize < n; blocksize *= 2) { + size_t i, max_extent = 0; + for(i = 0; i < n - blocksize; i += 2 * blocksize) { + /* this limits our actual merge to the length of the array, since we will + * not likely be a perfect power of two. */ + size_t right_blocksize = blocksize; + if(i + blocksize * 2 > n) { + right_blocksize = n - i - blocksize; + } + files_merge(files + i, files + i + blocksize, work + i, + blocksize, right_blocksize); + max_extent = i + blocksize + right_blocksize; + } + /* ensure we only copy what we actually touched on this merge pass, + * no more, no less */ + memcpy(files, work, max_extent * sizeof(alpm_file_t)); + } + free(work); + return files; +} + alpm_pkg_t *_alpm_pkg_new(void) { alpm_pkg_t *pkg; diff --git a/lib/libalpm/package.h b/lib/libalpm/package.h index c473549..3c539ef 100644 --- a/lib/libalpm/package.h +++ b/lib/libalpm/package.h @@ -127,6 +127,7 @@ struct __alpm_pkg_t { alpm_file_t *_alpm_file_copy(alpm_file_t *dest, const alpm_file_t *src); int _alpm_files_cmp(const void *f1, const void *f2); +alpm_file_t *_alpm_files_msort(alpm_file_t *files, size_t n); alpm_pkg_t *_alpm_pkg_new(void); int _alpm_pkg_dup(alpm_pkg_t *pkg, alpm_pkg_t **new_ptr); -- 1.7.11.2

1 3

[pacman-dev] [PATCH] Provide a full filelist to the pactests that need it
by Allan McRae 17 Jul '12

17 Jul '12

It turns out when you set the filelist for a package to include "usr/lib/foo" in the pactest suite, it thinks there is only the file "usr/lib/foo" in there... No "usr/" or "usr/lib/" directory. This makes life difficult when testing code that scrolls through a filelist looking for directory entries. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- test/pacman/tests/fileconflict001.py | 8 ++++++-- test/pacman/tests/fileconflict016.py | 7 +++++-- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/test/pacman/tests/fileconflict001.py b/test/pacman/tests/fileconflict001.py index dec6151..e1371f8 100644 --- a/test/pacman/tests/fileconflict001.py +++ b/test/pacman/tests/fileconflict001.py @@ -7,11 +7,15 @@ self.addpkg2db("local", lp) p1 = pmpkg("pkg1") -p1.files = ["dir/realdir/file"] +p1.files = ["dir/", + "dir/realdir/", + "dir/realdir/file"] self.addpkg(p1) p2 = pmpkg("pkg2") -p2.files = ["dir/symdir/file"] +p2.files = ["dir/", + "dir/symdir/", + "dir/symdir/file"] self.addpkg(p2) self.args = "-U %s" % " ".join([p.filename() for p in p1, p2]) diff --git a/test/pacman/tests/fileconflict016.py b/test/pacman/tests/fileconflict016.py index 93797f6..5625984 100644 --- a/test/pacman/tests/fileconflict016.py +++ b/test/pacman/tests/fileconflict016.py @@ -7,11 +7,14 @@ self.addpkg2db("local", lp1) p1 = pmpkg("pkg1") -p1.files = ["lib/foo"] +p1.files = ["lib/", + "lib/foo"] self.addpkg2db("sync", p1) p2 = pmpkg("pkg2") -p2.files = ["usr/lib/foo"] +p2.files = ["usr/", + "usr/lib/", + "usr/lib/foo"] self.addpkg2db("sync", p2) self.args = "-S pkg1 pkg2" -- 1.7.11.2

1 0

[pacman-dev] package versions with vcs packages
by Christian Hesse 17 Jul '12

17 Jul '12

Hello everybody, hg and svn packages do set vcs revision infos for package versions. Any reason this does not apply to git, cvs and darcs? Using a date string has the drawback that two revisions compiled the same day result in packages with the same version... A patch for git is attached. Any chance to get this merged? I could take a look at darcs and cvs as well. -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Chris get my mail address: */=0;b=c[a++];) putchar(b-1/(/* gcc -o sig sig.c && ./sig */b/42*2-3)*42);}

2 2

[pacman-dev] [PATCH] Detect inter-package conflicts between files and directories
by Allan McRae 17 Jul '12

17 Jul '12

Detect a conflict between a file/symlink in one package and a directory in another when both are being installed at once. A side effect is the creation on conflicts between a direcotry symlink and a real directory (e.g lib -> usr/lib in pkg1 and /lib in pkg2). Given we can not guarantee pkg1 is installed before pkg2, this is a genuine conflict. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- lib/libalpm/conflict.c | 48 +++++++++++++++++++++++++----------- test/pacman/tests/fileconflict002.py | 2 -- test/pacman/tests/fileconflict015.py | 2 -- 3 files changed, 34 insertions(+), 18 deletions(-) diff --git a/lib/libalpm/conflict.c b/lib/libalpm/conflict.c index f3b269f..041ba6f 100644 --- a/lib/libalpm/conflict.c +++ b/lib/libalpm/conflict.c @@ -274,28 +274,48 @@ static alpm_list_t *filelist_intersection(alpm_filelist_t *filesA, size_t ctrA = 0, ctrB = 0; while(ctrA < filesA->count && ctrB < filesB->count) { + int cmp, isdirA, isdirB; + alpm_file_t *fileA = filesA->files + ctrA; alpm_file_t *fileB = filesB->files + ctrB; - const char *strA = fileA->name; - const char *strB = fileB->name; - /* skip directories, we don't care about them */ + char *strA = strdup(fileA->name); + char *strB = strdup(fileB->name); + + isdirA = 0; if(strA[strlen(strA)-1] == '/') { + isdirA = 1; + strA[strlen(strA)-1] = '\0'; + } + + isdirB = 0; + if(strB[strlen(strB)-1] == '/') { + isdirB = 1; + strB[strlen(strB)-1] = '\0'; + } + + cmp = strcmp(strA, strB); + if(cmp < 0) { ctrA++; - } else if(strB[strlen(strB)-1] == '/') { + } else if(cmp > 0) { ctrB++; } else { - int cmp = strcmp(strA, strB); - if(cmp < 0) { - ctrA++; - } else if(cmp > 0) { - ctrB++; - } else { - /* item in both, qualifies as an intersect */ + /* TODO: this creates conflicts between a symlink to a directory in + * one package and a real directory in the other. For example, + * lib -> /usr/lib in pkg1 and /lib in pkg2. This would be allowed + * when installing one package at a time _provided_ pkg1 is installed + * first. This will need adjusted if the order of package install can + * be guaranteed to install the symlink first */ + + /* when not directories, item in both qualifies as an intersect */ + if(! (isdirA && isdirB)) { ret = alpm_list_add(ret, fileA); - ctrA++; - ctrB++; } - } + ctrA++; + ctrB++; + } + + free(strA); + free(strB); } return ret; diff --git a/test/pacman/tests/fileconflict002.py b/test/pacman/tests/fileconflict002.py index e107cd2..1e6113c 100644 --- a/test/pacman/tests/fileconflict002.py +++ b/test/pacman/tests/fileconflict002.py @@ -19,5 +19,3 @@ self.addrule("!PKG_EXIST=pkg1") self.addrule("!PKG_EXIST=pkg2") self.addrule("!FILE_EXIST=dir/realdir/file") - -self.expectfailure = True diff --git a/test/pacman/tests/fileconflict015.py b/test/pacman/tests/fileconflict015.py index 78634d7..3c80bbc 100644 --- a/test/pacman/tests/fileconflict015.py +++ b/test/pacman/tests/fileconflict015.py @@ -13,5 +13,3 @@ self.addrule("PACMAN_RETCODE=1") self.addrule("!PKG_EXIST=pkg1") self.addrule("!PKG_EXIST=pkg2") - -self.expectfailure = True -- 1.7.11.2

2 2