- Pacman-dev - lists.archlinux.org

[pacman-dev] [PATCH] makepkg: Add docmunetation for separate pacakge function
by Allan McRae 22 Dec '08

22 Dec '08

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- doc/PKGBUILD.5.txt | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/doc/PKGBUILD.5.txt b/doc/PKGBUILD.5.txt index e42a8b8..5171798 100644 --- a/doc/PKGBUILD.5.txt +++ b/doc/PKGBUILD.5.txt @@ -254,6 +254,15 @@ If you create any variables of your own in the build function, it is recommended to use the bash `local` keyword to scope the variable to inside the build function. +package() Function +------------------ +An optional package() function can be specified in addtion to the build() function. +This function is directly sourced by makepkg and run immediately after the build() +function. When specified in combination with the fakeroot BUILDENV option in +linkman:makepkg.conf[5], fakeroot usage will be limited to running the package() +function and the subsequent packing of the package. The build() function will be +run as the user calling makepkg. + Install/Upgrade/Remove Scripting -------------------------------- Pacman has the ability to store and execute a package-specific script when it -- 1.6.0.5

3 6

[pacman-dev] [PATCH] makepkg: Replace getopt with internal function
by Allan McRae 22 Dec '08

22 Dec '08

This will allow makepkg to work on systems like Mac OS X where the default getopt is too old to properly handle long options. The new parse_options function should replicate getopt's behaviour completely. Original work: Yun Zheng Hu <yunzheng.hu(a)gmail.com> [Allan: Rewrite and bug fixes] Signed-off-by: Allan McRae <allan(a)archlinux.org> --- scripts/makepkg.sh.in | 91 +++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 89 insertions(+), 2 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index ea034ad..8cf301c 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1186,6 +1186,93 @@ devel_update() { fi } +# getopt like parser +parse_options() { + local short_options=$1; shift; + local long_options=$1; shift; + local ret=0; + local unused_options="" + + while [ -n "$1" ]; do + if [ ${1:0:2} = '--' ]; then + if [ -n "${1:2}" ]; then + local match="" + for i in ${long_options//,/ }; do + if [ ${1:2} = ${i//:} ]; then + match=$i + break + fi + done + if [ -n "$match" ]; then + if [ ${1:2} = $match ]; then + printf ' %s' "$1" + else + if [ -n "$2" ]; then + printf ' %s' "$1" + shift + printf " '%s'" "$1" + else + echo "makepkg: option '$1' $(gettext "requires an argument")" >&2 + ret=1 + fi + fi + else + echo "makepkg: $(gettext "unrecognized option") '$1'" >&2 + ret=1 + fi + else + shift + break + fi + elif [ ${1:0:1} = '-' ]; then + for ((i=1; i<${#1}; i++)); do + if [[ "$short_options" =~ "${1:i:1}" ]]; then + if [[ "$short_options" =~ "${1:i:1}:" ]]; then + if [ -n "${1:$i+1}" ]; then + printf ' -%s' "${1:i:1}" + printf " '%s'" "${1:$i+1}" + else + if [ -n "$2" ]; then + printf ' -%s' "${1:i:1}" + shift + printf " '%s'" "${1}" + else + echo "makepkg: option $(gettext "requires an argument") -- '${1:i:1}'" >&2 + ret=1 + fi + fi + break + else + printf ' -%s' "${1:i:1}" + fi + else + echo "makepkg: $(gettext "invalid option") -- '${1:i:1}'" >&2 + ret=1 + fi + done + else + unused_options="${unused_options} '$1'" + fi + shift + done + + printf " --" + if [ -n "$unused_options" ]; then + for i in ${unused_options[@]}; do + printf ' %s' "$i" + done + fi + if [ -n "$1" ]; then + while [ -n "$1" ]; do + printf " '%s'" "${1}" + shift + done + fi + printf "\n" + + return $ret +} + usage() { printf "makepkg (pacman) %s\n" "$myver" echo @@ -1251,8 +1338,8 @@ OPT_LONG="$OPT_LONG,install,log,nocolor,nobuild,rmdeps,repackage,source" OPT_LONG="$OPT_LONG,syncdeps,version,config:" # Pacman Options OPT_LONG="$OPT_LONG,noconfirm,noprogressbar" -OPT_TEMP="$(getopt -o "$OPT_SHORT" -l "$OPT_LONG" -n "$(basename "$0")" -- "$@" || echo 'GETOPT GO BANG!')" -if echo "$OPT_TEMP" | grep -q 'GETOPT GO BANG!'; then +OPT_TEMP="$(parse_options $OPT_SHORT $OPT_LONG "$@" || echo 'PARSE_OPTIONS FAILED')" +if echo "$OPT_TEMP" | grep -q 'PARSE_OPTIONS FAILED'; then # This is a small hack to stop the script bailing with 'set -e' echo; usage; exit 1 # E_INVALID_OPTION; fi -- 1.6.0.5

1 0

[pacman-dev] [PATCH] Added the function parse_options to replace getopt
by Yun Zheng Hu 22 Dec '08

22 Dec '08

This will make makepkg work properly on systems like Mac OS X, where the default installed getopt does not handle long options. The new parse_options function tries to behave like the original getopt command as much as possible. see also: http://www.archlinux.org/pipermail/pacman-dev/2008-May/011830.html Signed-off-by: Yun Zheng Hu <yunzheng.hu(a)gmail.com> --- scripts/makepkg.sh.in | 81 +++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 79 insertions(+), 2 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 6dc7db4..8e3ad25 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1117,6 +1117,83 @@ devel_update() { fi } +## +# usage : match_long_opt( $needle, $options ) +# return : matched option +## +match_long_opt() { + local needle=$1; shift + local matches item + for item in "$@"; do + [[ "$item" =~ ^"$needle" ]] && matches="$matches $item" + done + matches=( $matches ) + [[ ${#matches[@]} == 1 ]] && echo ${matches[0]} +} + +# getopt like parser, returns 1 on error +parse_options() { + local short_options=$1; shift; + local long_options=$1; shift; + + local needs_args=0 consume_args=0 retcode=0 + local opt_args="" last_opt="" + until [ -z "$1" ]; do + local handle_opt=$((!$needs_args && !$consume_args)) + + if [[ ${1:0:2} = '--' && $handle_opt == 1 ]]; then + # handle long option + opt=${1:2} + if [ ! -z "$opt" ]; then + longopt_match=$(match_long_opt $opt ${long_options//,/ }) + longopt_real="${longopt_match/:/}" # strip possible ':' + [[ "${longopt_match}" == "${longopt_real}:" ]] && needs_args=1 + [[ ! -z "$longopt_match" ]] && printf ' --%s' "$longopt_real" || { + echo "makepkg: $(gettext "unrecognized option") '$1'" >&2 + retcode=1 + } + elif [ $consume_args == 0 ]; then + # option '--', stop parsing options + consume_args=1 + fi + elif [[ ${1:0:1} = '-' && $handle_opt == 1 ]]; then + # handle short option(s) + opts=${1:1} + local i=0 args="" + while [ $i -lt ${#opts} ]; do + opt=${opts:$((i++)):1} + [[ "$needs_args" == 1 ]] && args="${param}${opt}" && continue + [[ "$short_options" =~ "${opt}:" ]] && needs_args=1 + [[ "$short_options" =~ "${opt}" ]] && printf ' -%s' "${opt}" || { + echo "makepkg: $(gettext "invalid option") -- '$opt'" >&2 + retcode=1 + } + done + # check if the user supplied the argument directly after the option + [ ! -z "$args" ] && printf " '$param'" && needs_args=0 + else + # handle non option + [ "$needs_args" == 1 ] && printf " '$1'" && needs_args=0 + [ "$consume_args" == 1 ] && opt_args="$opt_args '$1'" + fi + last_opt="$1" + shift + done + + # check if the user forgot to supply a required argument + if [[ "$needs_args" == 1 ]]; then + if [[ ${last_opt:0:2} = '--' ]]; then + echo "makepkg: option '$last_opt' $(gettext "requires an argument")" >&2 + else + echo "makepkg: option $(gettext "requires an argument") -- '${last_opt:1}'" >&2 + fi + retcode=1 + fi + + printf " --$opt_args\n" + return $retcode +} + usage() { printf "makepkg (pacman) %s\n" "$myver" echo @@ -1182,8 +1259,8 @@ OPT_LONG="$OPT_LONG,install,log,nocolor,nobuild,rmdeps,repackage,source" OPT_LONG="$OPT_LONG,syncdeps,version,config:" # Pacman Options OPT_LONG="$OPT_LONG,noconfirm,noprogressbar" -OPT_TEMP="$(getopt -o "$OPT_SHORT" -l "$OPT_LONG" -n "$(basename "$0")" -- "$@" || echo 'GETOPT GO BANG!')" -if echo "$OPT_TEMP" | grep -q 'GETOPT GO BANG!'; then +OPT_TEMP="$(parse_options $OPT_SHORT $OPT_LONG "$@" || echo 'PARSE_OPTIONS FAILED')" +if echo "$OPT_TEMP" | grep -q 'PARSE_OPTIONS FAILED'; then # This is a small hack to stop the script bailing with 'set -e' echo; usage; exit 1 # E_INVALID_OPTION; fi -- 1.5.6.5

3 6

[pacman-dev] PKGBUILD.proto - Take3
by Jud 19 Dec '08

19 Dec '08

Hi, Edit: Take3, suggested changes - thanks everyone! Dan suggested I send this to the pacman-dev list. After completing some research and asking alot of questions I present some minor changes to PKGBUILD.proto supplied as a .diff to be merged after your approval. I believe it helps the intended audience create a better PKGBUILD in less time according to the latest Arch Packaging Standards. Cheers Jud Inline: --- PKGBUILD.proto 2008-12-05 23:32:33.000005000 +1000 +++ PKGBUILD.proto.new 2008-12-08 17:17:15.536302100 +1000 @@ -4,12 +4,13 @@ # then please put 'unknown'. # Contributor: Your Name <youremail(a)domain.com> + pkgname=NAME -pkgver=VERSION +pkgver=VERSION # Note: Cannot contain hyphens pkgrel=1 pkgdesc="" +url="http://ADDRESS/" arch=() -url="" license=('GPL') groups=() depends=() @@ -20,17 +21,14 @@ replaces=() backup=() options=() -install= -source=($pkgname-$pkgver.tar.gz) +install=$pkgname.install +source=(http://ADDRESS/TO/FILE/$pkgname-$pkgver.tar.gz) noextract=() -md5sums=() #generate with 'makepkg -g' +md5sums=() # Generate with 'makepkg -g' build() { cd "$srcdir/$pkgname-$pkgver" - ./configure --prefix=/usr make || return 1 - make DESTDIR="$pkgdir/" install + make DESTDIR="$pkgdir/" install || return 1 } - -# vim:set ts=2 sw=2 et:

4 3

[pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home directory.
by Chris Brannon 18 Dec '08

18 Dec '08

GnuPG looks for configuration files and keyrings in its home directory. For a user, that is typically ~/.gnupg. This patch causes pacman to use /etc/pacman.d/gnupg/ as the default GnuPG home. One may override the default using --gpgdir on the command-line or GPGDir in pacman's configuration file. Signed-off-by: Chris Brannon <cmbrannon(a)cox.net> --- doc/pacman.8.txt | 7 +++++++ doc/pacman.conf.5.txt | 6 ++++++ src/pacman/Makefile.am | 2 ++ src/pacman/conf.h | 1 + src/pacman/pacman.c | 25 +++++++++++++++++++++++++ 5 files changed, 41 insertions(+), 0 deletions(-) diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt index 6f071ba..a780627 100644 --- a/doc/pacman.8.txt +++ b/doc/pacman.8.txt @@ -136,6 +136,13 @@ Options *\--config* <'file'>:: Specify an alternate configuration file. +*\--gpgdir* <'dir':: + Specify a directory of files used by GnuPG to verify package + signatures. This directory should contain two files: + ``pubring.gpg'' and ``trustdb.gpg''. ``pubring.gpg'' holds the public + keys of all packagers. ``trustdb.gpg'' contains a so-called + trust database, which specifies that the keys are authentic and trusted. + *\--logfile* <'file'>:: Specify an alternate log file. This is an absolute path, regardless of the installation root setting. diff --git a/doc/pacman.conf.5.txt b/doc/pacman.conf.5.txt index 8ef11ec..fa69bfa 100644 --- a/doc/pacman.conf.5.txt +++ b/doc/pacman.conf.5.txt @@ -69,6 +69,12 @@ Options path, the root path is not automatically prepended. +*GPGDir =* path/to/gpg/dir:: + Overrides the default location of the directory containing + configuration files for GnuPG. + A typical default is ``/etc/pacman.d/gnupg''. + This is an absolute path, and the root directory is not prepended. + *LogFile =* '/path/to/file':: Overrides the default location of the pacman log file. A typical default is ``/var/log/pacman.log''. This is an absolute path and the root directory diff --git a/src/pacman/Makefile.am b/src/pacman/Makefile.am index 220ee9c..4da6ef3 100644 --- a/src/pacman/Makefile.am +++ b/src/pacman/Makefile.am @@ -1,6 +1,7 @@ # paths set at make time conffile = ${sysconfdir}/pacman.conf dbpath = ${localstatedir}/lib/pacman/ +gpgdir = ${sysconfdir}/pacman.d/gnupg/ cachedir = ${localstatedir}/cache/pacman/pkg/ logfile = ${localstatedir}/log/pacman.log @@ -10,6 +11,7 @@ DEFS = -DLOCALEDIR=\"@localedir@\" \ -DCONFFILE=\"$(conffile)\" \ -DROOTDIR=\"$(ROOTDIR)\" \ -DDBPATH=\"$(dbpath)\" \ + -DGPGDIR=\"$(gpgdir)\" \ -DCACHEDIR=\"$(cachedir)\" \ -DLOGFILE=\"$(logfile)\" \ @DEFS@ diff --git a/src/pacman/conf.h b/src/pacman/conf.h index 8ea6662..f491057 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -37,6 +37,7 @@ typedef struct __config_t { char *rootdir; char *dbpath; char *logfile; + char *gpgdir; /* TODO how to handle cachedirs? */ unsigned short op_q_isfile; diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c index 3255cdf..18fd3a8 100644 --- a/src/pacman/pacman.c +++ b/src/pacman/pacman.c @@ -138,6 +138,7 @@ static void usage(int op, const char * const myname) printf(_(" -q, --quiet show less information for query and search\n")); } printf(_(" --config <path> set an alternate configuration file\n")); + printf(_(" --gpgdir <path> set an alternate home directory for GnuPG\n")); printf(_(" --logfile <path> set an alternate log file\n")); printf(_(" --noconfirm do not ask for any confirmation\n")); printf(_(" --noprogressbar do not show a progress bar when downloading files\n")); @@ -306,6 +307,20 @@ static void setlibpaths(void) } } + /* + * Set GnuPG's home directory. This is not relative to + * rootdir, even if rootdir is defined. + * Reasoning: gpgdir contains configuration data. +*/ + if(config->gpgdir) { + ret = alpm_option_set_signaturedir(config->gpgdir); + if(ret != 0) { + pm_printf(PM_LOG_ERROR, _("problem setting gpgdir '%s' (%s)\n"), + config->gpgdir, alpm_strerrorlast()); + cleanup(ret); + } + } + /* add a default cachedir if one wasn't specified */ if(alpm_option_get_cachedirs() == NULL) { alpm_option_add_cachedir(CACHEDIR); @@ -366,6 +381,7 @@ static int parseargs(int argc, char *argv[]) {"debug", optional_argument, 0, 1003}, {"noprogressbar", no_argument, 0, 1004}, {"noscriptlet", no_argument, 0, 1005}, + {"gpgdir", required_argument, 0, 1006}, {"cachedir", required_argument, 0, 1007}, {"asdeps", no_argument, 0, 1008}, {"logfile", required_argument, 0, 1009}, @@ -446,6 +462,9 @@ static int parseargs(int argc, char *argv[]) case 1012: config->flags |= PM_TRANS_FLAG_ALLEXPLICIT; break; + case 1006: + config->gpgdir = strdup(optarg); + break; case 'Q': config->op = (config->op != PM_OP_MAIN ? 0 : PM_OP_QUERY); break; case 'R': config->op = (config->op != PM_OP_MAIN ? 0 : PM_OP_REMOVE); break; case 'S': config->op = (config->op != PM_OP_MAIN ? 0 : PM_OP_SYNC); break; @@ -725,6 +744,11 @@ static int _parseconfig(const char *file, const char *givensection, config->rootdir = strdup(ptr); pm_printf(PM_LOG_DEBUG, "config: rootdir: %s\n", ptr); } + } else if (strcmp(key, "GPGDir") == 0) { + if(!config->gpgdir) { + config->gpgdir = strdup(ptr); + pm_printf(PM_LOG_DEBUG, "config: gpgdir: %s\n", ptr); + } } else if (strcmp(key, "LogFile") == 0) { if(!config->logfile) { config->logfile = strdup(ptr); @@ -864,6 +888,7 @@ int main(int argc, char *argv[]) /* define paths to reasonable defaults */ alpm_option_set_root(ROOTDIR); alpm_option_set_dbpath(DBPATH); + alpm_option_set_signaturedir(GPGDIR); alpm_option_set_logfile(LOGFILE); /* Priority of options: -- 1.6.0.5

6 16

[pacman-dev] [PATCH 1/2] Added a new database backend
by Sivert Berg 18 Dec '08

18 Dec '08

This is an attempt to add a one-file-per-database backend. The main reason I wrote this was because load times of the database on my laptop when it's not yet cached is around 40-50 seconds. When you just want to do a quick "pacman -Ss" that can be a little annoying. I know you have discussed the current database vs. one-file earlier, and I think someone said another backend could be supported too. This first patch contains the main database code and a program to convert .db.tar.gz and file databases into the new format. Signed-off-by: Sivert Berg <sivertb(a)stud.ntnu.no> --- lib/libalpm/be_packed.h | 757 +++++++++++++++++++++++++++++++++++++++++++++++ src/util/Makefile.am | 4 +- src/util/convdb.c | 236 +++++++++++++++ 3 files changed, 996 insertions(+), 1 deletions(-) create mode 100644 lib/libalpm/be_packed.h create mode 100644 src/util/convdb.c diff --git a/lib/libalpm/be_packed.h b/lib/libalpm/be_packed.h new file mode 100644 index 0000000..e118a3a --- /dev/null +++ b/lib/libalpm/be_packed.h @@ -0,0 +1,757 @@ +/* + * Copyright (c) 2008 Sivert Berg + * + * Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + */ + +/* _HOW IT WORKS_ + * + * If you've ever looked at the ext2 filesystem you will find alot + * of similarities (among others the inode that I didn't have the + * imagination to find a new name for). + * + * _GROUP_ + * Basicly the database is divided into groups. Each group got the + * same number of blocks and inodes. + * + * _INODE_ + * Each file is assigned an unique inode. The inode got a list of + * blocks where the actual data for the file is stored. Worth noting + * is that the last entry in the block list is indirect. That means + * it points to a block that points to blocks. + */ + +#include <stdint.h> +#include <sys/mman.h> +#include <stdlib.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <unistd.h> + + +#ifndef BE_PACKED_H +#define BE_PACKED_H + +#define BLOCK_SIZE 256 +#define GROUP_SIZE (BLOCK_SIZE * 4096) +#define BLOCKS_PER_GROUP 3936 +#define INODES_PER_GROUP 1024 + +#define MIN(a, b) (((a) < (b))?(a):(b)) + +/* _pack_open_file flags */ +#define PO_READ 1 +#define PO_WRITE 2 + + +typedef struct __inode_t { + int32_t size; + int32_t block[7]; +} inode_t; + +typedef struct __group_desc_t { + uint32_t inodes_free; + uint32_t blocks_free; + uint32_t checksum; + inode_t i_table[INODES_PER_GROUP]; + uint32_t bitmap[BLOCKS_PER_GROUP / 32]; + char data[]; +} group_desc_t; + +typedef struct __package_t { + int32_t inode; + uint16_t rec_len; + uint16_t name_len; + char name[]; +} package_t; + +typedef struct __pack_db_t { + int fd; + int groups; + int read_only; + group_desc_t *map; +} pack_db_t; + +typedef struct __pack_file_t { + int inode; + size_t pos; +} pack_file_t; + +typedef struct __pack_dir_t { + size_t pos; +} pack_dir_t; + + +/** + ** GROUP FUNCTIONS + */ +/* Initilizes a new group */ +void _pack_init_group(group_desc_t *group) +{ + int i, j; + + for (i = 0; i < INODES_PER_GROUP; i++) { + group->i_table[i].size = -1; + for (j = 0; j < 7; j++) + group->i_table[i].block[j] = -1; + } + + for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) { + group->bitmap[i] = 0; + } + + group->inodes_free = INODES_PER_GROUP; + group->blocks_free = BLOCKS_PER_GROUP; + group->checksum = 0; +} + + +/* Adds a new group to the file */ +void _pack_new_group(pack_db_t *db) +{ + db->groups++; + ftruncate(db->fd, db->groups * GROUP_SIZE); + db->map = mremap(db->map, (db->groups - 1) * GROUP_SIZE, + db->groups * GROUP_SIZE, MREMAP_MAYMOVE); + + _pack_init_group((group_desc_t*)((char*)db->map + + GROUP_SIZE * (db->groups - 1))); +} + + +/** + ** POINTER FUNCTIONS (return pointers to various structures) + */ +group_desc_t *_pack_get_group(pack_db_t *db, int group) +{ + return (group_desc_t*)((char*)db->map + group*GROUP_SIZE); +} + + +inode_t *_pack_get_inode(pack_db_t *db, int inode) +{ + int group = inode / INODES_PER_GROUP; + group_desc_t *groupd = _pack_get_group(db, group); + inode %= INODES_PER_GROUP; + + return groupd->i_table + inode; +} + +void *_pack_get_block(pack_db_t *db, int block) +{ + if (block == -1) + return NULL; + + group_desc_t *group = _pack_get_group(db, block / BLOCKS_PER_GROUP); + return group->data + (block % BLOCKS_PER_GROUP) * BLOCK_SIZE; +} + + +/* Searches for a free block in the bitmap and marks the bit + * it finds as used. + */ +int _pack_check_bitmap(group_desc_t *group) +{ + int i = 0, j = 0, t = 0; + + for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) { + if (group->bitmap[i] != 0xffffffff) { + t = group->bitmap[i]; + for (j = 0; j < 32 && (t & 1); j++, t >>= 1); + break; + } + } + + group->bitmap[i] |= 1 << j; + return i * 32 + j; +} + + +/* Clears the n'th bit */ +int _pack_bitmap_free(group_desc_t *group, int n) +{ + int i = n / 32; + int j = n % 32; + + group->bitmap[i] &= ~(i << j); + return 0; +} + + +/* Returns the number of a new unused block + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_get_free_block(pack_db_t *db) +{ + int i; + group_desc_t *group; + int ret; + + for (i = 0; i < db->groups; i++) { + group = _pack_get_group(db, i); + if (group->blocks_free) { + group->blocks_free--; + ret = _pack_check_bitmap(group); + ret += i * BLOCKS_PER_GROUP; + return ret;; + } + } + + _pack_new_group(db); + group = _pack_get_group(db, i); + group->blocks_free--; + ret = _pack_check_bitmap(group); + ret += i * BLOCKS_PER_GROUP; + + return ret; +} + + +/* Frees the block given */ +int _pack_block_free(pack_db_t *db, int block) +{ + group_desc_t *group; + int g, b; + + g = block / BLOCKS_PER_GROUP; + b = block & BLOCKS_PER_GROUP; + + group = _pack_get_group(db, g); + group->blocks_free++; + + return _pack_bitmap_free(group, b); +} + + +/* Returns the number of a new free inode. + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_get_free_inode(pack_db_t *db) +{ + int i = 0, j = 0; + group_desc_t *group; + + for (i = 0; i < db->groups; i++) { + group = _pack_get_group(db, i); + if (group->inodes_free) { + group->inodes_free--; + for (j = 0; j < INODES_PER_GROUP; j++) { + if (group->i_table[j].size == -1) { + group->i_table[j].size = 0; + return i * INODES_PER_GROUP + j; + } + } + } + } + + _pack_new_group(db); + group = _pack_get_group(db, i); + group->inodes_free--; + + return i * INODES_PER_GROUP; +} + + +/* Frees an inode and all of the blocks it might be using */ +int _pack_inode_free(pack_db_t *db, int in) +{ + inode_t *inode = _pack_get_inode(db, in); + group_desc_t *group = _pack_get_group(db, in / INODES_PER_GROUP); + int i; + int block; + int32_t *blocks; + int size = BLOCK_SIZE / sizeof(uint32_t) - 1; + + inode->size = -1; + + for (i = 0; i < 6; i++) { + if (inode->block[i] != -1) { + _pack_block_free(db, inode->block[i]); + inode->block[i] = -1; + } + } + + block = inode->block[6]; + blocks = _pack_get_block(db, block); + + while(blocks != NULL) { + for (i = 0; i < size; i++) { + if (blocks[i] != -1) + _pack_block_free(db, blocks[i]); + } + + _pack_block_free(db, block); + block = blocks[size]; + blocks = _pack_get_block(db, block); + } + + group->inodes_free++; + + return 0; +} + + +/* Returns the number of the block associated with the inode */ +int _pack_inode_get_block(pack_db_t *db, int i, int block) +{ + inode_t *inode = _pack_get_inode(db, i); + if (block < 6) + return inode->block[block]; + + block -= 6; + int size = BLOCK_SIZE / sizeof(uint32_t) - 1; + uint32_t *blocks = _pack_get_block(db, inode->block[6]); + + for (;blocks != NULL && block >= size; block -= size) + blocks = _pack_get_block(db, blocks[size]); + + if (blocks == NULL) + return -1; + + return blocks[block]; +} + + +/* Returns the number of the block block associated with the inode. + * If there is no such block, it gets allocated. + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_inode_add_block(pack_db_t *db, int in, int block) +{ + inode_t *inode = _pack_get_inode(db, in); + int tmp; + + if (block < 6) { + if (inode->block[block] != -1) + return inode->block[block]; + + tmp = _pack_get_free_block(db); + inode = _pack_get_inode(db, in); + return inode->block[block] = tmp; + } + + block -= 6; + int size = BLOCK_SIZE / sizeof(int32_t) - 1; + int i; + int old_block; + int32_t *blocks; + + if (inode->block[6] == -1) { + tmp = _pack_get_free_block(db); + inode = _pack_get_inode(db, in); + inode->block[6] = tmp; + blocks = _pack_get_block(db, inode->block[6]); + for (i = 0; i < size + 1; i++) + blocks[i] = -1; + } + else + blocks = _pack_get_block(db, inode->block[6]); + + old_block = inode->block[6]; + + for (;block >= size; block -= size) { + if (blocks[size] == -1) { + tmp = _pack_get_free_block(db); + blocks = _pack_get_block(db, old_block); + blocks[size] = tmp; + old_block = tmp; + blocks = _pack_get_block(db, tmp); + for (i = 0; i < size + 1; i++) + blocks[i] = -1; + } + else { + old_block = blocks[size]; + blocks = _pack_get_block(db, blocks[size]); + } + } + + if (blocks[block] == -1) { + tmp = _pack_get_free_block(db); + blocks = _pack_get_block(db, old_block); + blocks[block] = tmp; + } + + return blocks[block]; +} + +/* Returns a pointer to the package with the name pkgname. + * Returns NULL if it's not found + */ +package_t *_pack_find_package(pack_db_t *db, const char *pkgname) +{ + inode_t *inode = _pack_get_inode(db, 0); + int s_len = strlen(pkgname); + package_t *pkg; + int block = 0; + int b_pos = 0; + + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + + while (block < (inode->size / BLOCK_SIZE)) { + if (pkg->inode != 0 && + pkg->name_len == s_len && + strncmp(pkgname, pkg->name, s_len) == 0) + break; + + b_pos += pkg->rec_len; + pkg = (package_t*)((char*)pkg + pkg->rec_len); + if (b_pos >= BLOCK_SIZE) { + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block)); + b_pos = 0; + } + } + + return pkg; +} + +/* Creates a new package with the name pkgname. + * Returns a pointer to the new package. + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +package_t *_pack_add_package(pack_db_t *db, const char *pkgname) +{ + inode_t *inode = _pack_get_inode(db, 0); + int s_len = strlen(pkgname); + int len = s_len + sizeof(package_t); + package_t *pkg; + int block = 0; + int b_pos = 0; + + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + + while (pkg && block < (inode->size / BLOCK_SIZE)) { + if (pkg->rec_len >= len && pkg->inode == 0) + break; + + b_pos += pkg->rec_len; + pkg = (package_t*)((char*)pkg + pkg->rec_len); + if (b_pos >= BLOCK_SIZE) { + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block)); + b_pos = 0; + } + } + + if (pkg == NULL) { + pkg = _pack_get_block(db, _pack_inode_add_block(db, 0, block)); + pkg->rec_len = BLOCK_SIZE; + pkg->inode = 0; + inode = _pack_get_inode(db, 0); + inode->size += BLOCK_SIZE; + } + + strncpy(pkg->name, pkgname, s_len); + if (pkg->rec_len > (len + sizeof(package_t) + 10)) { + package_t *foo = (package_t*)((char*)pkg + len); + foo->rec_len = pkg->rec_len - len; + foo->inode = 0; + pkg->rec_len = len; + } + + pkg->inode = -1; + pkg->name_len = s_len; + + return pkg; +} + + +/* Reads data from the file until buf is full or we hit a newline */ +int _pack_file_gets(pack_db_t *db, pack_file_t *file, char *buf, size_t size) +{ + inode_t *inode = _pack_get_inode(db, file->inode); + size_t i; + size_t b_pos = file->pos % BLOCK_SIZE; + const char *src = _pack_get_block(db, + _pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE)); + int newline = 0; + + for (i = 0 + ;i < size && file->pos < inode->size && !newline + ;i++, b_pos++, file->pos++) { + if (b_pos >= BLOCK_SIZE) { + src = _pack_get_block(db, + _pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE)); + b_pos = 0; + } + + buf[i] = src[b_pos]; + + if (src[b_pos] == '\n') + newline = 1; + } + + if (i < size - 1) + buf[i] = '\0'; + + return i; +} + + +/* Write the data in buf into the file + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_file_puts(pack_db_t *db, pack_file_t *file, + const char *buf, size_t size) +{ + size_t i; + size_t b_pos = file->pos % BLOCK_SIZE; + char *dest = _pack_get_block(db, + _pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE)); + int block; + + for (i = 0; i < size; i++, file->pos++, b_pos++) + { + if (b_pos >= BLOCK_SIZE) { + block = _pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE); + dest = _pack_get_block(db, block); + b_pos = 0; + } + dest[b_pos] = buf[i]; + } + + inode_t *inode = _pack_get_inode(db, file->inode); + inode->size = file->pos; + + return i; +} + + +int _pack_file_eof(pack_db_t *db, pack_file_t *file) +{ + inode_t *inode = _pack_get_inode(db, file->inode); + return file->pos >= inode->size; +} + + +/* Opens a file in the database + * flags: if you pass PO_READ, it will return -1 + * if the file does not exist, if you pass PO_WRITE + * the file will be created if it does not exist + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +pack_file_t *_pack_open_file(pack_db_t *db, const char *pkgname, + const char *filename, int flags) +{ + char name[512]; + + snprintf(name, 512, "%s/%s", pkgname, filename); + + package_t *pkg = _pack_find_package(db, name); + int w = flags & PO_WRITE; + int tmp; + + if (w && db->read_only) + return NULL; + + if (pkg == NULL && w) + pkg = _pack_add_package(db, name); + else if (pkg == NULL) + return NULL; + + pack_file_t *ret = malloc(sizeof(pack_file_t)); + ret->pos = 0; + + if (pkg->inode == -1 && w) { + tmp = _pack_get_free_inode(db); + pkg = _pack_find_package(db, name); + ret->inode = pkg->inode = tmp; + } + else + ret->inode = pkg->inode; + + return ret; +} + + +/* Closes the file given */ +void _pack_close_file(pack_file_t *file) +{ + free(file); +} + + +/* Removes a package along with all it's files */ +int _pack_package_remove(pack_db_t *db, const char *pkgname) +{ + inode_t *inode = _pack_get_inode(db, 0); + int s_len = strlen(pkgname); + package_t *prev, *pkg; + int block = 0; + int b_pos = 0; + + if (db->read_only) + return -1; + + prev = NULL; + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + + while (block < (inode->size / BLOCK_SIZE)) { + if (pkg->inode != 0 && + pkg->name_len > s_len && + strncmp(pkgname, pkg->name, s_len) == 0 && + pkg->name[s_len] == '/') { + _pack_inode_free(db, pkg->inode); + + if (prev != NULL && prev->inode == 0) { + prev->rec_len += pkg->rec_len; + pkg = prev; + } + else + pkg->inode = 0; + } + + prev = pkg; + b_pos += pkg->rec_len; + pkg = (package_t*)((char*)pkg + pkg->rec_len); + if (b_pos >= BLOCK_SIZE) { + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block)); + b_pos = 0; + prev = NULL; + } + } + + return 0; +} + + +/* Gets the next entry in the directory */ +int _pack_dir_next(pack_db_t *db, pack_dir_t *dir, char *name, size_t len) +{ + inode_t *inode = _pack_get_inode(db, 0); + + if (dir->pos >= inode->size) + return -1; + + int block, b_pos; + package_t *pkg; + + do { + block = dir->pos / BLOCK_SIZE; + b_pos = dir->pos % BLOCK_SIZE; + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + if (!pkg) + return -1; + pkg = (package_t*)((char*)pkg + b_pos); + dir->pos += pkg->rec_len; + } while (pkg->inode == 0); + + strncpy(name, pkg->name, MIN(len, pkg->name_len)); + if (len > pkg->name_len) + name[pkg->name_len] = '\0'; + + return 0; +} + + +/* Opens the directory for reading */ +pack_dir_t *_pack_open_dir(pack_db_t *db) +{ + pack_dir_t *dir = malloc(sizeof(pack_dir_t)); + dir->pos = 0; + + return dir; +} + + +/* Closes the given directory */ +void _pack_close_dir(pack_dir_t *dir) +{ + free(dir); +} + + +/* Opens a packed database */ +pack_db_t *_pack_db_open(const char *filename) +{ + struct stat st; + int is_new = 0; + + pack_db_t *db = malloc(sizeof(pack_db_t)); + db->fd = open(filename, O_RDWR, 0); + db->read_only = 0; + + if (db->fd == -1) { + db->fd = open(filename, O_RDONLY, 0); + if (db->fd == -1) { + db->fd = open(filename, O_RDWR | O_CREAT, 0644); + + if (db->fd == -1) { + free(db); + printf("Could not open %s\n", filename); + return NULL; + } + + ftruncate(db->fd, GROUP_SIZE); + is_new = 1; + } + else + db->read_only = 1; + } + + fstat(db->fd, &st); + db->groups = st.st_size / GROUP_SIZE; + + int flags = PROT_READ; + if (!db->read_only) + flags |= PROT_WRITE; + + db->map = mmap(NULL, GROUP_SIZE * db->groups, flags, MAP_SHARED, + db->fd, 0); + + if (db->map == MAP_FAILED) { + perror("mmap"); + free(db); + exit(0); + } + + if (is_new) { + _pack_init_group(db->map); + db->map->i_table[0].size = 0; + } + + return db; +} + + +/* Closes a packed database */ +void _pack_db_close(pack_db_t *db) +{ + munmap(db->map, db->groups * GROUP_SIZE); + close(db->fd); + free(db); +} + +#endif /* BE_PACKED_H */ diff --git a/src/util/Makefile.am b/src/util/Makefile.am index 97a0ffa..64e18ad 100644 --- a/src/util/Makefile.am +++ b/src/util/Makefile.am @@ -3,7 +3,7 @@ conffile = ${sysconfdir}/pacman.conf dbpath = ${localstatedir}/lib/pacman/ cachedir = ${localstatedir}/cache/pacman/pkg/ -bin_PROGRAMS = vercmp testpkg testdb +bin_PROGRAMS = vercmp testpkg testdb convdb DEFS = -DLOCALEDIR=\"@localedir@\" \ -DCONFFILE=\"$(conffile)\" \ @@ -18,6 +18,8 @@ AM_CFLAGS = -pedantic -D_GNU_SOURCE vercmp_SOURCES = vercmp.c vercmp_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la +convdb_SOURCES = convdb.c + testpkg_SOURCES = testpkg.c testpkg_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la diff --git a/src/util/convdb.c b/src/util/convdb.c new file mode 100644 index 0000000..f1f39bd --- /dev/null +++ b/src/util/convdb.c @@ -0,0 +1,236 @@ +/* + * Copyright (c) 2008 Sivert Berg + * + * Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + */ + + +/* WARNING: This file is pretty much just a quick hack thrown together + * in a hurry. It seems to work, but don't take my word for it. + */ + +#include <be_packed.h> +#include <dirent.h> +#include <archive.h> +#include <archive_entry.h> + + +/* Convert an archieved database into a packed database + * Expects a gziped database ending in .db.tar.gz + */ +void convert_archive(const char *filename) +{ + struct archive_entry *aent = NULL; + struct archive *arch = archive_read_new(); + archive_read_support_compression_gzip(arch); + archive_read_support_format_tar(arch); + if (archive_read_open_filename(arch, filename, 1) != ARCHIVE_OK) { + printf("Could not open archive %s\n", archive_error_string(arch)); + return; + } + + char *db_name = strndup(filename, strlen(filename) - strlen(".tar.gz")); + pack_db_t *db = _pack_db_open(db_name); + free(db_name); + char buffer[4096]; + size_t size; + pack_file_t *pf; + + while (!archive_read_next_header(arch, &aent)) { + const char *type = NULL; + if (strstr(archive_entry_pathname(aent), "/desc")) + type = "desc"; + else if (strstr(archive_entry_pathname(aent), "/depends")) + type = "depends"; + else if (strstr(archive_entry_pathname(aent), "/install")) + type = "install"; + else if (strstr(archive_entry_pathname(aent), "/files")) + type = "files"; + + if (type == NULL) + continue; + + char *name = strdup(archive_entry_pathname(aent)); + int i; + for (i = 0; name[i] != '/' && name[i]; i++); + name[i] = '\0'; + + pf = _pack_open_file(db, name, type, PO_WRITE); + + free(name); + + do { + size = archive_read_data(arch, buffer, 4096); + _pack_file_puts(db, pf, buffer, size); + } while (size == 4096); + + _pack_close_file(pf); + + } + + _pack_db_close(db); + archive_read_close(arch); + archive_read_finish(arch); +} + + + +const char *dir_name(const char *full_path) +{ + const char *slash = full_path; + while (*full_path) + if (*full_path++ == '/') + slash = full_path; + return slash; +} + + +/* Convert a directory database into a packed database */ +void convert_directory(const char *pathname) +{ + char db_name[1024]; + pack_db_t *db; + DIR *dir = opendir(pathname); + struct dirent *dire; + char path[1024]; + char buffer[8196]; + size_t size; + FILE *f; + pack_file_t *pf; + + strcpy(db_name, dir_name(pathname)); + strcat(db_name, ".db"); + db = _pack_db_open(db_name); + + if (dir == NULL) { + perror("opendir"); + return; + } + + dire = readdir(dir); + while (dire != NULL) { + if (strcmp(dire->d_name, ".") && strcmp(dire->d_name, "..")) { + + + size_t foo = 0; + sprintf(path, "%s/%s/desc", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto depends; + pf = _pack_open_file(db, dire->d_name, "desc", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +depends: + sprintf(path, "%s/%s/depends", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto files; + pf = _pack_open_file(db, dire->d_name, "depends", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +files: + sprintf(path, "%s/%s/files", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto install; + pf = _pack_open_file(db, dire->d_name, "files", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +install: + sprintf(path, "%s/%s/install", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto changelog; + pf = _pack_open_file(db, dire->d_name, "install", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +changelog: + sprintf(path, "%s/%s/changelog", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto deltas; + pf = _pack_open_file(db, dire->d_name, "changelog", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +deltas: + sprintf(path, "%s/%s/deltas", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto done; + pf = _pack_open_file(db, dire->d_name, "deltas", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +done: + pf = NULL; + } + + dire = readdir(dir); + } + + closedir(dir); + _pack_db_close(db); +} + +int main(int argc, char *argv[]) +{ + if (argc < 2) { + printf("Usage: %s <dir/file>\n", argv[0]); + exit(0); + } + + if (strstr(argv[1], ".tar.gz") != NULL) + convert_archive(argv[1]); + else + convert_directory(argv[1]); + + return 0; +} -- 1.6.0.5

3 7

[pacman-dev] [patch]GPG verification
by Jatheendra 17 Dec '08

17 Dec '08

These patches will add VerifySig option to pacman.conf. VerifySig takes options Always, Optional or Never [repo-name] Server = ServerName VerifySig = Always Include = IncludePath >From 77be2c5cbfa3c7a750fe46d115c23096d2cf51e5 Mon Sep 17 00:00:00 2001 From: shankar <jatheendra(a)gmail.com> Date: Wed, 17 Dec 2008 20:52:21 +0530 Subject: [PATCH] changed gpg verification logic Signed-off-by: shankar <jatheendra(a)gmail.com> --- lib/libalpm/signing.c | 3 +++ lib/libalpm/sync.c | 26 ++++++++++++++++++++++---- 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/lib/libalpm/signing.c b/lib/libalpm/signing.c index ddb89bc..0835b5e 100644 --- a/lib/libalpm/signing.c +++ b/lib/libalpm/signing.c @@ -166,6 +166,9 @@ pgpcheck_t _alpm_gpgme_checksig(const char *pkgpath, const pmpgpsig_t *sig) if(gpgsig->summary & GPGME_SIGSUM_VALID) { /* good signature, continue */ + ret = PM_PGP_SIG_VALID; + _alpm_log(PM_LOG_DEBUG, _("Package %s has a valid signature.\n"), + pkgpath); } else if(gpgsig->summary & GPGME_SIGSUM_GREEN) { /* 'green' signature, not sure what to do here */ _alpm_log(PM_LOG_WARNING, _("Package %s has a green signature.\n"), diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c index 24f2b98..f658ae2 100644 --- a/lib/libalpm/sync.c +++ b/lib/libalpm/sync.c @@ -901,12 +901,30 @@ int _alpm_sync_commit(pmtrans_t *trans, pmdb_t *db_local, alpm_list_t **data) *data = alpm_list_add(*data, strdup(filename)); } /* check PGP signature next */ - if(_alpm_gpgme_checksig(filepath, pgpsig) == PM_PGP_SIG_INVALID) { - errors++; - *data = alpm_list_add(*data, strdup(filename)); + pmdb_t *sdb = alpm_pkg_get_db(spkg); + + if(sdb->verify_gpg == PM_GPG_VERIFY_ALWAYS) { + if(_alpm_gpgme_checksig(filepath, pgpsig) != PM_PGP_SIG_VALID) { + errors++; + *data = alpm_list_add(*data, strdup(filename)); + _alpm_log(PM_LOG_ERROR, _("Invalid GPG signature on package: %s\n"),alpm_pkg_get_name(spkg)); + } + FREE(filepath); + } else if (sdb->verify_gpg == PM_GPG_VERIFY_OPTIONAL) { + pgpcheck_t ret1 = _alpm_gpgme_checksig(filepath, pgpsig); + + if(ret1 == PM_PGP_SIG_MISSING) { + /*no problems here*/ + } else if (ret1 != PM_PGP_SIG_VALID) { + errors++; + *data = alpm_list_add(*data, strdup(filename)); + _alpm_log(PM_LOG_ERROR, _("Invalid GPG signature on package: %s\n"),alpm_pkg_get_name(spkg)); + } + FREE(filepath); } - FREE(filepath); } + + if(errors) { pm_errno = PM_ERR_PKG_INVALID; goto error; -- 1.6.0.4

1 0

[pacman-dev] [PATCH] Added a new database backend
by Sivert Berg 16 Dec '08

16 Dec '08

This is an attempt to add a one-file-per-database backend. This leads to significatly faster loading of the database when the database is not cached. As you can see from the patch minimal changes to the rest of ALPM was needed. The only change was to make reading of the changelog files happen through the backend. The database is basically a scaled down ext2-like filesystem. To make corruption detection possible in an early phase I have planned to add a checksum, but have not come that far yet. I have been using it on my laptop for a week or so now, and everything seems to be working as expected. I'm sorry it became such a large patch, but about half of it is mostly old code (be_packed.c is essentially be_files.c with fopen fprintf etc changed with new functions). Signed-off-by: Sivert Berg <sivertb(a)stud.ntnu.no> --- configure.ac | 7 + lib/libalpm/Makefile.am | 2 +- lib/libalpm/be_files.c | 21 ++ lib/libalpm/be_packed.c | 845 +++++++++++++++++++++++++++++++++++++++++++++++ lib/libalpm/be_packed.h | 757 ++++++++++++++++++++++++++++++++++++++++++ lib/libalpm/db.h | 3 + lib/libalpm/package.c | 12 +- src/util/Makefile.am | 4 +- src/util/convdb.c | 236 +++++++++++++ 9 files changed, 1876 insertions(+), 11 deletions(-) create mode 100644 lib/libalpm/be_packed.c create mode 100644 lib/libalpm/be_packed.h create mode 100644 src/util/convdb.c diff --git a/configure.ac b/configure.ac index c7841b8..6e28ac9 100644 --- a/configure.ac +++ b/configure.ac @@ -73,6 +73,12 @@ AC_ARG_WITH(root-dir, AS_HELP_STRING([--with-root-dir=path], [set the location of pacman's root operating directory]), [ROOTDIR=$withval], [ROOTDIR=/]) +# Help line for database backend +AC_ARG_WITH(backend, + AS_HELP_STRING([--with-backend=backend], [choose which backend to use. You can choose between 'files' and 'packed', it defaults to files]), + [BACKEND=$withval], [BACKEND=files]) +AC_SUBST(BACKEND) + # Help line for package extension AC_ARG_WITH(pkg-ext, AS_HELP_STRING([--with-pkg-ext=ext], [set the file extension used by packages]), @@ -362,6 +368,7 @@ ${PACKAGE_NAME}: package extension : ${PKGEXT} source pkg extension : ${SRCEXT} database extension : ${DBEXT} + database backend : ${BACKEND} Compilation options: Run make in doc/ dir : ${wantdoc} diff --git a/lib/libalpm/Makefile.am b/lib/libalpm/Makefile.am index 871855e..79b9d26 100644 --- a/lib/libalpm/Makefile.am +++ b/lib/libalpm/Makefile.am @@ -25,7 +25,7 @@ libalpm_la_SOURCES = \ alpm.h alpm.c \ alpm_list.h alpm_list.c \ backup.h backup.c \ - be_files.c \ + be_@BACKEND@.c \ be_package.c \ cache.h cache.c \ conflict.h conflict.c \ diff --git a/lib/libalpm/be_files.c b/lib/libalpm/be_files.c index b9ff646..81d7a17 100644 --- a/lib/libalpm/be_files.c +++ b/lib/libalpm/be_files.c @@ -874,4 +874,25 @@ int _alpm_db_remove(pmdb_t *db, pmpkg_t *info) return(ret); } +void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg) +{ + char clfile[PATH_MAX]; + snprintf(clfile, PATH_MAX, "%s/%s/%s-%s/changelog", + alpm_option_get_dbpath(), + alpm_db_get_name(handle->db_local), + alpm_pkg_get_name(pkg), + alpm_pkg_get_version(pkg)); + return fopen(clfile, "r"); +} + +int _alpm_db_changelog_close(pmdb_t *db, void *stream) +{ + return fclose(stream); +} + +size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size) +{ + return fread(ptr, 1, size, pf); +} + /* vim: set ts=2 sw=2 noet: */ diff --git a/lib/libalpm/be_packed.c b/lib/libalpm/be_packed.c new file mode 100644 index 0000000..3e4ea68 --- /dev/null +++ b/lib/libalpm/be_packed.c @@ -0,0 +1,845 @@ +/* + * be_packed.c + * + * Copyright (c) 2006 by Christian Hamar <krics(a)linuxforum.hu> + * Copyright (c) 2006 by Miklos Vajna <vmiklos(a)frugalware.org> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#include "config.h" + +#include <unistd.h> +#include <stdio.h> +#include <stdlib.h> +#include <errno.h> +#include <string.h> +#include <stdint.h> /* uintmax_t, intmax_t */ +#include <sys/stat.h> +#include <dirent.h> +#include <ctype.h> +#include <time.h> +#include <utime.h> +#include <limits.h> /* PATH_MAX */ +#include <locale.h> /* setlocale */ + +/* libalpm */ +#include "db.h" +#include "alpm_list.h" +#include "cache.h" +#include "log.h" +#include "util.h" +#include "alpm.h" +#include "handle.h" +#include "package.h" +#include "delta.h" +#include "deps.h" +#include "dload.h" + +#include "be_packed.h" + +/** Update a package database + * @param force if true, then forces the update, otherwise update only in case + * the database isn't up to date + * @param db pointer to the package database to update + * @return 0 on success, > 0 on error (pm_errno is set accordingly), < 0 if up + * to date + */ +int SYMEXPORT alpm_db_update(int force, pmdb_t *db) +{ + char *dbfile, *dbfilepath; + time_t newmtime = 0; + char dbpath[PATH_MAX]; + char dbname[PATH_MAX]; + char buffer[1024]; + size_t len; + struct stat st; + struct utimbuf utb; + + int ret; + + ALPM_LOG_FUNC; + + /* Sanity checks */ + ASSERT(handle != NULL, RET_ERR(PM_ERR_HANDLE_NULL, -1)); + ASSERT(db != NULL && db != handle->db_local, RET_ERR(PM_ERR_WRONG_ARGS, -1)); + /* Verify we are in a transaction. This is done _mainly_ because we need a DB + * lock - if we update without a db lock, we may kludge some other pacman + * process that _has_ a lock. + */ + ASSERT(handle->trans != NULL, RET_ERR(PM_ERR_TRANS_NULL, -1)); + ASSERT(handle->trans->state == STATE_INITIALIZED, RET_ERR(PM_ERR_TRANS_NOT_INITIALIZED, -1)); + ASSERT(handle->trans->type == PM_TRANS_TYPE_SYNC, RET_ERR(PM_ERR_TRANS_TYPE, -1)); + + if(!alpm_list_find_ptr(handle->dbs_sync, db)) { + RET_ERR(PM_ERR_DB_NOT_FOUND, -1); + } + + strncpy(dbname, db->path, PATH_MAX - 3); + /* Strip of the trailing / */ + dbname[strlen(dbname) - 1] = '\0'; + strcat(dbname, ".db"); + stat(dbname, &st); + + if(!force) { + /* get the lastupdate time */ + if(st.st_mtime == 0) { + _alpm_log(PM_LOG_DEBUG, "failed to get last write time for %s\n", + db->treename); + } + } + else + st.st_mtime = 0; + + len = strlen(db->treename) + strlen(DBEXT) + 1; + MALLOC(dbfile, len, RET_ERR(PM_ERR_MEMORY, -1)); + sprintf(dbfile, "%s" DBEXT, db->treename); + + strncpy(dbpath, alpm_option_get_dbpath(), PATH_MAX); + strcpy(dbpath + strlen(dbpath), "sync/"); + + ret = _alpm_download_single_file(dbfile, db->servers, dbpath, + st.st_mtime, &newmtime); + free(dbfile); + + if(ret == 1) { + /* mtimes match, do nothing */ + pm_errno = 0; + return(1); + } else if(ret == -1) { + /* pm_errno was set by the download code */ + _alpm_log(PM_LOG_DEBUG, "failed to sync db: %s\n", alpm_strerrorlast()); + return(-1); + } else { + /* remove the old db */ + _alpm_log(PM_LOG_DEBUG, "removing database %s\n", db->path); + unlink(dbname); + + /* cache needs to be rebuilt */ + _alpm_db_free_pkgcache(db); + + /* form the path to the db location */ + len = strlen(dbpath) + strlen(db->treename) + strlen(DBEXT) + 1; + MALLOC(dbfilepath, len, RET_ERR(PM_ERR_MEMORY, -1)); + sprintf(dbfilepath, "%s%s" DBEXT, dbpath, db->treename); + + /* convert the database file */ + snprintf(buffer, 1024, "convdb %s", dbfilepath); + system(buffer); + + /* set the correct time on the new database */ + utb.actime = newmtime; + utb.modtime = newmtime; + utime(dbname, &utb); + + /* Reopen the database */ + _pack_db_close(db->handle); + db->handle = _pack_db_open(dbname); + + unlink(dbfilepath); + free(dbfilepath); + } + + return(0); +} + +int _alpm_db_open(pmdb_t *db) +{ + ALPM_LOG_FUNC; + char path[PATH_MAX]; + + if(db == NULL) { + RET_ERR(PM_ERR_DB_NULL, -1); + } + + strncpy(path, db->path, PATH_MAX - 3); + /* Strip of the trailing / */ + path[strlen(path) - 1] = '\0'; + strcat(path, ".db"); + + _alpm_log(PM_LOG_DEBUG, "opening database from path '%s'\n", db->path); + db->handle = _pack_db_open(path); + + if(db->handle == NULL) { + RET_ERR(PM_ERR_DB_OPEN, -1); + } + + return(0); +} + +void _alpm_db_close(pmdb_t *db) +{ + ALPM_LOG_FUNC; + + if(db == NULL) { + return; + } + + if(db->handle) { + _pack_db_close(db->handle); + db->handle = NULL; + } +} + +static int splitname(const char *target, pmpkg_t *pkg) +{ + /* the format of a db entry is as follows: + * package-version-rel/ + * package name can contain hyphens, so parse from the back- go back + * two hyphens and we have split the version from the name. + */ + char *tmp, *p, *q; + + if(target == NULL || pkg == NULL) { + return(-1); + } + STRDUP(tmp, target, RET_ERR(PM_ERR_MEMORY, -1)); + p = tmp + strlen(tmp); + + /* do the magic parsing- find the beginning of the version string + * by doing two iterations of same loop to lop off two hyphens */ + for(q = --p; *q && *q != '-'; q--); + for(p = --q; *p && *p != '-'; p--); + if(*p != '-' || p == tmp) { + return(-1); + } + + /* copy into fields and return */ + if(pkg->version) { + FREE(pkg->version); + } + STRDUP(pkg->version, p+1, RET_ERR(PM_ERR_MEMORY, -1)); + /* insert a terminator at the end of the name (on hyphen)- then copy it */ + *p = '\0'; + if(pkg->name) { + FREE(pkg->name); + } + STRDUP(pkg->name, tmp, RET_ERR(PM_ERR_MEMORY, -1)); + + free(tmp); + return(0); +} + +int _alpm_db_populate(pmdb_t *db) +{ + int count = 0; + int i; + char name[PATH_MAX]; + pack_dir_t *dir; + + ALPM_LOG_FUNC; + + ASSERT(db != NULL, RET_ERR(PM_ERR_DB_NULL, -1)); + + dir = _pack_open_dir(db->handle); + + while(_pack_dir_next(db->handle, dir, name, PATH_MAX) != -1) { + pmpkg_t *pkg; + + pkg = _alpm_pkg_new(); + if(pkg == NULL) { + return(-1); + } + + /* Deal only with desc entries, that way we're sure we do + * not add the same package twice + */ + if(strstr(name, "/desc") == NULL) + continue; + + for (i = 0; name[i] && name[i] != '/'; i++); + name[i] = '\0'; + + /* split the db entry name */ + if(splitname(name, pkg) != 0) { + _alpm_log(PM_LOG_ERROR, _("invalid name for database entry '%s'\n"), + name); + _alpm_pkg_free(pkg); + continue; + } + + /* explicitly read with only 'BASE' data, accessors will handle the rest */ + if(_alpm_db_read(db, pkg, INFRQ_BASE) == -1) { + _alpm_log(PM_LOG_ERROR, _("corrupted database entry '%s'\n"), name); + _alpm_pkg_free(pkg); + continue; + } + pkg->origin = PKG_FROM_CACHE; + pkg->origin_data.db = db; + /* add to the collection */ + _alpm_log(PM_LOG_FUNCTION, "adding '%s' to package cache for db '%s'\n", + pkg->name, db->treename); + db->pkgcache = alpm_list_add(db->pkgcache, pkg); + count++; + } + + db->pkgcache = alpm_list_msort(db->pkgcache, count, _alpm_pkg_cmp); + return(count); +} + +int _alpm_db_read(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq) +{ + char line[513]; + char pkgname[PATH_MAX]; + pack_file_t *pf; + ALPM_LOG_FUNC; + + if(db == NULL) { + RET_ERR(PM_ERR_DB_NULL, -1); + } + + if(info == NULL || info->name == NULL || info->version == NULL) { + _alpm_log(PM_LOG_DEBUG, "invalid package entry provided to _alpm_db_read, skipping\n"); + return(-1); + } + + if(info->origin == PKG_FROM_FILE) { + _alpm_log(PM_LOG_DEBUG, "request to read database info for a file-based package '%s', skipping...\n", info->name); + return(-1); + } + + /* bitmask logic here: + * infolevel: 00001111 + * inforeq: 00010100 + * & result: 00000100 + * == to inforeq? nope, we need to load more info. */ + if((info->infolevel & inforeq) == inforeq) { + /* already loaded this info, do nothing */ + return(0); + } + _alpm_log(PM_LOG_FUNCTION, "loading package data for %s : level=0x%x\n", + info->name, inforeq); + + /* clear out 'line', to be certain - and to make valgrind happy */ + memset(line, 0, 513); + snprintf(pkgname, PATH_MAX, "%s-%s", info->name, info->version); + + /* DESC */ + if(inforeq & INFRQ_DESC) { + if((pf = _pack_open_file(db->handle, pkgname, "desc", PO_READ)) == NULL) { + _alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname); + goto error; + } + while(!_pack_file_eof(db->handle, pf)) { + if(_pack_file_gets(db->handle, pf, line, 256) == 0) { + break; + } + _alpm_strtrim(line); + if(strcmp(line, "%NAME%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + if(strcmp(_alpm_strtrim(line), info->name) != 0) { + _alpm_log(PM_LOG_ERROR, _("%s database is inconsistent: name " + "mismatch on package %s %s\n"), db->treename, info->name, line); + } + } else if(strcmp(line, "%VERSION%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + if(strcmp(_alpm_strtrim(line), info->version) != 0) { + _alpm_log(PM_LOG_ERROR, _("%s database is inconsistent: version " + "mismatch on package %s\n"), db->treename, info->name); + } + } else if(strcmp(line, "%FILENAME%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + STRDUP(info->filename, _alpm_strtrim(line), goto error); + } else if(strcmp(line, "%DESC%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + STRDUP(info->desc, _alpm_strtrim(line), goto error); + } else if(strcmp(line, "%GROUPS%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->groups = alpm_list_add(info->groups, linedup); + } + } else if(strcmp(line, "%URL%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + STRDUP(info->url, _alpm_strtrim(line), goto error); + } else if(strcmp(line, "%LICENSE%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->licenses = alpm_list_add(info->licenses, linedup); + } + } else if(strcmp(line, "%ARCH%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + STRDUP(info->arch, _alpm_strtrim(line), goto error); + } else if(strcmp(line, "%BUILDDATE%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + _alpm_strtrim(line); + + char first = tolower(line[0]); + if(first > 'a' && first < 'z') { + struct tm tmp_tm = {0}; //initialize to null incase of failure + setlocale(LC_TIME, "C"); + strptime(line, "%a %b %e %H:%M:%S %Y", &tmp_tm); + info->builddate = mktime(&tmp_tm); + setlocale(LC_TIME, ""); + } else { + info->builddate = atol(line); + } + } else if(strcmp(line, "%INSTALLDATE%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + _alpm_strtrim(line); + + char first = tolower(line[0]); + if(first > 'a' && first < 'z') { + struct tm tmp_tm = {0}; //initialize to null incase of failure + setlocale(LC_TIME, "C"); + strptime(line, "%a %b %e %H:%M:%S %Y", &tmp_tm); + info->installdate = mktime(&tmp_tm); + setlocale(LC_TIME, ""); + } else { + info->installdate = atol(line); + } + } else if(strcmp(line, "%PACKAGER%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + STRDUP(info->packager, _alpm_strtrim(line), goto error); + } else if(strcmp(line, "%REASON%") == 0) { + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + info->reason = atol(_alpm_strtrim(line)); + } else if(strcmp(line, "%SIZE%") == 0 || strcmp(line, "%CSIZE%") == 0) { + /* NOTE: the CSIZE and SIZE fields both share the "size" field + * in the pkginfo_t struct. This can be done b/c CSIZE + * is currently only used in sync databases, and SIZE is + * only used in local databases. + */ + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + info->size = atol(_alpm_strtrim(line)); + /* also store this value to isize if isize is unset */ + if(info->isize == 0) { + info->isize = info->size; + } + } else if(strcmp(line, "%ISIZE%") == 0) { + /* ISIZE (installed size) tag only appears in sync repositories, + * not the local one. */ + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + info->isize = atol(_alpm_strtrim(line)); + } else if(strcmp(line, "%MD5SUM%") == 0) { + /* MD5SUM tag only appears in sync repositories, + * not the local one. */ + if(_pack_file_gets(db->handle, pf, line, 512) == 0) { + goto error; + } + STRDUP(info->md5sum, _alpm_strtrim(line), goto error); + } else if(strcmp(line, "%REPLACES%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->replaces = alpm_list_add(info->replaces, linedup); + } + } else if(strcmp(line, "%FORCE%") == 0) { + info->force = 1; + } + } + _pack_close_file(pf); + pf = NULL; + } + + /* FILES */ + if(inforeq & INFRQ_FILES) { + if((pf = _pack_open_file(db->handle, pkgname, "files", PO_READ)) == NULL) { + _alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname); + goto error; + } + while(_pack_file_gets(db->handle, pf, line, 256) && + strlen(_alpm_strtrim(line))) { + _alpm_strtrim(line); + if(strcmp(line, "%FILES%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->files = alpm_list_add(info->files, linedup); + } + } else if(strcmp(line, "%BACKUP%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->backup = alpm_list_add(info->backup, linedup); + } + } + } + _pack_close_file(pf); + pf = NULL; + } + + /* DEPENDS */ + if(inforeq & INFRQ_DEPENDS) { + if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_READ)) == NULL) { + _alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname); + goto error; + } + while(!_pack_file_eof(db->handle, pf)) { + _pack_file_gets(db->handle, pf, line, 255); + _alpm_strtrim(line); + if(strcmp(line, "%DEPENDS%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + pmdepend_t *dep = _alpm_splitdep(_alpm_strtrim(line)); + info->depends = alpm_list_add(info->depends, dep); + } + } else if(strcmp(line, "%OPTDEPENDS%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->optdepends = alpm_list_add(info->optdepends, linedup); + } + } else if(strcmp(line, "%CONFLICTS%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->conflicts = alpm_list_add(info->conflicts, linedup); + } + } else if(strcmp(line, "%PROVIDES%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + char *linedup; + STRDUP(linedup, _alpm_strtrim(line), goto error); + info->provides = alpm_list_add(info->provides, linedup); + } + } + } + _pack_close_file(pf); + pf = NULL; + } + + /* DELTAS */ + if(inforeq & INFRQ_DELTAS) { + if((pf = _pack_open_file(db->handle, pkgname, "deltas", PO_READ)) != NULL) { + while(!_pack_file_eof(db->handle, pf)) { + _pack_file_gets(db->handle, pf, line, 255); + _alpm_strtrim(line); + if(strcmp(line, "%DELTAS%") == 0) { + while(_pack_file_gets(db->handle, pf, line, 512) && + strlen(_alpm_strtrim(line))) { + pmdelta_t *delta = _alpm_delta_parse(line); + if(delta) { + info->deltas = alpm_list_add(info->deltas, delta); + } + } + } + } + _pack_close_file(pf); + pf = NULL; + } + } + /* INSTALL */ + if(inforeq & INFRQ_SCRIPTLET) { + if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_READ)) != NULL) { + info->scriptlet = 1; + _pack_close_file(pf); + } + } + + /* internal */ + info->infolevel |= inforeq; + + return(0); + +error: + if(pf) { + _pack_close_file(pf); + } + return(-1); +} + +int _alpm_db_write(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq) +{ + char pkgname[PATH_MAX]; + alpm_list_t *lp = NULL; + int retval = 0; + int local = 0; + pack_file_t *pf; + char buffer[1024]; + size_t len; + + ALPM_LOG_FUNC; + + if(db == NULL || info == NULL) { + return(-1); + } + + if(strcmp(db->treename, "local") == 0) { + local = 1; + } + + snprintf(pkgname, PATH_MAX, "%s-%s", info->name, info->version); + + /* DESC */ + if(inforeq & INFRQ_DESC) { + _alpm_log(PM_LOG_DEBUG, "writing %s-%s DESC information back to db\n", + info->name, info->version); + if((pf = _pack_open_file(db->handle, pkgname, "desc", PO_WRITE)) == NULL) { + _alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname); + retval = -1; + goto cleanup; + } + + len = snprintf(buffer, 1024, "%%NAME%%\n%s\n\n" + "%%VERSION%%\n%s\n\n", info->name, info->version); + _pack_file_puts(db->handle, pf, buffer, len); + if(info->desc) { + len = snprintf(buffer, 1024, "%%DESC%%\n" + "%s\n\n", info->desc); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->groups) { + _pack_file_puts(db->handle, pf, "%GROUPS%\n", 9); + for(lp = info->groups; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + _pack_file_puts(db->handle, pf, "\n", 1); + } + if(info->replaces) { + _pack_file_puts(db->handle, pf, "%REPLACES%\n", 11); + for(lp = info->replaces; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->force) { + len = snprintf(buffer, 1024, "%%FORCE%%\n\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(local) { + if(info->url) { + len = snprintf(buffer, 1024, "%%URL%%\n" + "%s\n\n", info->url); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->licenses) { + _pack_file_puts(db->handle, pf, "%LICENSE%\n", 10); + for(lp = info->licenses; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->arch) { + len = snprintf(buffer, 1024, "%%ARCH%%\n" + "%s\n\n", info->arch); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->builddate) { + len = snprintf(buffer, 1024, "%%BUILDDATE%%\n" + "%ju\n\n", (uintmax_t)info->builddate); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->installdate) { + len = snprintf(buffer, 1024, "%%INSTALLDATE%%\n" + "%ju\n\n", (uintmax_t)info->installdate); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->packager) { + len = snprintf(buffer, 1024, "%%PACKAGER%%\n" + "%s\n\n", info->packager); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->isize) { + /* only write installed size, csize is irrelevant once installed */ + len = snprintf(buffer, 1024, "%%SIZE%%\n" + "%ju\n\n", (intmax_t)info->isize); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->reason) { + len = snprintf(buffer, 1024, "%%REASON%%\n" + "%u\n\n", info->reason); + _pack_file_puts(db->handle, pf, buffer, len); + } + } else { + if(info->size) { + len = snprintf(buffer, 1024, "%%CSIZE%%\n" + "%ju\n\n", (intmax_t)info->size); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->isize) { + len = snprintf(buffer, 1024, "%%ISIZE%%\n" + "%ju\n\n", (intmax_t)info->isize); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->md5sum) { + len = snprintf(buffer, 1024, "%%MD5SUM%%\n" + "%s\n\n", info->md5sum); + _pack_file_puts(db->handle, pf, buffer, len); + } + } + _pack_close_file(pf); + pf = NULL; + } + + /* FILES */ + if(local && (inforeq & INFRQ_FILES)) { + _alpm_log(PM_LOG_DEBUG, "writing %s-%s FILES information back to db\n", + info->name, info->version); + if((pf = _pack_open_file(db->handle, pkgname, "files", PO_WRITE)) == NULL) { + _alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname); + retval = -1; + goto cleanup; + } + + if(info->files) { + len = snprintf(buffer, 1024, "%%FILES%%\n"); + _pack_file_puts(db->handle, pf, buffer, len); + for(lp = info->files; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->backup) { + len = snprintf(buffer, 1024, "%%BACKUP%%\n"); + _pack_file_puts(db->handle, pf, buffer, len); + for(lp = info->backup; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + _pack_close_file(pf); + pf = NULL; + } + + /* DEPENDS */ + if(inforeq & INFRQ_DEPENDS) { + _alpm_log(PM_LOG_DEBUG, "writing %s-%s DEPENDS information back to db\n", + info->name, info->version); + if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_WRITE)) == NULL) { + _alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname); + retval = -1; + goto cleanup; + } + + if(info->depends) { + _pack_file_puts(db->handle, pf, "%DEPENDS%\n", 10); + for(lp = info->depends; lp; lp = lp->next) { + char *depstring = alpm_dep_get_string(lp->data); + len = snprintf(buffer, 1024, "%s\n", depstring); + _pack_file_puts(db->handle, pf, buffer, len); + free(depstring); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->optdepends) { + _pack_file_puts(db->handle, pf, "%OPTDEPENDS%\n", 13); + for(lp = info->optdepends; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->conflicts) { + _pack_file_puts(db->handle, pf, "%CONFLICTS%\n", 12); + for(lp = info->conflicts; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + if(info->provides) { + _pack_file_puts(db->handle, pf, "%PROVIDES%\n", 11); + for(lp = info->provides; lp; lp = lp->next) { + len = snprintf(buffer, 1024, "%s\n", (char *)lp->data); + _pack_file_puts(db->handle, pf, buffer, len); + } + len = snprintf(buffer, 1024, "\n"); + _pack_file_puts(db->handle, pf, buffer, len); + } + _pack_close_file(pf); + pf = NULL; + } + + /* INSTALL */ + /* nothing needed here (script is automatically extracted) */ + +cleanup: + if(pf) { + _pack_close_file(pf); + } + + return(retval); +} + +int _alpm_db_remove(pmdb_t *db, pmpkg_t *info) +{ + int ret = 0; + char pkgname[512]; + + ALPM_LOG_FUNC; + + snprintf(pkgname, 512, "%s-%s", info->name, info->version); + _pack_package_remove(db->handle, pkgname); + + if(db == NULL || info == NULL) { + RET_ERR(PM_ERR_DB_NULL, -1); + } + + return(ret); +} + +void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg) +{ + char name[512]; + snprintf(name, 512, "%s-%s", pkg->name, pkg->version); + return _pack_open_file(db->handle, name, "changelog", PO_READ); +} + +int _alpm_db_changelog_close(pmdb_t *db, void *stream) +{ + _pack_close_file(stream); + return 0; +} + +size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size) +{ + return _pack_file_gets(db->handle, pf, ptr, size); +} + +/* vim: set ts=2 sw=2 noet: */ diff --git a/lib/libalpm/be_packed.h b/lib/libalpm/be_packed.h new file mode 100644 index 0000000..e118a3a --- /dev/null +++ b/lib/libalpm/be_packed.h @@ -0,0 +1,757 @@ +/* + * Copyright (c) 2008 Sivert Berg + * + * Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + */ + +/* _HOW IT WORKS_ + * + * If you've ever looked at the ext2 filesystem you will find alot + * of similarities (among others the inode that I didn't have the + * imagination to find a new name for). + * + * _GROUP_ + * Basicly the database is divided into groups. Each group got the + * same number of blocks and inodes. + * + * _INODE_ + * Each file is assigned an unique inode. The inode got a list of + * blocks where the actual data for the file is stored. Worth noting + * is that the last entry in the block list is indirect. That means + * it points to a block that points to blocks. + */ + +#include <stdint.h> +#include <sys/mman.h> +#include <stdlib.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <unistd.h> + + +#ifndef BE_PACKED_H +#define BE_PACKED_H + +#define BLOCK_SIZE 256 +#define GROUP_SIZE (BLOCK_SIZE * 4096) +#define BLOCKS_PER_GROUP 3936 +#define INODES_PER_GROUP 1024 + +#define MIN(a, b) (((a) < (b))?(a):(b)) + +/* _pack_open_file flags */ +#define PO_READ 1 +#define PO_WRITE 2 + + +typedef struct __inode_t { + int32_t size; + int32_t block[7]; +} inode_t; + +typedef struct __group_desc_t { + uint32_t inodes_free; + uint32_t blocks_free; + uint32_t checksum; + inode_t i_table[INODES_PER_GROUP]; + uint32_t bitmap[BLOCKS_PER_GROUP / 32]; + char data[]; +} group_desc_t; + +typedef struct __package_t { + int32_t inode; + uint16_t rec_len; + uint16_t name_len; + char name[]; +} package_t; + +typedef struct __pack_db_t { + int fd; + int groups; + int read_only; + group_desc_t *map; +} pack_db_t; + +typedef struct __pack_file_t { + int inode; + size_t pos; +} pack_file_t; + +typedef struct __pack_dir_t { + size_t pos; +} pack_dir_t; + + +/** + ** GROUP FUNCTIONS + */ +/* Initilizes a new group */ +void _pack_init_group(group_desc_t *group) +{ + int i, j; + + for (i = 0; i < INODES_PER_GROUP; i++) { + group->i_table[i].size = -1; + for (j = 0; j < 7; j++) + group->i_table[i].block[j] = -1; + } + + for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) { + group->bitmap[i] = 0; + } + + group->inodes_free = INODES_PER_GROUP; + group->blocks_free = BLOCKS_PER_GROUP; + group->checksum = 0; +} + + +/* Adds a new group to the file */ +void _pack_new_group(pack_db_t *db) +{ + db->groups++; + ftruncate(db->fd, db->groups * GROUP_SIZE); + db->map = mremap(db->map, (db->groups - 1) * GROUP_SIZE, + db->groups * GROUP_SIZE, MREMAP_MAYMOVE); + + _pack_init_group((group_desc_t*)((char*)db->map + + GROUP_SIZE * (db->groups - 1))); +} + + +/** + ** POINTER FUNCTIONS (return pointers to various structures) + */ +group_desc_t *_pack_get_group(pack_db_t *db, int group) +{ + return (group_desc_t*)((char*)db->map + group*GROUP_SIZE); +} + + +inode_t *_pack_get_inode(pack_db_t *db, int inode) +{ + int group = inode / INODES_PER_GROUP; + group_desc_t *groupd = _pack_get_group(db, group); + inode %= INODES_PER_GROUP; + + return groupd->i_table + inode; +} + +void *_pack_get_block(pack_db_t *db, int block) +{ + if (block == -1) + return NULL; + + group_desc_t *group = _pack_get_group(db, block / BLOCKS_PER_GROUP); + return group->data + (block % BLOCKS_PER_GROUP) * BLOCK_SIZE; +} + + +/* Searches for a free block in the bitmap and marks the bit + * it finds as used. + */ +int _pack_check_bitmap(group_desc_t *group) +{ + int i = 0, j = 0, t = 0; + + for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) { + if (group->bitmap[i] != 0xffffffff) { + t = group->bitmap[i]; + for (j = 0; j < 32 && (t & 1); j++, t >>= 1); + break; + } + } + + group->bitmap[i] |= 1 << j; + return i * 32 + j; +} + + +/* Clears the n'th bit */ +int _pack_bitmap_free(group_desc_t *group, int n) +{ + int i = n / 32; + int j = n % 32; + + group->bitmap[i] &= ~(i << j); + return 0; +} + + +/* Returns the number of a new unused block + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_get_free_block(pack_db_t *db) +{ + int i; + group_desc_t *group; + int ret; + + for (i = 0; i < db->groups; i++) { + group = _pack_get_group(db, i); + if (group->blocks_free) { + group->blocks_free--; + ret = _pack_check_bitmap(group); + ret += i * BLOCKS_PER_GROUP; + return ret;; + } + } + + _pack_new_group(db); + group = _pack_get_group(db, i); + group->blocks_free--; + ret = _pack_check_bitmap(group); + ret += i * BLOCKS_PER_GROUP; + + return ret; +} + + +/* Frees the block given */ +int _pack_block_free(pack_db_t *db, int block) +{ + group_desc_t *group; + int g, b; + + g = block / BLOCKS_PER_GROUP; + b = block & BLOCKS_PER_GROUP; + + group = _pack_get_group(db, g); + group->blocks_free++; + + return _pack_bitmap_free(group, b); +} + + +/* Returns the number of a new free inode. + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_get_free_inode(pack_db_t *db) +{ + int i = 0, j = 0; + group_desc_t *group; + + for (i = 0; i < db->groups; i++) { + group = _pack_get_group(db, i); + if (group->inodes_free) { + group->inodes_free--; + for (j = 0; j < INODES_PER_GROUP; j++) { + if (group->i_table[j].size == -1) { + group->i_table[j].size = 0; + return i * INODES_PER_GROUP + j; + } + } + } + } + + _pack_new_group(db); + group = _pack_get_group(db, i); + group->inodes_free--; + + return i * INODES_PER_GROUP; +} + + +/* Frees an inode and all of the blocks it might be using */ +int _pack_inode_free(pack_db_t *db, int in) +{ + inode_t *inode = _pack_get_inode(db, in); + group_desc_t *group = _pack_get_group(db, in / INODES_PER_GROUP); + int i; + int block; + int32_t *blocks; + int size = BLOCK_SIZE / sizeof(uint32_t) - 1; + + inode->size = -1; + + for (i = 0; i < 6; i++) { + if (inode->block[i] != -1) { + _pack_block_free(db, inode->block[i]); + inode->block[i] = -1; + } + } + + block = inode->block[6]; + blocks = _pack_get_block(db, block); + + while(blocks != NULL) { + for (i = 0; i < size; i++) { + if (blocks[i] != -1) + _pack_block_free(db, blocks[i]); + } + + _pack_block_free(db, block); + block = blocks[size]; + blocks = _pack_get_block(db, block); + } + + group->inodes_free++; + + return 0; +} + + +/* Returns the number of the block associated with the inode */ +int _pack_inode_get_block(pack_db_t *db, int i, int block) +{ + inode_t *inode = _pack_get_inode(db, i); + if (block < 6) + return inode->block[block]; + + block -= 6; + int size = BLOCK_SIZE / sizeof(uint32_t) - 1; + uint32_t *blocks = _pack_get_block(db, inode->block[6]); + + for (;blocks != NULL && block >= size; block -= size) + blocks = _pack_get_block(db, blocks[size]); + + if (blocks == NULL) + return -1; + + return blocks[block]; +} + + +/* Returns the number of the block block associated with the inode. + * If there is no such block, it gets allocated. + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_inode_add_block(pack_db_t *db, int in, int block) +{ + inode_t *inode = _pack_get_inode(db, in); + int tmp; + + if (block < 6) { + if (inode->block[block] != -1) + return inode->block[block]; + + tmp = _pack_get_free_block(db); + inode = _pack_get_inode(db, in); + return inode->block[block] = tmp; + } + + block -= 6; + int size = BLOCK_SIZE / sizeof(int32_t) - 1; + int i; + int old_block; + int32_t *blocks; + + if (inode->block[6] == -1) { + tmp = _pack_get_free_block(db); + inode = _pack_get_inode(db, in); + inode->block[6] = tmp; + blocks = _pack_get_block(db, inode->block[6]); + for (i = 0; i < size + 1; i++) + blocks[i] = -1; + } + else + blocks = _pack_get_block(db, inode->block[6]); + + old_block = inode->block[6]; + + for (;block >= size; block -= size) { + if (blocks[size] == -1) { + tmp = _pack_get_free_block(db); + blocks = _pack_get_block(db, old_block); + blocks[size] = tmp; + old_block = tmp; + blocks = _pack_get_block(db, tmp); + for (i = 0; i < size + 1; i++) + blocks[i] = -1; + } + else { + old_block = blocks[size]; + blocks = _pack_get_block(db, blocks[size]); + } + } + + if (blocks[block] == -1) { + tmp = _pack_get_free_block(db); + blocks = _pack_get_block(db, old_block); + blocks[block] = tmp; + } + + return blocks[block]; +} + +/* Returns a pointer to the package with the name pkgname. + * Returns NULL if it's not found + */ +package_t *_pack_find_package(pack_db_t *db, const char *pkgname) +{ + inode_t *inode = _pack_get_inode(db, 0); + int s_len = strlen(pkgname); + package_t *pkg; + int block = 0; + int b_pos = 0; + + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + + while (block < (inode->size / BLOCK_SIZE)) { + if (pkg->inode != 0 && + pkg->name_len == s_len && + strncmp(pkgname, pkg->name, s_len) == 0) + break; + + b_pos += pkg->rec_len; + pkg = (package_t*)((char*)pkg + pkg->rec_len); + if (b_pos >= BLOCK_SIZE) { + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block)); + b_pos = 0; + } + } + + return pkg; +} + +/* Creates a new package with the name pkgname. + * Returns a pointer to the new package. + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +package_t *_pack_add_package(pack_db_t *db, const char *pkgname) +{ + inode_t *inode = _pack_get_inode(db, 0); + int s_len = strlen(pkgname); + int len = s_len + sizeof(package_t); + package_t *pkg; + int block = 0; + int b_pos = 0; + + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + + while (pkg && block < (inode->size / BLOCK_SIZE)) { + if (pkg->rec_len >= len && pkg->inode == 0) + break; + + b_pos += pkg->rec_len; + pkg = (package_t*)((char*)pkg + pkg->rec_len); + if (b_pos >= BLOCK_SIZE) { + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block)); + b_pos = 0; + } + } + + if (pkg == NULL) { + pkg = _pack_get_block(db, _pack_inode_add_block(db, 0, block)); + pkg->rec_len = BLOCK_SIZE; + pkg->inode = 0; + inode = _pack_get_inode(db, 0); + inode->size += BLOCK_SIZE; + } + + strncpy(pkg->name, pkgname, s_len); + if (pkg->rec_len > (len + sizeof(package_t) + 10)) { + package_t *foo = (package_t*)((char*)pkg + len); + foo->rec_len = pkg->rec_len - len; + foo->inode = 0; + pkg->rec_len = len; + } + + pkg->inode = -1; + pkg->name_len = s_len; + + return pkg; +} + + +/* Reads data from the file until buf is full or we hit a newline */ +int _pack_file_gets(pack_db_t *db, pack_file_t *file, char *buf, size_t size) +{ + inode_t *inode = _pack_get_inode(db, file->inode); + size_t i; + size_t b_pos = file->pos % BLOCK_SIZE; + const char *src = _pack_get_block(db, + _pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE)); + int newline = 0; + + for (i = 0 + ;i < size && file->pos < inode->size && !newline + ;i++, b_pos++, file->pos++) { + if (b_pos >= BLOCK_SIZE) { + src = _pack_get_block(db, + _pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE)); + b_pos = 0; + } + + buf[i] = src[b_pos]; + + if (src[b_pos] == '\n') + newline = 1; + } + + if (i < size - 1) + buf[i] = '\0'; + + return i; +} + + +/* Write the data in buf into the file + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +int _pack_file_puts(pack_db_t *db, pack_file_t *file, + const char *buf, size_t size) +{ + size_t i; + size_t b_pos = file->pos % BLOCK_SIZE; + char *dest = _pack_get_block(db, + _pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE)); + int block; + + for (i = 0; i < size; i++, file->pos++, b_pos++) + { + if (b_pos >= BLOCK_SIZE) { + block = _pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE); + dest = _pack_get_block(db, block); + b_pos = 0; + } + dest[b_pos] = buf[i]; + } + + inode_t *inode = _pack_get_inode(db, file->inode); + inode->size = file->pos; + + return i; +} + + +int _pack_file_eof(pack_db_t *db, pack_file_t *file) +{ + inode_t *inode = _pack_get_inode(db, file->inode); + return file->pos >= inode->size; +} + + +/* Opens a file in the database + * flags: if you pass PO_READ, it will return -1 + * if the file does not exist, if you pass PO_WRITE + * the file will be created if it does not exist + * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP! + * that means ANY pointers into the database may be invalid + * after you've called this function. + */ +pack_file_t *_pack_open_file(pack_db_t *db, const char *pkgname, + const char *filename, int flags) +{ + char name[512]; + + snprintf(name, 512, "%s/%s", pkgname, filename); + + package_t *pkg = _pack_find_package(db, name); + int w = flags & PO_WRITE; + int tmp; + + if (w && db->read_only) + return NULL; + + if (pkg == NULL && w) + pkg = _pack_add_package(db, name); + else if (pkg == NULL) + return NULL; + + pack_file_t *ret = malloc(sizeof(pack_file_t)); + ret->pos = 0; + + if (pkg->inode == -1 && w) { + tmp = _pack_get_free_inode(db); + pkg = _pack_find_package(db, name); + ret->inode = pkg->inode = tmp; + } + else + ret->inode = pkg->inode; + + return ret; +} + + +/* Closes the file given */ +void _pack_close_file(pack_file_t *file) +{ + free(file); +} + + +/* Removes a package along with all it's files */ +int _pack_package_remove(pack_db_t *db, const char *pkgname) +{ + inode_t *inode = _pack_get_inode(db, 0); + int s_len = strlen(pkgname); + package_t *prev, *pkg; + int block = 0; + int b_pos = 0; + + if (db->read_only) + return -1; + + prev = NULL; + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + + while (block < (inode->size / BLOCK_SIZE)) { + if (pkg->inode != 0 && + pkg->name_len > s_len && + strncmp(pkgname, pkg->name, s_len) == 0 && + pkg->name[s_len] == '/') { + _pack_inode_free(db, pkg->inode); + + if (prev != NULL && prev->inode == 0) { + prev->rec_len += pkg->rec_len; + pkg = prev; + } + else + pkg->inode = 0; + } + + prev = pkg; + b_pos += pkg->rec_len; + pkg = (package_t*)((char*)pkg + pkg->rec_len); + if (b_pos >= BLOCK_SIZE) { + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block)); + b_pos = 0; + prev = NULL; + } + } + + return 0; +} + + +/* Gets the next entry in the directory */ +int _pack_dir_next(pack_db_t *db, pack_dir_t *dir, char *name, size_t len) +{ + inode_t *inode = _pack_get_inode(db, 0); + + if (dir->pos >= inode->size) + return -1; + + int block, b_pos; + package_t *pkg; + + do { + block = dir->pos / BLOCK_SIZE; + b_pos = dir->pos % BLOCK_SIZE; + pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block)); + if (!pkg) + return -1; + pkg = (package_t*)((char*)pkg + b_pos); + dir->pos += pkg->rec_len; + } while (pkg->inode == 0); + + strncpy(name, pkg->name, MIN(len, pkg->name_len)); + if (len > pkg->name_len) + name[pkg->name_len] = '\0'; + + return 0; +} + + +/* Opens the directory for reading */ +pack_dir_t *_pack_open_dir(pack_db_t *db) +{ + pack_dir_t *dir = malloc(sizeof(pack_dir_t)); + dir->pos = 0; + + return dir; +} + + +/* Closes the given directory */ +void _pack_close_dir(pack_dir_t *dir) +{ + free(dir); +} + + +/* Opens a packed database */ +pack_db_t *_pack_db_open(const char *filename) +{ + struct stat st; + int is_new = 0; + + pack_db_t *db = malloc(sizeof(pack_db_t)); + db->fd = open(filename, O_RDWR, 0); + db->read_only = 0; + + if (db->fd == -1) { + db->fd = open(filename, O_RDONLY, 0); + if (db->fd == -1) { + db->fd = open(filename, O_RDWR | O_CREAT, 0644); + + if (db->fd == -1) { + free(db); + printf("Could not open %s\n", filename); + return NULL; + } + + ftruncate(db->fd, GROUP_SIZE); + is_new = 1; + } + else + db->read_only = 1; + } + + fstat(db->fd, &st); + db->groups = st.st_size / GROUP_SIZE; + + int flags = PROT_READ; + if (!db->read_only) + flags |= PROT_WRITE; + + db->map = mmap(NULL, GROUP_SIZE * db->groups, flags, MAP_SHARED, + db->fd, 0); + + if (db->map == MAP_FAILED) { + perror("mmap"); + free(db); + exit(0); + } + + if (is_new) { + _pack_init_group(db->map); + db->map->i_table[0].size = 0; + } + + return db; +} + + +/* Closes a packed database */ +void _pack_db_close(pack_db_t *db) +{ + munmap(db->map, db->groups * GROUP_SIZE); + close(db->fd); + free(db); +} + +#endif /* BE_PACKED_H */ diff --git a/lib/libalpm/db.h b/lib/libalpm/db.h index 96fac0d..96291fc 100644 --- a/lib/libalpm/db.h +++ b/lib/libalpm/db.h @@ -62,6 +62,9 @@ int _alpm_db_populate(pmdb_t *db); int _alpm_db_read(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq); int _alpm_db_write(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq); int _alpm_db_remove(pmdb_t *db, pmpkg_t *info); +void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg); +int _alpm_db_changelog_close(pmdb_t *db, void *stream); +size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size); #endif /* _ALPM_DB_H */ diff --git a/lib/libalpm/package.c b/lib/libalpm/package.c index ee0ff6f..7ab9710 100644 --- a/lib/libalpm/package.c +++ b/lib/libalpm/package.c @@ -447,13 +447,7 @@ void SYMEXPORT *alpm_pkg_changelog_open(pmpkg_t *pkg) ASSERT(pkg != NULL, return(NULL)); if(pkg->origin == PKG_FROM_CACHE) { - char clfile[PATH_MAX]; - snprintf(clfile, PATH_MAX, "%s/%s/%s-%s/changelog", - alpm_option_get_dbpath(), - alpm_db_get_name(handle->db_local), - alpm_pkg_get_name(pkg), - alpm_pkg_get_version(pkg)); - return fopen(clfile, "r"); + return(_alpm_db_changelog_open(handle->db_local, pkg)); } else if(pkg->origin == PKG_FROM_FILE) { struct archive *archive = NULL; struct archive_entry *entry; @@ -500,7 +494,7 @@ size_t SYMEXPORT alpm_pkg_changelog_read(void *ptr, size_t size, { size_t ret = 0; if(pkg->origin == PKG_FROM_CACHE) { - ret = fread(ptr, 1, size, (FILE*)fp); + ret = _alpm_db_changelog_read(handle->db_local, fp, ptr, size); } else if(pkg->origin == PKG_FROM_FILE) { ret = archive_read_data((struct archive*)fp, ptr, size); } @@ -533,7 +527,7 @@ int SYMEXPORT alpm_pkg_changelog_close(const pmpkg_t *pkg, void *fp) { int ret = 0; if(pkg->origin == PKG_FROM_CACHE) { - ret = fclose((FILE*)fp); + ret = _alpm_db_changelog_close(handle->db_local, fp); } else if(pkg->origin == PKG_FROM_FILE) { ret = archive_read_finish((struct archive *)fp); } diff --git a/src/util/Makefile.am b/src/util/Makefile.am index 97a0ffa..64e18ad 100644 --- a/src/util/Makefile.am +++ b/src/util/Makefile.am @@ -3,7 +3,7 @@ conffile = ${sysconfdir}/pacman.conf dbpath = ${localstatedir}/lib/pacman/ cachedir = ${localstatedir}/cache/pacman/pkg/ -bin_PROGRAMS = vercmp testpkg testdb +bin_PROGRAMS = vercmp testpkg testdb convdb DEFS = -DLOCALEDIR=\"@localedir@\" \ -DCONFFILE=\"$(conffile)\" \ @@ -18,6 +18,8 @@ AM_CFLAGS = -pedantic -D_GNU_SOURCE vercmp_SOURCES = vercmp.c vercmp_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la +convdb_SOURCES = convdb.c + testpkg_SOURCES = testpkg.c testpkg_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la diff --git a/src/util/convdb.c b/src/util/convdb.c new file mode 100644 index 0000000..f1f39bd --- /dev/null +++ b/src/util/convdb.c @@ -0,0 +1,236 @@ +/* + * Copyright (c) 2008 Sivert Berg + * + * Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + */ + + +/* WARNING: This file is pretty much just a quick hack thrown together + * in a hurry. It seems to work, but don't take my word for it. + */ + +#include <be_packed.h> +#include <dirent.h> +#include <archive.h> +#include <archive_entry.h> + + +/* Convert an archieved database into a packed database + * Expects a gziped database ending in .db.tar.gz + */ +void convert_archive(const char *filename) +{ + struct archive_entry *aent = NULL; + struct archive *arch = archive_read_new(); + archive_read_support_compression_gzip(arch); + archive_read_support_format_tar(arch); + if (archive_read_open_filename(arch, filename, 1) != ARCHIVE_OK) { + printf("Could not open archive %s\n", archive_error_string(arch)); + return; + } + + char *db_name = strndup(filename, strlen(filename) - strlen(".tar.gz")); + pack_db_t *db = _pack_db_open(db_name); + free(db_name); + char buffer[4096]; + size_t size; + pack_file_t *pf; + + while (!archive_read_next_header(arch, &aent)) { + const char *type = NULL; + if (strstr(archive_entry_pathname(aent), "/desc")) + type = "desc"; + else if (strstr(archive_entry_pathname(aent), "/depends")) + type = "depends"; + else if (strstr(archive_entry_pathname(aent), "/install")) + type = "install"; + else if (strstr(archive_entry_pathname(aent), "/files")) + type = "files"; + + if (type == NULL) + continue; + + char *name = strdup(archive_entry_pathname(aent)); + int i; + for (i = 0; name[i] != '/' && name[i]; i++); + name[i] = '\0'; + + pf = _pack_open_file(db, name, type, PO_WRITE); + + free(name); + + do { + size = archive_read_data(arch, buffer, 4096); + _pack_file_puts(db, pf, buffer, size); + } while (size == 4096); + + _pack_close_file(pf); + + } + + _pack_db_close(db); + archive_read_close(arch); + archive_read_finish(arch); +} + + + +const char *dir_name(const char *full_path) +{ + const char *slash = full_path; + while (*full_path) + if (*full_path++ == '/') + slash = full_path; + return slash; +} + + +/* Convert a directory database into a packed database */ +void convert_directory(const char *pathname) +{ + char db_name[1024]; + pack_db_t *db; + DIR *dir = opendir(pathname); + struct dirent *dire; + char path[1024]; + char buffer[8196]; + size_t size; + FILE *f; + pack_file_t *pf; + + strcpy(db_name, dir_name(pathname)); + strcat(db_name, ".db"); + db = _pack_db_open(db_name); + + if (dir == NULL) { + perror("opendir"); + return; + } + + dire = readdir(dir); + while (dire != NULL) { + if (strcmp(dire->d_name, ".") && strcmp(dire->d_name, "..")) { + + + size_t foo = 0; + sprintf(path, "%s/%s/desc", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto depends; + pf = _pack_open_file(db, dire->d_name, "desc", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +depends: + sprintf(path, "%s/%s/depends", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto files; + pf = _pack_open_file(db, dire->d_name, "depends", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +files: + sprintf(path, "%s/%s/files", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto install; + pf = _pack_open_file(db, dire->d_name, "files", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +install: + sprintf(path, "%s/%s/install", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto changelog; + pf = _pack_open_file(db, dire->d_name, "install", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +changelog: + sprintf(path, "%s/%s/changelog", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto deltas; + pf = _pack_open_file(db, dire->d_name, "changelog", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +deltas: + sprintf(path, "%s/%s/deltas", pathname, dire->d_name); + f = fopen(path, "r"); + if (f == NULL) + goto done; + pf = _pack_open_file(db, dire->d_name, "deltas", PO_WRITE); + do { + size = fread(buffer, 1, 8196, f); + _pack_file_puts(db, pf, buffer, size); + foo += size; + } while (size == 8196); + _pack_close_file(pf); + fclose(f); +done: + pf = NULL; + } + + dire = readdir(dir); + } + + closedir(dir); + _pack_db_close(db); +} + +int main(int argc, char *argv[]) +{ + if (argc < 2) { + printf("Usage: %s <dir/file>\n", argv[0]); + exit(0); + } + + if (strstr(argv[1], ".tar.gz") != NULL) + convert_archive(argv[1]); + else + convert_directory(argv[1]); + + return 0; +} -- 1.6.0.5

3 4

[pacman-dev] [PATCH] makepkg: ensure PKGBUILD does not contain CRLF characters
by Dan McGee 16 Dec '08

16 Dec '08

Do a simple check before sourcing the file to ensure we are a valid bash script. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- scripts/makepkg.sh.in | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index b889625..179746d 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1354,6 +1354,12 @@ if [ ! -f "$BUILDSCRIPT" ]; then # PKGBUILD passed through a pipe BUILDSCRIPT=/dev/stdin fi +else + crlftest=$(file $BUILDSCRIPT | grep -F 'CRLF' || true) + if [ "$crlftest" != "" ]; then + error "$(gettext "%s contains CRLF characters and cannot be sourced.")" "$BUILDSCRIPT" + exit 1 + fi fi source "$BUILDSCRIPT" -- 1.6.0.4

4 3

[pacman-dev] GPG work
by Dan McGee 15 Dec '08

15 Dec '08

I did quite a bit more work with GPG today. I wrapped my head around GPGME, which presents a nice C interface to the GPG stuff so we are now a lot closer to a working implementation: http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/n… >From the script side of things, I didn't change much. The libalpm code has changed considerably, and there is still a lot of room for improvement. Let me know if you guys have questions. -Dan

8 13