[RFC][WIP][PATCH 0/5] adding extended package data
ALPM is currently parsing several fields that are not particularly relevant to managing package installation (e.g. pkgbase, pkgtype, makedepends, checkdepends). pkgtype in particular is not even exposed in any way through the API, it has to be included in the parser with a "not actually used" comment. This patchset adds an extended data field to packages that can be used by packagers or package-building infrastructure to store arbitrary package data in a way that doesn't require altering alpm for every field and allows the data to actually be retrieved via alpm (though not as easily as first-class fields). Andrew Gregory (5): query: only strip leading local/ for db packages query: allow querying extra info for package files add data field for arbitrary package data query: print extended data when extra info is requested makepkg: store package type in extended data lib/libalpm/alpm.h | 7 +++++++ lib/libalpm/be_local.c | 25 +++++++++++++++++++++++++ lib/libalpm/be_package.c | 6 ++++++ lib/libalpm/be_sync.c | 16 ++++++++++++++++ lib/libalpm/package.c | 17 +++++++++++++++++ lib/libalpm/package.h | 6 ++++++ scripts/makepkg.sh.in | 2 +- src/pacman/package.c | 13 +++++++++++++ src/pacman/query.c | 16 ++++++---------- 9 files changed, 97 insertions(+), 11 deletions(-) -- 2.35.0
Signed-off-by: Andrew Gregory
Signed-off-by: Andrew Gregory
This adds a mechanism for package builders to add arbitrary data to
packages that is not necessarily relevant to the package installation
process to gain first-class support in alpm. Currently these fields
have to be added to parsers with a "not actually used" comment and can't
be retrieved through the API.
In ini format these are represented by prefixing the field name with
"data-" similar to HTML5 data attributes. In db format they are put
into a "%DATA%" section in the format "<key> <value>".
Signed-off-by: Andrew Gregory
Signed-off-by: Andrew Gregory
Signed-off-by: Andrew Gregory
On 7/2/22 04:30, Andrew Gregory wrote:
ALPM is currently parsing several fields that are not particularly relevant to managing package installation (e.g. pkgbase, pkgtype, makedepends, checkdepends). pkgtype in particular is not even exposed in any way through the API, it has to be included in the parser with a "not actually used" comment. This patchset adds an extended data field to packages that can be used by packagers or package-building infrastructure to store arbitrary package data in a way that doesn't require altering alpm for every field and allows the data to actually be retrieved via alpm (though not as easily as first-class fields).
Andrew Gregory (5): query: only strip leading local/ for db packages query: allow querying extra info for package files add data field for arbitrary package data query: print extended data when extra info is requested makepkg: store package type in extended data
This looks a good idea to me. Nothing stood out on a quick skim of the patches. I assume patches 1 and 2 could be committed now?
On 03/06/22 at 05:17pm, Allan McRae wrote:
On 7/2/22 04:30, Andrew Gregory wrote:
ALPM is currently parsing several fields that are not particularly relevant to managing package installation (e.g. pkgbase, pkgtype, makedepends, checkdepends). pkgtype in particular is not even exposed in any way through the API, it has to be included in the parser with a "not actually used" comment. This patchset adds an extended data field to packages that can be used by packagers or package-building infrastructure to store arbitrary package data in a way that doesn't require altering alpm for every field and allows the data to actually be retrieved via alpm (though not as easily as first-class fields).
Andrew Gregory (5): query: only strip leading local/ for db packages query: allow querying extra info for package files add data field for arbitrary package data query: print extended data when extra info is requested makepkg: store package type in extended data
This looks a good idea to me. Nothing stood out on a quick skim of the patches.
One bikeshed question: I used what I felt was the most obvious storage for each of our formats, should they be unified? I.e. should .PKGINFO use something like "data = name value" instead of "data-name = value" to more closely match desc files?
I assume patches 1 and 2 could be committed now?
Yes.
On 7/3/22 13:44, Andrew Gregory wrote:
On 03/06/22 at 05:17pm, Allan McRae wrote:
On 7/2/22 04:30, Andrew Gregory wrote:
ALPM is currently parsing several fields that are not particularly relevant to managing package installation (e.g. pkgbase, pkgtype, makedepends, checkdepends). pkgtype in particular is not even exposed in any way through the API, it has to be included in the parser with a "not actually used" comment. This patchset adds an extended data field to packages that can be used by packagers or package-building infrastructure to store arbitrary package data in a way that doesn't require altering alpm for every field and allows the data to actually be retrieved via alpm (though not as easily as first-class fields).
Andrew Gregory (5): query: only strip leading local/ for db packages query: allow querying extra info for package files add data field for arbitrary package data query: print extended data when extra info is requested makepkg: store package type in extended data
This looks a good idea to me. Nothing stood out on a quick skim of the patches.
One bikeshed question: I used what I felt was the most obvious storage for each of our formats, should they be unified? I.e. should .PKGINFO use something like "data = name value" instead of "data-name = value" to more closely match desc files?
I'd prefer .PKGINFO as "data = name value". And we can put a restriction that data field names have no whitespace.
I assume patches 1 and 2 could be committed now?
Yes. .
v2 changes: * format changed to 'name=value' for both desc and .PKGINFO files. it is now consistent across files and looks a little less weird than using a space as the delimiter * implemented error handling and cleanup * removing parsing of pkgtype fields from alpm * renamed field from "data" to "xdata" to indicate it's "extended" data Andrew Gregory (3): add extended data field for arbitrary package data query: print extended data when extra info is requested makepkg: store pkgtype in xdata lib/libalpm/alpm.h | 7 +++++++ lib/libalpm/be_local.c | 28 ++++++++++++++++++++++++++++ lib/libalpm/be_package.c | 8 ++++++-- lib/libalpm/be_sync.c | 12 ++++++++++++ lib/libalpm/package.c | 35 +++++++++++++++++++++++++++++++++++ lib/libalpm/package.h | 7 +++++++ scripts/makepkg.sh.in | 2 +- src/pacman/package.c | 13 +++++++++++++ 8 files changed, 109 insertions(+), 3 deletions(-) -- 2.35.1
This adds a mechanism for package builders to add arbitrary data to
packages that is not necessarily relevant enough to the package
installation process to gain first-class support in alpm. Currently
these fields have to be added to parsers with a "not actually used"
comment and can't be retrieved through the API.
Extended data is stored in "name=value" format in the xdata field
(%XDATA% in desc files):
xdata = pkgtype=debug
or
%XDATA%
pkgtype=debug
Signed-off-by: Andrew Gregory
Signed-off-by: Andrew Gregory
Package type is not relevant to alpm or even exposed to front-ends in
any way.
Signed-off-by: Andrew Gregory
On 13/3/22 08:52, Andrew Gregory wrote:
This adds a mechanism for package builders to add arbitrary data to packages that is not necessarily relevant enough to the package installation process to gain first-class support in alpm. Currently these fields have to be added to parsers with a "not actually used" comment and can't be retrieved through the API.
Extended data is stored in "name=value" format in the xdata field (%XDATA% in desc files):
xdata = pkgtype=debug
or
%XDATA% pkgtype=debug
Signed-off-by: Andrew Gregory
Looks all god to me. Allan
--- lib/libalpm/alpm.h | 7 +++++++ lib/libalpm/be_local.c | 28 ++++++++++++++++++++++++++++ lib/libalpm/be_package.c | 6 ++++++ lib/libalpm/be_sync.c | 12 ++++++++++++ lib/libalpm/package.c | 35 +++++++++++++++++++++++++++++++++++ lib/libalpm/package.h | 7 +++++++ 6 files changed, 95 insertions(+)
diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index cdf71fdc9..3304ae77a 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -2291,6 +2291,13 @@ typedef enum _alpm_pkgvalidation_t { ALPM_PKG_VALIDATION_SIGNATURE = (1 << 3) } alpm_pkgvalidation_t;
+typedef struct _alpm_pkg_xdata_t { + char *name; + char *value; +} alpm_pkg_xdata_t; + +alpm_list_t *alpm_pkg_get_xdata(alpm_pkg_t *pkg); +
OK.
/** Create a package from a file. * If full is false, the archive is read only until all necessary * metadata is found. If it is true, the entire archive is read, which diff --git a/lib/libalpm/be_local.c b/lib/libalpm/be_local.c index 87fecbe52..e117b69df 100644 --- a/lib/libalpm/be_local.c +++ b/lib/libalpm/be_local.c @@ -195,6 +195,12 @@ static alpm_list_t *_cache_get_backup(alpm_pkg_t *pkg) return pkg->backup; }
+static alpm_list_t *_cache_get_xdata(alpm_pkg_t *pkg) +{ + LAZY_LOAD(INFRQ_DESC); + return pkg->xdata; +} +
OK
/** * Open a package changelog for reading. Similar to fopen in functionality, * except that the returned 'file stream' is from the database. @@ -349,6 +355,7 @@ static const struct pkg_operations local_pkg_ops = { .get_replaces = _cache_get_replaces, .get_files = _cache_get_files, .get_backup = _cache_get_backup, + .get_xdata = _cache_get_xdata,
OK
.changelog_open = _cache_changelog_open, .changelog_read = _cache_changelog_read, @@ -804,6 +811,18 @@ static int local_db_read(alpm_pkg_t *info, int inforeq) READ_AND_SPLITDEP(info->conflicts); } else if(strcmp(line, "%PROVIDES%") == 0) { READ_AND_SPLITDEP(info->provides); + } else if(strcmp(line, "%XDATA%") == 0) { + alpm_list_t *i, *lines = NULL; + READ_AND_STORE_ALL(lines); + for(i = lines; i; i = i->next) { + alpm_pkg_xdata_t *pd = _alpm_pkg_parse_xdata(i->data); + if(pd == NULL || !alpm_list_append(&info->xdata, pd)) { + _alpm_pkg_xdata_free(pd); + FREELIST(lines); + goto error; + } + } + FREELIST(lines);
I'm sure this can be done more efficiently, but I'm doing my best to ignore micro-optimisation potential! OK.
} } fclose(fp); @@ -1040,6 +1059,15 @@ int _alpm_local_db_write(alpm_db_t *db, alpm_pkg_t *info, int inforeq) write_deps(fp, "%CONFLICTS%", info->conflicts); write_deps(fp, "%PROVIDES%", info->provides);
+ if(info->xdata) { + fputs("%XDATA%\n", fp); + for(lp = info->xdata; lp; lp = lp->next) { + alpm_pkg_xdata_t *pd = lp->data; + fprintf(fp, "%s=%s\n", pd->name, pd->value); + } + fputc('\n', fp); + } +
OK.
fclose(fp); fp = NULL; } diff --git a/lib/libalpm/be_package.c b/lib/libalpm/be_package.c index 203c98c13..b4f3f1c26 100644 --- a/lib/libalpm/be_package.c +++ b/lib/libalpm/be_package.c @@ -246,6 +246,12 @@ static int parse_descfile(alpm_handle_t *handle, struct archive *a, alpm_pkg_t * newpkg->backup = alpm_list_add(newpkg->backup, backup); } else if(strcmp(key, "pkgtype") == 0) { /* not used atm */ + } else if(strcmp(key, "xdata") == 0) { + alpm_pkg_xdata_t *pd = _alpm_pkg_parse_xdata(ptr); + if(pd == NULL || !alpm_list_append(&newpkg->xdata, pd)) { + _alpm_pkg_xdata_free(pd); + return -1; + }
OK.
} else { _alpm_log(handle, ALPM_LOG_DEBUG, "%s: unknown key '%s' in description file line %d\n", newpkg->name ? newpkg->name : "error", key, linenum); diff --git a/lib/libalpm/be_sync.c b/lib/libalpm/be_sync.c index 8c8e01a0c..81676be96 100644 --- a/lib/libalpm/be_sync.c +++ b/lib/libalpm/be_sync.c @@ -666,6 +666,18 @@ static int sync_db_read(alpm_db_t *db, struct archive *archive, pkg->files.count = files_count; pkg->files.files = files; _alpm_filelist_sort(&pkg->files); + } else if(strcmp(line, "%DATA%") == 0) { + alpm_list_t *i, *lines = NULL; + READ_AND_STORE_ALL(lines); + for(i = lines; i; i = i->next) { + alpm_pkg_xdata_t *pd = _alpm_pkg_parse_xdata(i->data); + if(pd == NULL || !alpm_list_append(&pkg->xdata, pd)) { + _alpm_pkg_xdata_free(pd); + FREELIST(lines); + goto error; + } + } + FREELIST(lines);
OK.
} } if(ret != ARCHIVE_EOF) { diff --git a/lib/libalpm/package.c b/lib/libalpm/package.c index 4d89dcd8a..8c95dab3e 100644 --- a/lib/libalpm/package.c +++ b/lib/libalpm/package.c @@ -98,6 +98,7 @@ static alpm_list_t *_pkg_get_provides(alpm_pkg_t *pkg) { return pkg->provides; static alpm_list_t *_pkg_get_replaces(alpm_pkg_t *pkg) { return pkg->replaces; } static alpm_filelist_t *_pkg_get_files(alpm_pkg_t *pkg) { return &(pkg->files); } static alpm_list_t *_pkg_get_backup(alpm_pkg_t *pkg) { return pkg->backup; } +static alpm_list_t *_pkg_get_xdata(alpm_pkg_t *pkg) { return pkg->xdata; }
static void *_pkg_changelog_open(alpm_pkg_t UNUSED *pkg) { @@ -162,6 +163,7 @@ const struct pkg_operations default_pkg_ops = { .get_replaces = _pkg_get_replaces, .get_files = _pkg_get_files, .get_backup = _pkg_get_backup, + .get_xdata = _pkg_get_xdata,
.changelog_open = _pkg_changelog_open, .changelog_read = _pkg_changelog_read, @@ -485,6 +487,13 @@ int SYMEXPORT alpm_pkg_has_scriptlet(alpm_pkg_t *pkg) return pkg->ops->has_scriptlet(pkg); }
+alpm_list_t SYMEXPORT *alpm_pkg_get_xdata(alpm_pkg_t *pkg) +{ + ASSERT(pkg != NULL, return NULL); + pkg->handle->pm_errno = ALPM_ERR_OK; + return pkg->ops->get_xdata(pkg); +} +
OK.
static void find_requiredby(alpm_pkg_t *pkg, alpm_db_t *db, alpm_list_t **reqs, int optional) { @@ -677,6 +686,30 @@ static void free_deplist(alpm_list_t *deps) alpm_list_free(deps); }
+alpm_pkg_xdata_t *_alpm_pkg_parse_xdata(const char *string) +{ + alpm_pkg_xdata_t *pd; + const char *sep; + if(string == NULL || (sep = strchr(string, '=')) == NULL) { + return NULL; + } + + CALLOC(pd, 1, sizeof(alpm_pkg_xdata_t), return NULL); + STRNDUP(pd->name, string, sep - string, FREE(pd); return NULL); + STRDUP(pd->value, sep + 1, FREE(pd->name); FREE(pd); return NULL); + + return pd; +} OK.
+ +void _alpm_pkg_xdata_free(alpm_pkg_xdata_t *pd) +{ + if(pd) { + free(pd->name); + free(pd->value); + free(pd); + } +} +
OK.
void _alpm_pkg_free(alpm_pkg_t *pkg) { if(pkg == NULL) { @@ -707,6 +740,8 @@ void _alpm_pkg_free(alpm_pkg_t *pkg) } alpm_list_free_inner(pkg->backup, (alpm_list_fn_free)_alpm_backup_free); alpm_list_free(pkg->backup); + alpm_list_free_inner(pkg->xdata, (alpm_list_fn_free)_alpm_pkg_xdata_free); + alpm_list_free(pkg->xdata);
OK.
free_deplist(pkg->depends); free_deplist(pkg->optdepends); free_deplist(pkg->checkdepends); diff --git a/lib/libalpm/package.h b/lib/libalpm/package.h index 935225300..5ebe6bd12 100644 --- a/lib/libalpm/package.h +++ b/lib/libalpm/package.h @@ -67,6 +67,8 @@ struct pkg_operations { alpm_filelist_t *(*get_files) (alpm_pkg_t *); alpm_list_t *(*get_backup) (alpm_pkg_t *);
+ alpm_list_t *(*get_xdata) (alpm_pkg_t *); + void *(*changelog_open) (alpm_pkg_t *); size_t (*changelog_read) (void *, size_t, const alpm_pkg_t *, void *); int (*changelog_close) (const alpm_pkg_t *, void *); @@ -136,6 +138,8 @@ struct _alpm_pkg_t { alpm_pkgreason_t reason; int scriptlet;
+ alpm_list_t *xdata; + /* Bitfield from alpm_dbinfrq_t */ int infolevel; /* Bitfield from alpm_pkgvalidation_t */ @@ -158,4 +162,7 @@ alpm_pkg_t *_alpm_pkg_load_internal(alpm_handle_t *handle, int _alpm_pkg_cmp(const void *p1, const void *p2); int _alpm_pkg_compare_versions(alpm_pkg_t *local_pkg, alpm_pkg_t *pkg);
+alpm_pkg_xdata_t *_alpm_pkg_parse_xdata(const char *string); +void _alpm_pkg_xdata_free(alpm_pkg_xdata_t *pd); + #endif /* ALPM_PACKAGE_H */
participants (2)
-
Allan McRae
-
Andrew Gregory