[pacman-dev] makepkg3 improvements (for pacman-3.1)
Hi,
I've been using pacman for over a year now on DIY-Linux
(http://www.diy-linux.org), and to put it simply it's great easy to use
and easy to setup. So I thought it was about time I contribute something
back... an improved version of makepkg. For the impatient :p see the
script attached.
makepkg2's use of fakeroot was a bit hackish, it's now very hackish in
makepkg3. It also kicks up a few strange bugs (flyspray #6208) because
it's being used where it shouldn't be. So I've rewritten/reordered
makepkg to reduce the amount of time spent in fakeroot. Specificly it's
now only used for...
fake_install() (if it exists. See below)
build() (if there's no fake_install)
tidy_install()
- remove docs
- move/compress man pages
- strip binaries/libraries
- remove libtool files
- remove empty directories
create_package()
- create .PKGINFO & .FILELIST
- tar it all up
everything else is run outside of fakeroot.
Over at DIY-Linux we have a patch that adds an extra function to
PKGBUILD (fake_install). This way we can prepare, compile, and test the
package as the user running makepkg. Then fake_install is run inside
fakeroot to install the package. This further reduces the time spent in
fakeroot and the possibility of any weird bugs popping up. If the build
script doesn't contain a fake_install function the build function is
run inside of fakeroot instead (maintaining backwards compatibility with
how things work at the moment).
Other changes...
- reordered the code to reduce the number of if..then..else cases
- renamed a lot of variables so they're a bit more descriptive
- started to add error codes for the different reasons why makepkg exits
- quoted all variables that contain paths so '/home/me/a folder with
spaces' doesn't cause errors. (but build scripts are still a point of
failure as the majority don't have $startdir in quotes.)
- fixed flyspray bug #6468 -- Localized dates for build/install date
- added some more error catching
- added the ability to choose between usr/man and usr/share/man (var
(_MAN_DIR) at the top of the script.)
There are still a few things that need fixed, but some discussion is
probably required.
- Should build deps be removed after a successful build?
At the moment deps are only removed if '-r' is used and it removes build
deps and runtime deps. An extra option to remove build deps?
- Should root be a loud to run makepkg?
IMO running makepkg as root is a hanus crime, and anyone foolish enough
to do so deserves to have their system burn to the ground as the result
of a broken build script. One mistaken key stroke in a build script can
result in a successful build or the destruction of a system :p
I've added a warning that asks the user if they're sure the want to
continue running makepkg as root.
With sudo and fakeroot there's no need to run makepkg as root, but thats
just my opinion.
Andrew
#!/bin/bash
#
# makepkg - make packages compatable for use with pacman
#
# Copyright (c) 2002-2006 by Judd Vinet
On 3/23/07, Andrew Fyfe
Hi, I've been using pacman for over a year now on DIY-Linux (http://www.diy-linux.org), and to put it simply it's great easy to use and easy to setup. So I thought it was about time I contribute something back... an improved version of makepkg. For the impatient :p see the script attached.
Welcome to the list, good to have a few new voices.
makepkg2's use of fakeroot was a bit hackish, it's now very hackish in makepkg3. It also kicks up a few strange bugs (flyspray #6208) because it's being used where it shouldn't be. So I've rewritten/reordered makepkg to reduce the amount of time spent in fakeroot. Specificly it's now only used for...
fake_install() (if it exists. See below) build() (if there's no fake_install) tidy_install() - remove docs - move/compress man pages - strip binaries/libraries - remove libtool files - remove empty directories create_package() - create .PKGINFO & .FILELIST - tar it all up
everything else is run outside of fakeroot.
Yes, this is definitely a better way to go about doing this. makepkg itself is getting a bit unwieldy, and the original method of recalling itself with fakeroot is quite ugly.
Over at DIY-Linux we have a patch that adds an extra function to PKGBUILD (fake_install). This way we can prepare, compile, and test the package as the user running makepkg. Then fake_install is run inside fakeroot to install the package. This further reduces the time spent in fakeroot and the possibility of any weird bugs popping up. If the build script doesn't contain a fake_install function the build function is run inside of fakeroot instead (maintaining backwards compatibility with how things work at the moment).
So if I understand correctly, each PKGBUILD would have a build() function to actually build, and a fake_install() function that usually contains something like 'make install'? This seems like a pretty sane idea, although some of our PKGBUILDs are a bit convoluted.
Other changes... - reordered the code to reduce the number of if..then..else cases Probably needed on our end as well.
- renamed a lot of variables so they're a bit more descriptive. +1
- started to add error codes for the different reasons why makepkg exits +1, this is something I started thinking about but never got around to doing.
- quoted all variables that contain paths so '/home/me/a folder with spaces' doesn't cause errors. (but build scripts are still a point of failure as the majority don't have $startdir in quotes.) I think I got all of these in my last run through of the script, but don't quote me on that (wow, bad joke). :) - fixed flyspray bug #6468 -- Localized dates for build/install date Ooo, we'll definitely look at this.
- added some more error catching - added the ability to choose between usr/man and usr/share/man (var (_MAN_DIR) at the top of the script.) This should maybe even be in makepkg.conf.
There are still a few things that need fixed, but some discussion is probably required.
- Should build deps be removed after a successful build? At the moment deps are only removed if '-r' is used and it removes build deps and runtime deps. An extra option to remove build deps?
- Should root be a loud to run makepkg? IMO running makepkg as root is a hanus crime, and anyone foolish enough to do so deserves to have their system burn to the ground as the result of a broken build script. One mistaken key stroke in a build script can result in a successful build or the destruction of a system :p I've added a warning that asks the user if they're sure the want to continue running makepkg as root. With sudo and fakeroot there's no need to run makepkg as root, but thats just my opinion. It's my opinion too. I think we should definitely have a warning and prompt that has to be answered when building as root.
Welcome to the list, hope you can help us get a lot of these fixes into the upstream makepkg. -Dan
So if I understand correctly, each PKGBUILD would have a build() function to actually build, and a fake_install() function that usually contains something like 'make install'? This seems like a pretty sane idea, although some of our PKGBUILDs are a bit convoluted. Yes, that's correct.
- fixed flyspray bug #6468 -- Localized dates for build/install date Ooo, we'll definitely look at this. Well fixed probably isn't the right word. I followed the suggestion of using the ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (eg 1997-07-16T19:20:30+01:00) in .PKGINFO, pacman/libalm still needs patched to localize the date.
- added the ability to choose between usr/man and usr/share/man (var (_MAN_DIR) at the top of the script.) This should maybe even be in makepkg.conf. Yeah that would be better.
Welcome to the list, hope you can help us get a lot of these fixes into the upstream makepkg. Thanks :)
-Dan
_______________________________________________ pacman-dev mailing list pacman-dev@archlinux.org http://www.archlinux.org/mailman/listinfo/pacman-dev
Andrew Fyfe wrote:
makepkg2's use of fakeroot was a bit hackish, it's now very hackish in makepkg3. It also kicks up a few strange bugs (flyspray #6208) because it's being used where it shouldn't be. So I've rewritten/reordered makepkg to reduce the amount of time spent in fakeroot. Specificly it's now only used for...
fake_install() (if it exists. See below) build() (if there's no fake_install) tidy_install() - remove docs - move/compress man pages - strip binaries/libraries - remove libtool files - remove empty directories create_package() - create .PKGINFO & .FILELIST - tar it all up
everything else is run outside of fakeroot.
Attached are 6 patches to clean up fakeroot. I've split them up into
separate patches to make it's easier to read. It's mostly white space
and moving code from one end of the script to the other.
Patches 1-4 reorder and clean up makepkg in preperation for the cleaner
method of running fakeroot.
Patch 5 is the new fakeroot implementation.
Patch 6 is the fake_install implementation.
See the top of each patch for a more detailed outline of the changes.
Apply the patches in this order...
[1] pacman-cvs-chksums.patch
[2] pacman-cvs-create_package.patch
[3] pacman-cvs-tidy_install.patch
[4] pacman-cvs-run_build.patch
[5] pacman-cvs-fakeroot.patch
[6] pacman-cvs-fake_install.patch
When I get the chance I'll make patches for the other features/fixes
I've added to makepkg.
Andrew
------------------------------------------------------------
revno: 2
committer: Andrew Fyfe
- fixed flyspray bug #6468 -- Localized dates for build/install date Ooo, we'll definitely look at this. Well fixed probably isn't the right word. I followed the suggestion of using the ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (eg 1997-07-16T19:20:30+01:00) in .PKGINFO, pacman/libalm still needs patched to localize the date. Why the hell did I say fixed. I've thought of a better solution. Set builddate (in .PKGINFO) to unix epoch, then when pacman displays it use ctime() (time.h). It converts a unix epoch to a localized time string based on /etc/localtime or the TZ enviroment variable.
Andrew
On 3/24/07, Andrew Fyfe
- fixed flyspray bug #6468 -- Localized dates for build/install date Ooo, we'll definitely look at this. Well fixed probably isn't the right word. I followed the suggestion of using the ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (eg 1997-07-16T19:20:30+01:00) in .PKGINFO, pacman/libalm still needs patched to localize the date. Why the hell did I say fixed. I've thought of a better solution. Set builddate (in .PKGINFO) to unix epoch, then when pacman displays it use ctime() (time.h). It converts a unix epoch to a localized time string based on /etc/localtime or the TZ enviroment variable.
Interested in working on this? We need to make it backwards compatible as well. What if we stuck both an old formatted time and new formatted time in there, allowed pacman to read both (but prefer the newer), and eventually phase out the old style? We need a way for it to transition well. -Dan
On 6/10/07, Dan McGee
On 3/24/07, Andrew Fyfe
wrote: - fixed flyspray bug #6468 -- Localized dates for build/install date Ooo, we'll definitely look at this. Well fixed probably isn't the right word. I followed the suggestion of using the ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (eg 1997-07-16T19:20:30+01:00) in .PKGINFO, pacman/libalm still needs patched to localize the date. Why the hell did I say fixed. I've thought of a better solution. Set builddate (in .PKGINFO) to unix epoch, then when pacman displays it use ctime() (time.h). It converts a unix epoch to a localized time string based on /etc/localtime or the TZ enviroment variable.
Interested in working on this? We need to make it backwards compatible as well. What if we stuck both an old formatted time and new formatted time in there, allowed pacman to read both (but prefer the newer), and eventually phase out the old style? We need a way for it to transition well.
Carving my way through some old emails.... How does this sound for a sanity check while we do this conversion: If build/install date start with a non-digit, use old code, otherwise asssume it's an epoch? Dan checked this out, it doesn't seem to be localized and hasn't changed in a while: builddate=$(LC_ALL= ; LANG= ; date -u "+%a %b %e %H:%M:%S %Y") If no one sees problems with this, I will move it to the master branch when I get a chance. - Aaron
2007/9/19, Aaron Griffin
Carving my way through some old emails....
How does this sound for a sanity check while we do this conversion: If build/install date start with a non-digit, use old code, otherwise asssume it's an epoch?
What do you mean by epoch? ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (e.g. 1997-07-16T19:20:30+01:00) or just the Unix timestamp (number of seconds since 1970)?
Dan checked this out, it doesn't seem to be localized and hasn't changed in a while:
builddate=$(LC_ALL= ; LANG= ; date -u "+%a %b %e %H:%M:%S %Y")
If no one sees problems with this, I will move it to the master branch when I get a chance.
-- Roman Kyrylych (Роман Кирилич)
On 9/19/07, Roman Kyrylych
2007/9/19, Aaron Griffin
: Carving my way through some old emails....
How does this sound for a sanity check while we do this conversion: If build/install date start with a non-digit, use old code, otherwise asssume it's an epoch?
What do you mean by epoch? ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (e.g. 1997-07-16T19:20:30+01:00) or just the Unix timestamp (number of seconds since 1970)?
2007/9/19, Aaron Griffin
On 9/19/07, Roman Kyrylych
wrote: 2007/9/19, Aaron Griffin
: Carving my way through some old emails....
How does this sound for a sanity check while we do this conversion: If build/install date start with a non-digit, use old code, otherwise asssume it's an epoch?
What do you mean by epoch? ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (e.g. 1997-07-16T19:20:30+01:00) or just the Unix timestamp (number of seconds since 1970)?
Huh? "The Unix epoch is the time 00:00:00 UTC on January 1, 1970". :-P I knew this before, nothing new. I asked about the string format used to represent the time. "For brevity, the remainder of this section will use ISO 8601 date format, in which the Unix epoch is 1970-01-01T00:00:00Z." So? ;-) -- Roman Kyrylych (Роман Кирилич)
On 9/19/07, Roman Kyrylych
2007/9/19, Aaron Griffin
: On 9/19/07, Roman Kyrylych
wrote: 2007/9/19, Aaron Griffin
: Carving my way through some old emails....
How does this sound for a sanity check while we do this conversion: If build/install date start with a non-digit, use old code, otherwise asssume it's an epoch?
What do you mean by epoch? ISO 8601 date format YYYY-MM-DDThh:mm:ssTZD (e.g. 1997-07-16T19:20:30+01:00) or just the Unix timestamp (number of seconds since 1970)?
Huh?
"The Unix epoch is the time 00:00:00 UTC on January 1, 1970". :-P I knew this before, nothing new. I asked about the string format used to represent the time.
"For brevity, the remainder of this section will use ISO 8601 date format, in which the Unix epoch is 1970-01-01T00:00:00Z."
So? ;-)
"it is the number of seconds elapsed since midnight UTC of January 1, 1970" "number of seconds" is the operative term. It is not a string format, it is a number. Using _any_ string format will cause problems in the future. As an aside though, the strptime parsing of the old-format dates in the patch on my working branch also gives us timezone adjustments. Yay.
participants (4)
-
Aaron Griffin
-
Andrew Fyfe
-
Dan McGee
-
Roman Kyrylych