[arch-general] makepkg - any way to recompile only newly patched files in large packages?
All, There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c file. (it was a one-line fix where a variable needed reset to 0). Is there any way I can configure makepkg to behave as if the gcc -MP -MD options had been given to have it only recompile the one patched source without rebuilding the entire package -- again? I've been through the wiki and the man page, and other than --repackage, there doesn't seem to be anything that would work (or I'm just missing it). -- David C. Rankin, J.D.,P.E.
On 08/06/2017 10:23 PM, David C. Rankin wrote:
All,
There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c file. (it was a one-line fix where a variable needed reset to 0). Is there any way I can configure makepkg to behave as if the gcc -MP -MD options had been given to have it only recompile the one patched source without rebuilding the entire package -- again?
I've been through the wiki and the man page, and other than --repackage, there doesn't seem to be anything that would work (or I'm just missing it).
If it wasn't clear, I have already built the gtk2 package yesterday to --enable-debug=yes so I have all of the files in a state I could call --repackge on, except for the gtk/gtkrecentchooser.c file with the one line change. I was wondering if there was a way to avoid the full 15 minute rebuild of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then --repackage? -- David C. Rankin, J.D.,P.E.
On 07-08-17 09:39, David C. Rankin wrote: I was wondering if there was a way to avoid the full 15 minute rebuild
of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then --repackage?
I don't know a way to do that, but do you have ccache installed ? On my system it's set to use max 20G on a HDD . When building things like mesa or llvm repeatedly with small changes this can shaves minutes of the build time. LW https://wiki.archlinux.org/index.php/Ccache
On Mon, Aug 7, 2017 at 10:39 AM, David C. Rankin
If it wasn't clear, I have already built the gtk2 package yesterday to --enable-debug=yes so I have all of the files in a state I could call --repackge on, except for the gtk/gtkrecentchooser.c file with the one line change. I was wondering if there was a way to avoid the full 15 minute rebuild of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then --repackage?
Not the answer to your question, but you might consider configuring /etc/makepkg.conf to use ccache to speed up rebuilds. Also set MAKEFLAGS=-j4 or similar to make sure you are using all available cores. -- echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3
On 08/07/2017 06:06 AM, Christoph Gysin via arch-general wrote:
Not the answer to your question, but you might consider configuring /etc/makepkg.conf to use ccache to speed up rebuilds. Also set MAKEFLAGS=-j4 or similar to make sure you are using all available cores.
Thanks Cristoph, How do you know whether a package is capable of being built in parallel? I've run into problems before with big builds when building in parallel. Is it just a 'try and see if it works' issue, or is there some general rule-of-thumb you can use? -- David C. Rankin, J.D.,P.E.
On 08/07/2017 10:06 PM, David C. Rankin wrote:
On 08/07/2017 06:06 AM, Christoph Gysin via arch-general wrote:
Not the answer to your question, but you might consider configuring /etc/makepkg.conf to use ccache to speed up rebuilds. Also set MAKEFLAGS=-j4 or similar to make sure you are using all available cores.
Thanks Cristoph,
How do you know whether a package is capable of being built in parallel? I've run into problems before with big builds when building in parallel. Is it just a 'try and see if it works' issue, or is there some general rule-of-thumb you can use?
Generally, you try and see if it works. If it doesn't, the build rules don't properly specify dependencies, and you file an upstream bug report, which may or may not get fixed, and you make sure the PKGBUILD maintainer specifically resets MAKEFLAGS back to -j1 (this is the only time it is okay to specify the number of make jobs to run in parallel, if you are resetting back to the defaults because a package doesn't compile with custom MAKEFLAGS in makepkg.conf). -- Eli Schwartz
On 2017-08-07 09:39, David C. Rankin wrote:
On 08/06/2017 10:23 PM, David C. Rankin wrote:
All,
There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c file. (it was a one-line fix where a variable needed reset to 0). Is there any way I can configure makepkg to behave as if the gcc -MP -MD options had been given to have it only recompile the one patched source without rebuilding the entire package -- again?
I've been through the wiki and the man page, and other than --repackage, there doesn't seem to be anything that would work (or I'm just missing it).
If it wasn't clear, I have already built the gtk2 package yesterday to --enable-debug=yes so I have all of the files in a state I could call --repackge on, except for the gtk/gtkrecentchooser.c file with the one line change. I was wondering if there was a way to avoid the full 15 minute rebuild of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then --repackage?
That should be precisely the job of make. All the other preparation steps such as ./configure or patching should not harm either, likely rather be necessary. Hence, you could just try to rerun makepkg, providing that the $srcdir is still in the state after the last successful build. In fact, I already had problems doing this. Suppose a sed script replacing 'x' by 'xx', inline. In such cases one might consider using the --noprepare flag. In any case, it is a good idea to read the PKGBUILD (as always) and possibly modify it or even do all the work manually.
On 08/07/2017 03:39 AM, David C. Rankin wrote:
On 08/06/2017 10:23 PM, David C. Rankin wrote:
All,
There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c file. (it was a one-line fix where a variable needed reset to 0). Is there any way I can configure makepkg to behave as if the gcc -MP -MD options had been given to have it only recompile the one patched source without rebuilding the entire package -- again?
I've been through the wiki and the man page, and other than --repackage, there doesn't seem to be anything that would work (or I'm just missing it).
If it wasn't clear, I have already built the gtk2 package yesterday to --enable-debug=yes so I have all of the files in a state I could call --repackge on, except for the gtk/gtkrecentchooser.c file with the one line change. I was wondering if there was a way to avoid the full 15 minute rebuild of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then --repackage?
I think you're misunderstanding what -MP does. As for rebuilding only the one source, makepkg source extraction should result in extracting files with the same initial timestamp as the last time they were extracted. Although if you are using the gtk2 PKGBUILD from the ABS, it uses git anyway... So Make should consider all objects to be up-to-date, except the one you have sed'ed. You could also use `makepkg --noextract` which assumes you applied the fix by hand rather than in the PGBUILD, which is probably safer as it skips prepare(), and in theory re-running configure should not cause Make to expire all object files (though with autotools who really knows...) Aside: repackage does not do what you want, since it doesn't rebuild anything. Granted, Makefiles usually have the install target depend on the default target, so it would probably end up being built too... in fakeroot... -- Eli Schwartz
On 08/07/2017 09:36 AM, Eli Schwartz wrote:
You could also use `makepkg --noextract` which assumes you applied the fix by hand rather than in the PGBUILD, which is probably safer as it skips prepare(), and in theory re-running configure should not cause Make to expire all object files (though with autotools who really knows...)
That is one thing I didn't consider. It has always been beaten into my head that you don't touch the original source and you take the time to create a .patch to be applied though prepare(). I guess this is one instance where it would make sense to actually edit the files in the source-tree directly in order to skip prepare. Thanks.
Aside: repackage does not do what you want, since it doesn't rebuild anything. Granted, Makefiles usually have the install target depend on the default target, so it would probably end up being built too... in fakeroot...
Yes, the reference to --repackage was just a way of describing the state of the build directory. Repackaging the same executables hoping something changed is as productive as trying to destroy a shot-put with a rubber hammer... -- David C. Rankin, J.D.,P.E.
On 08/07/2017 10:12 PM, David C. Rankin wrote:
On 08/07/2017 09:36 AM, Eli Schwartz wrote:
You could also use `makepkg --noextract` which assumes you applied the fix by hand rather than in the PGBUILD, which is probably safer as it skips prepare(), and in theory re-running configure should not cause Make to expire all object files (though with autotools who really knows...)
That is one thing I didn't consider. It has always been beaten into my head that you don't touch the original source and you take the time to create a .patch to be applied though prepare().
I guess this is one instance where it would make sense to actually edit the files in the source-tree directly in order to skip prepare.
Thanks.
You should do both. Set it in prepare() to save yourself the effort in the future, and for your local run you can do it by hand to save time when not doing a clean rebuild. prepare() is not idempotent, its value lies in making sure you can replicate what you did next week when running `makepkg -C`. -- Eli Schwartz
Am 07.08.2017 um 09:39 schrieb David C. Rankin:
If it wasn't clear, I have already built the gtk2 package yesterday to --enable-debug=yes so I have all of the files in a state I could call --repackge on, except for the gtk/gtkrecentchooser.c file with the one line change. I was wondering if there was a way to avoid the full 15 minute rebuild of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then --repackage?
You can use makepkg with the -e (--noextract) option to rebuild from the existing source tree. If the configure step forces a full rebuild, you can comment it out in the PKGBUILD after the initial build. -- Andy
participants (6)
-
Christoph Gysin
-
David C. Rankin
-
Eli Schwartz
-
LoneVVolf
-
ProgAndy
-
respiranto