On 30/09/13 08:30 PM, Allan McRae wrote:

On 29/09/13 03:01, Lukas Jirkovsky wrote:

...

Accidentally sent this to a wrong address... I hope forwarding this won't break the patch.

--- Hi, This patch adds incremental build support for the packages with VCS sources. It is not complete work (only GIT is supported), but a mere request for comments. If there is a chance that something like this will be accepted, I will extend the support to other VCS, too.

Rationale: When someone uses a VCS package, there is a high chance that such user wants to follow the development more closely. This means that the user will be rebuilding the package often. However, rebuilding the whole package takes a lot of time. This can be somewhat mitigated by using ccache. However, the greatest speedup can be achieved by incremental build. The problem is that current VCS support makes this pretty much imposible by removing the cloned sources before building.

This patch adds a new option --incremental. Using this option forces makepkg to only update the cloned source directory instead of removing it and cloning anew. This allows make to use the incremental build feature.

See also FS#35050

I've been using this patch for a few days with kdevplatform-git and kdevelop-git (not the ones from AUR) at it works great. It already saved me a lot of time, while using nice and clean PKGBUILD (so far I have been using the old-style VCS packages that clone sources in their build() function to achive similar functionality).

Any comments are welcome.

Lukas

I have not looked at the patch yet, but my initial impression is that the additional flag should not be needed. For non-vcs files, we extract the source over the top of whatever is in $srcdir. We should do the same for VCS sources.

Why may be worth having a flag for is to delete the source directory before starting: https://bugs.archlinux.org/task/17175

Other opinions here?

Allan

I would get a lot of use out of an --incremental option. Let's say you build xorg-server-git, it takes 20 minutes and you install it with pacman. The next day, an upstream commit changes 5 lines in one file and you decide to stay on top of things and have your installed xorg-server-git update to reflect this. Only one file needs to be rebuilt but makepkg will still make you wait 20 minutes. I've resorted to make installing untracked files more than once because of this.

On 30/09/13 09:01 PM, Allan McRae wrote:

On 01/10/13 13:55, Connor Behan wrote:

...

On 30/09/13 08:30 PM, Allan McRae wrote:

...

On 29/09/13 03:01, Lukas Jirkovsky wrote:

...

Accidentally sent this to a wrong address... I hope forwarding this won't break the patch.

--- Hi, This patch adds incremental build support for the packages with VCS sources. It is not complete work (only GIT is supported), but a mere request for comments. If there is a chance that something like this will be accepted, I will extend the support to other VCS, too.

Rationale: When someone uses a VCS package, there is a high chance that such user wants to follow the development more closely. This means that the user will be rebuilding the package often. However, rebuilding the whole package takes a lot of time. This can be somewhat mitigated by using ccache. However, the greatest speedup can be achieved by incremental build. The problem is that current VCS support makes this pretty much imposible by removing the cloned sources before building.

This patch adds a new option --incremental. Using this option forces makepkg to only update the cloned source directory instead of removing it and cloning anew. This allows make to use the incremental build feature.

See also FS#35050

I've been using this patch for a few days with kdevplatform-git and kdevelop-git (not the ones from AUR) at it works great. It already saved me a lot of time, while using nice and clean PKGBUILD (so far I have been using the old-style VCS packages that clone sources in their build() function to achive similar functionality).

Any comments are welcome.

Lukas

I have not looked at the patch yet, but my initial impression is that the additional flag should not be needed. For non-vcs files, we extract the source over the top of whatever is in $srcdir. We should do the same for VCS sources.

Why may be worth having a flag for is to delete the source directory before starting: https://bugs.archlinux.org/task/17175

Other opinions here?

Allan

I would get a lot of use out of an --incremental option. Let's say you build xorg-server-git, it takes 20 minutes and you install it with pacman. The next day, an upstream commit changes 5 lines in one file and you decide to stay on top of things and have your installed xorg-server-git update to reflect this. Only one file needs to be rebuilt but makepkg will still make you wait 20 minutes. I've resorted to make installing untracked files more than once because of this.

Did you read my comment beyond the first sentence?

Oops, I read "we should do the same" as "we do the same".

On 01/10/13 21:18, Lukas Jirkovsky wrote:

On Tue, Oct 1, 2013 at 5:30 AM, Allan McRae wrote:

...

I have not looked at the patch yet, but my initial impression is that the additional flag should not be needed. For non-vcs files, we extract the source over the top of whatever is in $srcdir. We should do the same for VCS sources.

Why may be worth having a flag for is to delete the source directory before starting: https://bugs.archlinux.org/task/17175

Other opinions here?

Allan

I don't mind doing that either. I just didn't want to change the existing behaviour because I thought that there is some reason to delete sources by default. I guess changing the default behaviour to what --incremental is supposed to do and fixing FS#17175 may be in fact simpler.

I guess the current delete source was due to me either building in a clean chroot or outside the source directory meaning it made no difference for me. I am happy with the default changing. Allan