On Mon, Mar 4, 2013 at 1:20 PM, Allan McRae wrote:

This would mean --holdver would not work when a project has submodules given these would be updated. Also, that would require an internet connection during "extracting" the sources, but --holdver is suppose to allow offline VCS builds. Is there a way we can avoid this?

Anyone else got anything to add here?

Allan

The state of the submodules is fixed in the commit of the parent repo - the submodules are locked to commits as well. With --recursive, building with an internet connection would work, but without would not, as I think it will attempt to get the submodules from their sources, not the parent repo. Without --recursive, building would fail in any case, as the submodules aren't initialized at all. So I think adding --recursive is the right thing to do.

On 04/03/13 23:06, Allan McRae wrote:

On 04/03/13 22:58, Jan Steffens wrote:

...

On Mon, Mar 4, 2013 at 1:20 PM, Allan McRae wrote:

...

This would mean --holdver would not work when a project has submodules given these would be updated. Also, that would require an internet connection during "extracting" the sources, but --holdver is suppose to allow offline VCS builds. Is there a way we can avoid this?

Anyone else got anything to add here?

Allan

The state of the submodules is fixed in the commit of the parent repo - the submodules are locked to commits as well.

With --recursive, building with an internet connection would work, but without would not, as I think it will attempt to get the submodules from their sources, not the parent repo.

Is it possible to improve on that? Can we checkout the submodules earlier?

...

Without --recursive, building would fail in any case, as the submodules aren't initialized at all.

So I think adding --recursive is the right thing to do.

Pulling patch - if someone cn figure out how to have submodules and have all the source downloaded earlier, that would be good...

On Tue, Mar 5, 2013 at 10:58 AM, Allan McRae wrote:

On 04/03/13 23:06, Allan McRae wrote:

...

On 04/03/13 22:58, Jan Steffens wrote:

...

On Mon, Mar 4, 2013 at 1:20 PM, Allan McRae wrote:

...

This would mean --holdver would not work when a project has submodules given these would be updated. Also, that would require an internet connection during "extracting" the sources, but --holdver is suppose to allow offline VCS builds. Is there a way we can avoid this?

Anyone else got anything to add here?

Allan

The state of the submodules is fixed in the commit of the parent repo - the submodules are locked to commits as well.

With --recursive, building with an internet connection would work, but without would not, as I think it will attempt to get the submodules from their sources, not the parent repo.

Is it possible to improve on that? Can we checkout the submodules earlier?

...

Without --recursive, building would fail in any case, as the submodules aren't initialized at all.

So I think adding --recursive is the right thing to do.

Pulling patch - if someone cn figure out how to have submodules and have all the source downloaded earlier, that would be good...

I now think this should be reverted. Initializing and updating the submodules every build causes it to download all the submodules every build. That's hardly optimal, and sometimes the submodules aren't even needed for building. Rather, I think we need to have the PKGBUILD handle submodules manually. So, revert this patch and write PKGBUILDs like this: source=(git://foobar.com/git/foo.git git://foobar.com/git/foo-module-bar.git git://foobar.com/git/foo-module-baz.git) md5sums=(SKIP SKIP SKIP) build() { cd foo git submodule init git config submodule.qux/bar.url "$srcdir/foo-module-bar" git config submodule.qux/baz.url "$srcdir/foo-module-baz" git submodule update make } Here, foo is our top-level repo, and foo-module-ba[rz] are the submodules used. The names of the submodules in the foo repo are "qux/bar" and "qux/baz". The PKGBUILD initializes the submodules (copying the config from .gitmodules to the local .git/config), then sets the clone URL for the submodules to the clones we already have. The update then clones from there. Does somepony have an idea how to automate this? In the meantime, please revert the patch so submodules can be handled manually, as shown.

On 12/03/13 14:04, Allan McRae wrote:

On 12/03/13 00:08, Jan Steffens wrote:

...

On Tue, Mar 5, 2013 at 10:58 AM, Allan McRae wrote:

...

On 04/03/13 23:06, Allan McRae wrote:

...

On 04/03/13 22:58, Jan Steffens wrote:

...

On Mon, Mar 4, 2013 at 1:20 PM, Allan McRae wrote:

...

This would mean --holdver would not work when a project has submodules given these would be updated. Also, that would require an internet connection during "extracting" the sources, but --holdver is suppose to allow offline VCS builds. Is there a way we can avoid this?

Anyone else got anything to add here?

Allan

The state of the submodules is fixed in the commit of the parent repo - the submodules are locked to commits as well.

With --recursive, building with an internet connection would work, but without would not, as I think it will attempt to get the submodules from their sources, not the parent repo.

Is it possible to improve on that? Can we checkout the submodules earlier?

...

Without --recursive, building would fail in any case, as the submodules aren't initialized at all.

So I think adding --recursive is the right thing to do.

Pulling patch - if someone cn figure out how to have submodules and have all the source downloaded earlier, that would be good...

I now think this should be reverted. Initializing and updating the submodules every build causes it to download all the submodules every build. That's hardly optimal, and sometimes the submodules aren't even needed for building.

Rather, I think we need to have the PKGBUILD handle submodules manually. So, revert this patch and write PKGBUILDs like this:

source=(git://foobar.com/git/foo.git git://foobar.com/git/foo-module-bar.git git://foobar.com/git/foo-module-baz.git) md5sums=(SKIP SKIP SKIP)

build() { cd foo

git submodule init git config submodule.qux/bar.url "$srcdir/foo-module-bar" git config submodule.qux/baz.url "$srcdir/foo-module-baz" git submodule update

make }

Here, foo is our top-level repo, and foo-module-ba[rz] are the submodules used. The names of the submodules in the foo repo are "qux/bar" and "qux/baz". The PKGBUILD initializes the submodules (copying the config from .gitmodules to the local .git/config), then sets the clone URL for the submodules to the clones we already have. The update then clones from there.

Does somepony have an idea how to automate this? In the meantime, please revert the patch so submodules can be handled manually, as shown.

I am also in favour of reverting this. I have struck the repeated downloading of submodules for each build and it annoyed me.

The "workaround" you provided works fine and completely separates the download and build stages again. Using submodules is not all that common, so I'd say just document this in the wiki and be done.

Any other comments?

Just a further point - do this in prepare() not build().