[arch-general] Howto look outside chroot for local git tree?
Guys, Is it possible to look outside of the chroot while building in an archroot from the PKGBUILD to check for a local copy of a git tree? Logically, what I want to do from inside the archroot is: (1) check for an existing git source in the ${srcdir} (if so, then use it) (if not, then...) (2) check for a local copy of the git tree outside chroot (if found) (3) change to that directory and update and copy to ${srcdir} (else) (4) clone new copy into ${srcdir} Is this even possible? I have made an attempt at implementing this, but I think I'm prevented from looking outside the chroot. Everything works except for (2) above. The check for the local copy always fails forcing a new clone into ${srcdir} instead of simply being able to update the local tree and copy to ${srcdir}. ## GIT repository variables and location _gitlocal="/home/david/tde/tde/main" _gitname="${pkgname#*-}" _gitdir="${_gitlocal}/dependencies/${_gitname}" _giturl="$url" _gitsfx="-git" _prefix="/opt/tde" build() { cd ${srcdir} <snip> ## update or clone from GIT msg "Checking for existing GIT source: $_gitname" if [ -d $_gitname ] ; then msg "Using existing source in: ${srcdir}/${_gitname}" else msg "Looking for local GIT tree: $_gitdir" if [ -d $_gitdir ] ; then msg "Updating local tree: $_gitdir" cd $_gitdir && { git pull git submodule init git submodule update --recursive git submodule foreach --recursive "git checkout master" git submodule foreach --recursive "git pull" } cd ${srcdir} && cp -a "$_gitdir" . msg "The local files are updated." else msg "Connecting to GIT server...." msg "Cloning new source from: $url" git clone --depth 1 $_giturl fi msg "GIT checkout done or server timeout" fi msg "Starting configure..." # ./configure \ msg "Building - ${pkgname}..." # make } The output from makechrootpkg is always the same even though the local copy of the git tree is present: ==> Making package: tde-libart-lgpl 1.3.2.4-1 (Wed Feb 29 23:55:20 UTC 2012) ==> Checking runtime dependencies... ==> Checking buildtime dependencies... ==> Retrieving Sources... ==> Extracting Sources... ==> Starting build()... ==> Checking for existing GIT source: libart-lgpl ==> Looking for local GIT tree: /home/david/tde/tde/main/dependencies/libart-lgpl ==> Connecting to GIT server.... ==> Cloning new source from: http://scm.trinitydesktop.org/scm/git/libart-lgpl Cloning into 'libart-lgpl'... ==> GIT checkout done or server timeout ==> Starting configure... ==> Building - tde-libart-lgpl... ==> Entering fakeroot environment... ==> Starting package()... ==> Packaging - tde-libart-lgpl-1.3.2.4 Possible? Or would the solution be to set the local copy of the git tree up inside a apache (or whatever it takes) and pretend to run a local copy? Even then, there would be no way to check for its existence from inside the chroot? What say the experts? -- David C. Rankin, J.D.,P.E.
On 1 March 2012 00:13, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
Guys,
Is it possible to look outside of the chroot while building in an archroot from the PKGBUILD to check for a local copy of a git tree? Logically, what I want to do from inside the archroot is:
(1) check for an existing git source in the ${srcdir}
(if so, then use it) (if not, then...)
(2) check for a local copy of the git tree outside chroot
(if found)
(3) change to that directory and update and copy to ${srcdir}
(else)
(4) clone new copy into ${srcdir}
Is this even possible? I have made an attempt at implementing this, but I think I'm prevented from looking outside the chroot. Everything works except for (2) above. The check for the local copy always fails forcing a new clone into ${srcdir} instead of simply being able to update the local tree and copy to ${srcdir}.
## GIT repository variables and location _gitlocal="/home/david/tde/tde/main" _gitname="${pkgname#*-}" _gitdir="${_gitlocal}/dependencies/${_gitname}" _giturl="$url" _gitsfx="-git"
_prefix="/opt/tde"
build() {
cd ${srcdir} <snip>
## update or clone from GIT msg "Checking for existing GIT source: $_gitname" if [ -d $_gitname ] ; then msg "Using existing source in: ${srcdir}/${_gitname}" else msg "Looking for local GIT tree: $_gitdir" if [ -d $_gitdir ] ; then msg "Updating local tree: $_gitdir" cd $_gitdir && { git pull git submodule init git submodule update --recursive git submodule foreach --recursive "git checkout master" git submodule foreach --recursive "git pull" } cd ${srcdir} && cp -a "$_gitdir" . msg "The local files are updated." else msg "Connecting to GIT server...." msg "Cloning new source from: $url" git clone --depth 1 $_giturl fi msg "GIT checkout done or server timeout" fi
msg "Starting configure..." # ./configure \
msg "Building - ${pkgname}..."
# make
}
The output from makechrootpkg is always the same even though the local copy of the git tree is present:
==> Making package: tde-libart-lgpl 1.3.2.4-1 (Wed Feb 29 23:55:20 UTC 2012) ==> Checking runtime dependencies... ==> Checking buildtime dependencies... ==> Retrieving Sources... ==> Extracting Sources... ==> Starting build()... ==> Checking for existing GIT source: libart-lgpl ==> Looking for local GIT tree: /home/david/tde/tde/main/dependencies/libart-lgpl ==> Connecting to GIT server.... ==> Cloning new source from: http://scm.trinitydesktop.org/scm/git/libart-lgpl Cloning into 'libart-lgpl'... ==> GIT checkout done or server timeout ==> Starting configure... ==> Building - tde-libart-lgpl... ==> Entering fakeroot environment... ==> Starting package()... ==> Packaging - tde-libart-lgpl-1.3.2.4
Possible? Or would the solution be to set the local copy of the git tree up inside a apache (or whatever it takes) and pretend to run a local copy? Even then, there would be no way to check for its existence from inside the chroot?
You could mount --bind the location of your local git repositories to a point inside your chroot, that's probably the quickest and simplest way of doing it, although it offers no protection of your git repositories from being wiped out from within the chroot. Regards, Damien
On 2012-03-01 10:07, Damien Churchill wrote:
On 1 March 2012 00:13, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
Possible? Or would the solution be to set the local copy of the git tree up inside a apache (or whatever it takes) and pretend to run a local copy? Even then, there would be no way to check for its existence from inside the chroot?
You could mount --bind the location of your local git repositories to a point inside your chroot, that's probably the quickest and simplest way of doing it, although it offers no protection of your git repositories from being wiped out from within the chroot.
Or you could run `git daemon` on the outside, then change PKGBUILD to use <git://localhost/path/to/repo/> for read-only access. The check would be simple: 1) try cloning directly to $srcdir; 2) if it fails, clone upstream instead. -- Mantas Mikulėnas <grawity@gmail.com>
On Thu, Mar 1, 2012 at 2:07 AM, Damien Churchill <damoxc@gmail.com> wrote:
You could mount --bind the location of your local git repositories to a point inside your chroot, that's probably the quickest and simplest way of doing it, although it offers no protection of your git repositories from being wiped out from within the chroot.
you can readonly --bind mount at the VFS level: mount --bind /a /b mount -oremount,bind,ro /b ... i have an AUR package that does exactly this: (the for loop after "# Allow env passthru ...") http://aur.archlinux.org/packages/py/pyjamas-engine-pythonwebkit/PKGBUILD ... this block allows variables to be set from the ENV, but more related is the excerpt later on in build(): if [[ ! -e ${g}/objects ]]; then msg "[git] Creating NEW repository ... " git --git-dir="${g}" --work-tree="${w}" init elif [[ ! -w ${g}/objects ]]; then warning "[git] Repository read-only, setting up proxy ... " git --git-dir="${_gitrepo_proxy}" --work-tree="${w}" init echo "${g}/objects" > "${_gitrepo_proxy}/objects/info/alternates" cp -r "${g}/refs" "${_gitrepo_proxy}" g="${_gitrepo_proxy}" fi ... this block check for an objects directory: if missing clone new, if readonly create proxy. the proxy works by creating a new git repo, and assigning the original as an alternate object store ... this allows the build to not only reuse the existing repo but *also* download/use/change within it's own repo. i use this for webkit because it's a 1GiB+ repository. -- C Anthony
On 03/01/2012 09:19 AM, C Anthony Risinger wrote:
On Thu, Mar 1, 2012 at 2:07 AM, Damien Churchill <damoxc@gmail.com> wrote:
You could mount --bind the location of your local git repositories to a point inside your chroot, that's probably the quickest and simplest way of doing it, although it offers no protection of your git repositories from being wiped out from within the chroot.
you can readonly --bind mount at the VFS level:
mount --bind /a /b mount -oremount,bind,ro /b
... i have an AUR package that does exactly this:
(the for loop after "# Allow env passthru ...") http://aur.archlinux.org/packages/py/pyjamas-engine-pythonwebkit/PKGBUILD
... this block allows variables to be set from the ENV, but more related is the excerpt later on in build():
if [[ ! -e ${g}/objects ]]; then msg "[git] Creating NEW repository ... " git --git-dir="${g}" --work-tree="${w}" init elif [[ ! -w ${g}/objects ]]; then warning "[git] Repository read-only, setting up proxy ... " git --git-dir="${_gitrepo_proxy}" --work-tree="${w}" init echo "${g}/objects" > "${_gitrepo_proxy}/objects/info/alternates" cp -r "${g}/refs" "${_gitrepo_proxy}" g="${_gitrepo_proxy}" fi
... this block check for an objects directory: if missing clone new, if readonly create proxy. the proxy works by creating a new git repo, and assigning the original as an alternate object store ... this allows the build to not only reuse the existing repo but *also* download/use/change within it's own repo.
i use this for webkit because it's a 1GiB+ repository.
C Anthony, Damien, Mantas, Thank you! I will give these a try. Yes, thankfully the parts I'm dealing with are not 1G, but 208M is really bad over a git connection with less than .1M downsteam. (you get old waiting for the clone to finish...) -- David C. Rankin, J.D.,P.E.
participants (4)
-
C Anthony Risinger
-
Damien Churchill
-
David C. Rankin
-
Mantas M.