[pacman-dev] PATHNAME in sync/*/desc

[Mark Constable]

On Wednesday 12 March 2008 08:36:16 Dan McGee wrote:
> > Would it be feasible to add a %PATHNAME% variable  to the sync desc file ?
> > For instance in this case, fetching "any" arch packages would not need to
> > symlinked into the same directory to be found...
> >
> >   %FILENAME%
> >   hotstuff-0.9.2-1-any.pkg.tar.bz2
> >
> >   %PATHNAME%
> >   ../any/

> Not a huge fan of this behavior, as I really like the current scenario
> where the DB always lives with the packages.

One added complexity, that is currently available, is having
symlinks to packages outside of the current directory so, in a
way, the DB does not always live with the packages. The symlinks
remain in the same directory but not the actual packages. To me,
this *may* be clarified better by having the path component
explicitly noted in the DB itself rather than implied by an
external filesystem symlink. Maybe.

> It greatly simplifies the
> need for repo-add to do anything funny with the paths, as we just know
> to always add the basename of the file. That would need to be
> addressed if we added this functionality- how would you add relative
> paths using repo-add?

Well I just tried this below but the resulting package could not
be found and the local sync DB showed no sign of including the path...

 cd my/i686
 repo-add my.db.tar.gz *.pkg.* ../any/*.pkg.*

so I thought/hoped it might do as Aaron suggested and end up with
a full relative path but it didn't when I checked the local sync
DB entry for the packages in ../any/.

 %FILENAME%
 ../any/hotstuff-0.9.2-1-any.pkg.tar.bz2

> Would it generate the relative path on its own
> (possibly traversing out of webserver accessible directories which
> will break your repository),

I would think this relative path is matched to the remote DB and
would not get added if it could't be found at the time repo-add
is invoked. It would be assumed the same layout, as stated in the
DB, would exist in reality regardless of how the whole repo is
copied or mirrored.

> or would you have to specify an
> additional path? What if I build my DB in one place, but serve it in
> another (something I actually do)?

I rsync my local repo to a remote live server and in this case
there would be no problem that I can forsee.

> We haven't even begun to talk about
> deltas and where they should live.

They're just more file components that could follow the same
relative patch agenda.

> This came up when Xavier and I were
> researching the delta creation being done in repo-add.
> 
> Of course, I see the advantages this could offer with the -any
> packages, I just wanted to get my two cents in from the other side.

Sure, as is any change worth the tradeoff between added benefit
or possible breakage.

***

On the PATHNAME = URL idea, I could imagine this could also be
useful, if at all possible, because I could offload some packages
to a variety of other backend servers according to demand. I could
have the DBs hosted on a server with limited bandwidth but high
availability and offload some or all of the packages to another
server with a higher and/or cheaper bandwidth quota. I could
perhaps use Amazon S3 cloud for important popular packages and a
Dreamhost cheapie account for less popular packages, all from
the same repo sync DB.

I'm not suggesting this idea has any high yield benefits but if
it were available then I could imagine taking advantage of it.

However, I suspect bittorenting packages would probably have the
best impact on distribution load. Better still, if it were possible
to "capture" what would be otherwise transferred via an rsync update
to a local binary blob diff (for the whole archive, or per repo
section) and use bittorent technology to transfer that blob and
patch the other end in one go.

--markc