Hey there Archers, I decided to work on this little project over the weekend. It is a complete git clone of the AUR from the source and it will be updated regularly (at least once a day) though probably as frequent as 30minutes (please see `Side Notes' for explanation). What this means is that history for the AUR will now be possible! Also, for those of you wanting to have an abs like tree for all of the aur, it is now possible. I hope this opens up new opportunities, and new possibilities for everyone.
To get the complete git repo, run: git clone git://pkgbuild.com/aur.git
The web interface is at: http://pkgbuild.com/git/aur.git/
Also, please take a look at the official readme at: http://pkgbuild.com/~td123/readme
Special Thanks to Ioni for enabling the git server & Bluewind for setting up the web interface.
Cheers! -Thomas Dziedzic
Side Note: Although it will be updated at least once a day, it will most likely be updated more frequently (around 30 minute intervals with a script), I just can't guarantee that for now. Work is ongoing to automate the sync with the aur tree, but at this moment, there is no secure way of doing this. Automating the sync will guarantee the intervals between syncs, I am aiming for around 30 minute to 3 hour intervals between syncs. A final goal might be to get continuous updates. Enjoy! =)
On 6 February 2011 20:31, Thomas Dziedzic gostrc@gmail.com wrote:
Hey there Archers, I decided to work on this little project over the weekend. It is a complete git clone of the AUR from the source and it will be updated regularly (at least once a day) though probably as frequent as 30minutes (please see `Side Notes' for explanation). What this means is that history for the AUR will now be possible! Also, for those of you wanting to have an abs like tree for all of the aur, it is now possible. I hope this opens up new opportunities, and new possibilities for everyone.
To get the complete git repo, run: git clone git://pkgbuild.com/aur.git
The web interface is at: http://pkgbuild.com/git/aur.git/
Also, please take a look at the official readme at: http://pkgbuild.com/~td123/readme
Special Thanks to Ioni for enabling the git server & Bluewind for setting up the web interface.
Cheers! -Thomas Dziedzic
Side Note: Although it will be updated at least once a day, it will most likely be updated more frequently (around 30 minute intervals with a script), I just can't guarantee that for now. Work is ongoing to automate the sync with the aur tree, but at this moment, there is no secure way of doing this. Automating the sync will guarantee the intervals between syncs, I am aiming for around 30 minute to 3 hour intervals between syncs. A final goal might be to get continuous updates. Enjoy! =)
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Yes, I know, the aur tree isn't completely clean, for now, you will have to deal with it.
2011/2/6 Thomas Dziedzic gostrc@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Yes, I know, the aur tree isn't completely clean, for now, you will have to deal with it.
I will try to clean up my rules this evening.
Thanks for your suggestions :)
2011/2/6 Thomas Dziedzic gostrc@gmail.com
2011/2/6 Thomas Dziedzic gostrc@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Yes, I know, the aur tree isn't completely clean, for now, you will have to deal with it.
I will try to clean up my rules this evening.
Thanks for your suggestions :)
This is awesome, thanks!
Very good idea, I will check it out.
AUR was under some kind of version control before or it never was?
On Sun, Feb 6, 2011 at 2:15 PM, Bernardo Barros bernardobarros2@gmail.com wrote:
Very good idea, I will check it out.
AUR was under some kind of version control before or it never was?
I think it was never.
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b206...
The only leftover files are the .tgz files which will also have to get cleaned up in the main aur tree. Thanks again!
On Sun 06 Feb 2011 14:34 -0600, Thomas Dziedzic wrote:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b206...
That was me hehe. I was curious to see what would happen. So it looks like you're just copying over what's in the filesystem. It might be a good idea to work with aur-dev to help make the source cleaner, then everyone can benefit - even those who don't use your git repo.
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
On 2/6/11, Loui Chang louipc.ist@gmail.com wrote:
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
Hey, you are stealing my idea! :-) AUR3 does that, and it saves several hundred megabytes. Completely worth it.
-Kyle http://kmkeen.com
On Sun, Feb 6, 2011 at 4:58 PM, keenerd keenerd@gmail.com wrote:
On 2/6/11, Loui Chang louipc.ist@gmail.com wrote:
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
Hey, you are stealing my idea! :-) AUR3 does that, and it saves several hundred megabytes. Completely worth it.
-Kyle http://kmkeen.com
I fail to see how this is worth it, imo, a better system is to convert to git and not track the src.tar.gz
Is there a good reason for this switch? To save 450mb is not a good reason imo, for an incomplete listing of all the files.
-Thomas
On Sun 06 Feb 2011 17:52 -0600, Thomas Dziedzic wrote:
On Sun, Feb 6, 2011 at 4:58 PM, keenerd keenerd@gmail.com wrote:
On 2/6/11, Loui Chang louipc.ist@gmail.com wrote:
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
Hey, you are stealing my idea! :-) AUR3 does that, and it saves several hundred megabytes. Completely worth it.
I fail to see how this is worth it, imo, a better system is to convert to git and not track the src.tar.gz
Is there a good reason for this switch? To save 450mb is not a good reason imo, for an incomplete listing of all the files.
Well, there are several reasons. Lukas' commit message from commit ec0dfc2 briefly summarizes it.
Automatic tarball extraction was vulnerable in different ways. Users should also only use source tarballs to build packages, so this has been removed completely. From now on, only the PKGBUILD is extracted in a secure manner.
Also,
I'm not really sure that git is the best way to distribute source packages, but I'm glad that you're exploring different options. :D
If I want to obtain or share a few build scripts for a few packages I really don't want to keep a 450mb repo.
I have heard about shallow checkouts being implemented in git though, so maybe it could work. devtools uses subversion at least partially because of this large checkout issue.
On Sun, Feb 6, 2011 at 6:22 PM, Loui Chang louipc.ist@gmail.com wrote:
On Sun 06 Feb 2011 17:52 -0600, Thomas Dziedzic wrote:
On Sun, Feb 6, 2011 at 4:58 PM, keenerd keenerd@gmail.com wrote:
On 2/6/11, Loui Chang louipc.ist@gmail.com wrote:
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
Hey, you are stealing my idea! :-) AUR3 does that, and it saves several hundred megabytes. Completely worth it.
I fail to see how this is worth it, imo, a better system is to convert to git and not track the src.tar.gz
Is there a good reason for this switch? To save 450mb is not a good reason imo, for an incomplete listing of all the files.
Well, there are several reasons. Lukas' commit message from commit ec0dfc2 briefly summarizes it.
Automatic tarball extraction was vulnerable in different ways. Users should also only use source tarballs to build packages, so this has been removed completely. From now on, only the PKGBUILD is extracted in a secure manner.
Also,
I'm not really sure that git is the best way to distribute source packages, but I'm glad that you're exploring different options. :D
If I want to obtain or share a few build scripts for a few packages I really don't want to keep a 450mb repo.
I have heard about shallow checkouts being implemented in git though, so maybe it could work. devtools uses subversion at least partially because of this large checkout issue.
Well, whatever the case, I stayed up a while working on finishing this new version of the system. The new version syncs only the .tar.gz files, which makes it less resource hungry, and this also adds a natural ability to commit per packages :) I still need to test out deleting functionality, so if you could run the cleanup script at some point today, that would be awesome.
Cheers!
On Mon 07 Feb 2011 09:21 -0600, Thomas Dziedzic wrote:
Well, whatever the case, I stayed up a while working on finishing this new version of the system. The new version syncs only the .tar.gz files, which makes it less resource hungry, and this also adds a natural ability to commit per packages :) I still need to test out deleting functionality, so if you could run the cleanup script at some point today, that would be awesome.
Just ran a clean up for you.
On Sun, Feb 6, 2011 at 4:43 PM, Loui Chang louipc.ist@gmail.com wrote:
On Sun 06 Feb 2011 14:34 -0600, Thomas Dziedzic wrote:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b206...
That was me hehe. I was curious to see what would happen. So it looks like you're just copying over what's in the filesystem. It might be a good idea to work with aur-dev to help make the source cleaner, then everyone can benefit - even those who don't use your git repo.
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
So PKGBUILDs aren't going to be able to be viewed?
On Sun, Feb 6, 2011 at 5:19 PM, Thomas Dziedzic gostrc@gmail.com wrote:
On Sun, Feb 6, 2011 at 4:43 PM, Loui Chang louipc.ist@gmail.com wrote:
On Sun 06 Feb 2011 14:34 -0600, Thomas Dziedzic wrote:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b206...
That was me hehe. I was curious to see what would happen. So it looks like you're just copying over what's in the filesystem. It might be a good idea to work with aur-dev to help make the source cleaner, then everyone can benefit - even those who don't use your git repo.
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
So PKGBUILDs aren't going to be able to be viewed?
Nvm, I read your mail wrong.
On Sun 06 Feb 2011 17:19 -0600, Thomas Dziedzic wrote:
On Sun, Feb 6, 2011 at 4:43 PM, Loui Chang louipc.ist@gmail.com wrote:
On Sun 06 Feb 2011 14:34 -0600, Thomas Dziedzic wrote:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
2011/2/6 Lukáš Jirkovský l.jirkovsky@gmail.com:
What's this? http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more: apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b206...
That was me hehe. I was curious to see what would happen. So it looks like you're just copying over what's in the filesystem. It might be a good idea to work with aur-dev to help make the source cleaner, then everyone can benefit - even those who don't use your git repo.
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
So PKGBUILDs aren't going to be able to be viewed?
Yes they will, but not the other files.
On Sun, 6 Feb 2011 17:43:59 -0500, Loui Chang louipc.ist@gmail.com wrote:
You probably want to grab the tarballs, and extract what's in those. The next release of the AUR will only have tarballs and PKGBUILDs. The other files won't be extracted.
Would it be possible to also extract .install scripts? I usually read those before installing AUR packages since they don't come from a trusted source.
On 2/6/11, Thomas Dziedzic gostrc@gmail.com wrote:
Hey there Archers, I decided to work on this little project over the weekend. It is a complete git clone of the AUR from the source and it will be updated regularly (at least once a day) though probably as frequent as 30minutes (please see `Side Notes' for explanation). What this means is that history for the AUR will now be possible! Also, for those of you wanting to have an abs like tree for all of the aur, it is now possible. I hope this opens up new opportunities, and new possibilities for everyone.
This is awesome. Glad to see some official support for such crazy stuff. The change sets look much more impressive than I thought they would. There is still the Hard Problem of getting write access control for 20k accounts and grafting a comment system onto git (both of which were really discouraging when I tried solving it) but today is a huge win for CLI usability in Arch.
I am also glad to see it done, as now version tracking can be removed from the list of silly-things-for-AUR3-to-try as there is no reason for me to duplicate efforts :-)
-Kyle http://kmkeen.com
aur-general@lists.archlinux.org