Re: [aur-general] AUR Best Practice for New Package Upload
-------------------------------------------- On Sat, 9/6/14, Johannes Löthberg <johannes@kyriasis.com> wrote: Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: "Discussion about the Arch User Repository (AUR)" <aur-general@archlinux.org> Date: Saturday, September 6, 2014, 1:03 PM On 06 Sep 2014, at 12:06, stef <stef_204@yahoo.com> wrote:
1) the main script which is CLI (available for python2 and/or python3)
2) a GTK+ GUI (optional) for python2
3) the GTK+ GUI (optional) for python3
I use the GTK+GUI for
use python2.
Question 1):
The program is small (approx 250kb only) and my tendency would be to just do the one package with all 3 components which users will then use as they see fit, CLI only or GUI, etc. This would allow me to
python3 on my box but perhaps some users might want (or have to) maintain only one package for this program.
Is only one package OK
Question 2) I just basically downloaded the source tarball, extracted it to /opt and am running it from there.
My idea is to achieve
with the AUR community? You shouldn’t mix python2 and 3. Make it a split package. the same with PKGBUILD, following proper protocol, etc.
Is “installing”
(extracting dir structure) to /opt a problem? Don’t do that, it’s very ewwy. Read up on how and where python packages are normally packaged. Or actually, read up on setuptools and write a setup.py that you can use in the PKGBUILD. =========================== @Johannes @nuno @list Sorry for not snipping the above quotes, but I wanted to leave all of the relevant info which might help. This is the program <https://github.com/jdleicher/MassHash/> MassHash: A set of file hashing tools. To be clear, this program, for which I am trying to build my first AUR PKGBUILD (bear with me please, still learning) is NOT a python package or library. So perhaps a setup.py may not necessarily be needed or relevant in this case. (I could be wrong.) 1. The program is not "expanding/extending" the Python Language itself. In other words it is not installing TO or writing code FOR Python, but USING the language for an end user application. Again, it is a PROGRAM, not a "python package". 2. Both the CLI and GUI are programs and not libraries. It might make sense to install a python library/package to the language installation itself so that other scripts can easily "import" said library/package into the code. But possibly not in this case. Right now, personally, on my box, I have the package as so (I just downloaded it since it is self-contained, there is nothing to build; and am just running it from there): /opt/MassHash /opt/MassHash/python2 /opt/MassHash/python2/masshash-launcher /opt/MassHash/python2/masshash /opt/MassHash/data /opt/MassHash/data/masshash-launcher.glade /opt/MassHash/data/masshash-launcher.svg /opt/MassHash/README /opt/MassHash/python3 /opt/MassHash/python3/masshash-launcher /opt/MassHash/python3/masshash /opt/MassHash/AUTHORS /opt/MassHash/COPYING *The program is self-contained.* In each of /opt/MassHash/python2/ and /opt/MassHash/python3/ there are 2 executables: masshash for the CLI and masshash-launcher to launch the program with the GTK+ GUI. So looking at <https://wiki.archlinux.org/index.php/Python_Package_Guidelines> I am not sure this applies here. And looking at <https://wiki.archlinux.org/index.php/Arch_Packaging_Standards#Directories> one idea would be to install as such: /usr/share/program/ /usr/share/program/data/ /usr/share/masshash/data/masshash-launcher.glade /usr/share/masshash/data/masshash-launcher.svg /usr/share/masshash/python3/ /usr/share/masshash/python3/masshash /usr/share/masshash/python3/masshash-launcher /usr/share/doc/ /usr/share/doc/masshash/AUTHORS /usr/share/doc/masshash/COPYING /usr/share/doc/masshash/README.md Then the PKGBUILD script creates another script or symlink which is placed in /usr/bin Would either /opt or /usr/share as above work or do you guys confirm that, according to AUR packaging standards it should have a setup.py ? (Johannes has already told me not to use /opt but I had not given the full data previously. Johannes, based on the additional info given here, feel free to confirm your earlier recommendation; or change it, as the case may be.) Guys, please also, let me know if the AUR forum might be better suited for this discussion and if so, I will move it there. Thanks a lot for your feedback and patience.
* stef <stef_204@yahoo.com> [2014-09-16 03:31:13 -0700]:
This is the program <https://github.com/jdleicher/MassHash/> MassHash: A set of file hashing tools.
To be clear, this program, for which I am trying to build my first AUR PKGBUILD (bear with me please, still learning) is NOT a python package or library.
Usually, python applications have a proper setup.py as well. However in this case (as upstream doesn't provide one and the script is two single files) IMHO it's unnecessary to write one.
Right now, personally, on my box, I have the package as so (I just downloaded it since it is self-contained, there is nothing to build; and am just running it from there):
/opt/MassHash /opt/MassHash/python2 /opt/MassHash/python2/masshash-launcher /opt/MassHash/python2/masshash /opt/MassHash/data /opt/MassHash/data/masshash-launcher.glade /opt/MassHash/data/masshash-launcher.svg /opt/MassHash/README /opt/MassHash/python3 /opt/MassHash/python3/masshash-launcher /opt/MassHash/python3/masshash /opt/MassHash/AUTHORS /opt/MassHash/COPYING
*The program is self-contained.*
As said previously: installing to /opt really only should be done for complex packages which are very difficult to impossible to get running otherwise.
And looking at <https://wiki.archlinux.org/index.php/Arch_Packaging_Standards#Directories> one idea would be to install as such:
/usr/share/program/ /usr/share/program/data/ /usr/share/masshash/data/masshash-launcher.glade /usr/share/masshash/data/masshash-launcher.svg /usr/share/masshash/python3/ /usr/share/masshash/python3/masshash /usr/share/masshash/python3/masshash-launcher /usr/share/doc/ /usr/share/doc/masshash/AUTHORS /usr/share/doc/masshash/COPYING /usr/share/doc/masshash/README.md
Then the PKGBUILD script creates another script or symlink which is placed in /usr/bin
I'd install it like this: - AUTHORS and README.md to /usr/share/doc/masshash. COPYING probably isn't necessary as it's a standard GPL3 license anyways. - The python3 masshash and masshash-launcher scripts to /usr/bin directly - no need to create another wrapper around them. - The python2 scripts not installed at all. I think it's reasonable to say nobody will want to run Arch with python2 only but not python3. - The masshash-launcher.glade and masshash-launcher.svg files directly to /usr/share/masshash, as the script searches them there if they don't exist relatively to the script: https://github.com/jdleicher/MassHash/blob/master/python3/masshash-launcher#... Florian -- http://www.the-compiler.org | me@the-compiler.org (Mail/XMPP) GPG 0xFD55A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/
-------------------------------------------- On Tue, 9/16/14, Florian Bruhin <me@the-compiler.org> wrote: Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: aur-general@archlinux.org Date: Tuesday, September 16, 2014, 1:07 PM [snip] However in this case (as upstream doesn't provide one and the script is two single files) IMHO it's unnecessary to write one.
*The program is self-contained.*
As said previously: installing to /opt really only should be done for complex packages which are very difficult to impossible to get running otherwise. I'd install it like this: - AUTHORS and README.md to /usr/share/doc/masshash. COPYING probably isn't necessary as it's a standard GPL3 license anyways. - The python3 masshash and masshash-launcher scripts to /usr/bin directly - no need to create another wrapper around them. - The python2 scripts not installed at all. I think it's reasonable to say nobody will want to run Arch with python2 only but not python3. - The masshash-launcher.glade and masshash-launcher.svg files directly to /usr/share/masshash, as the script searches them there if they don't exist relatively to the script: https://github.com/jdleicher/MassHash/blob/master/python3/masshash-launcher#... Florian ========================================= Florian, Thanks a lot, very helpful. Will follow your suggestions and those of Nuno. Thanks to everyone which provided feedback. (I will probably send the PKGBUILD to this list for review before uploading it.)
To be clear, this program, for which I am trying to build my first AUR PKGBUILD (bear with me please, still learning) is NOT a python package or library.
So perhaps a setup.py may not necessarily be needed or relevant in this case. (I could be wrong.)
There is no obligation to have a setup.py. Usually, when it is provided by upstream, it is used to ease the installation process.
Right now, personally, on my box, I have the package as so (I just downloaded it since it is self-contained, there is nothing to build; and am just running it from there):
[...]
This is not the best approach. Usually /opt is used for *large* self-contained packages.
*The program is self-contained.*
In each of /opt/MassHash/python2/ and /opt/MassHash/python3/ there are 2 executables: masshash for the CLI and masshash-launcher to launch the program with the GTK+ GUI.
Since it's an application and not a library, one should only install the python3 version of the scripts, since this is the default version of python in ArchLinux now.
So looking at <https://wiki.archlinux.org/index.php/Python_Package_Guidelines> I am not sure this applies here.
And looking at <https://wiki.archlinux.org/index.php/Arch_Packaging_Standards#Directories> one idea would be to install as such:
/usr/share/program/ /usr/share/program/data/ /usr/share/masshash/data/masshash-launcher.glade /usr/share/masshash/data/masshash-launcher.svg /usr/share/masshash/python3/ /usr/share/masshash/python3/masshash /usr/share/masshash/python3/masshash-launcher /usr/share/doc/ /usr/share/doc/masshash/AUTHORS /usr/share/doc/masshash/COPYING /usr/share/doc/masshash/README.md
Then the PKGBUILD script creates another script or symlink which is placed in /usr/bin
You're not far from truth. I would install masshash and masshash-launcher directly in in /usr/bin. No need for the symlinks. I wouldn't install the README.md and the AUTHORS file. Usually, those files belong to the source archive and are not deployed. However, you can as well put them in /usr/share/doc/masshash if you like. I would also add a .desktop file to allow the mashhash-launcher to be run from your desktop environment without having to open a terminal for it. And finally, I would create a split package for this application. One for the console application only and one for the GUI.
Would either /opt or /usr/share as above work or do you guys confirm that, according to AUR packaging standards it should have a setup.py ? (Johannes has already told me not to use /opt but I had not given the full data previously. Johannes, based on the additional info given here, feel free to confirm your earlier recommendation; or change it, as the case may be.)
No /opt for your package ;)
Guys, please also, let me know if the AUR forum might be better suited for this discussion and if so, I will move it there.
Thanks a lot for your feedback and patience.
For the exercice, I packaged the application, and you can find the result here [1] (in the masshash directory). Feel free to inspire yourself, or simply to take it as it is if you wish. Try to study it and feel free to ask any questions you want about it. [1] https://cloud.russo79.com/public.php?service=files&t=f00d660f9ae6001dd09276e960a02e9d -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com
That sounds like the most reasonable way to build it. I would also like to suggest that Florian should read the man page for file-hierarchy as it is quite helpful if you are unsure of where files should be going. Regards, Justin Dray E: justin@dray.be M: 0433348284
-------------------------------------------- On Tue, 9/16/14, Nuno Araujo <nuno.araujo@russo79.com> wrote: Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: aur-general@archlinux.org Date: Tuesday, September 16, 2014, 1:42 PM There is no obligation to have a setup.py. Usually, when it is provided by upstream, it is used to ease the installation process. This is not the best approach. Usually /opt is used for *large* self-contained packages. Since it's an application and not a library, one should only install the python3 version of the scripts, since this is the default version of python in ArchLinux now. I would install masshash and masshash-launcher directly in in /usr/bin. No need for the symlinks. I wouldn't install the README.md and the AUTHORS file. Usually, those files belong to the source archive and are not deployed. However, you can as well put them in /usr/share/doc/masshash if you like. I would also add a .desktop file to allow the mashhash-launcher to be run from your desktop environment without having to open a terminal for it. And finally, I would create a split package for this application. One for the console application only and one for the GUI. No /opt for your package ;) For the exercice, I packaged the application, and you can find the result here [1] (in the masshash directory). Feel free to inspire yourself, or simply to take it as it is if you wish. Try to study it and feel free to ask any questions you want about it. [1] https://cloud.russo79.com/public.php?service=files&t=f00d660f9ae6001dd09276e960a02e9d -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com ================================================ Nuno, Thanks a lot for helpful feedback. I was kind of hoping no one would write a PKGBUILD for this so I would not be tempted to use a shortcut and "cheat" but hey.... ;) I will use your PKGBUILD and files as a model, really make sure I understand them fully, make any modifications as desired and post here final result (if different from yours.) I may have some questions and will post here if any. Thanks.
On mar, 2014-09-16 at 05:56 -0700, stef wrote:
Nuno,
Thanks a lot for helpful feedback. I was kind of hoping no one would write a PKGBUILD for this so I would not be tempted to use a shortcut and "cheat" but hey.... ;) I will use your PKGBUILD and files as a model, really make sure I understand them fully, make any modifications as desired and post here final result (if different from yours.) I may have some questions and will post here if any. Thanks.
I don't think that giving a look at existing PKGBUILDs is "cheating". Referring to examples is a very good way of learning. As long was you try to understand what's done and not just blindly do some copy and paste. Feel free to post your questions here. -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com
* Nuno Araujo <nuno.araujo@russo79.com> [2014-09-16 13:42:24 +0200]:
For the exercice, I packaged the application, and you can find the result here [1] (in the masshash directory). Feel free to inspire yourself, or simply to take it as it is if you wish. Try to study it and feel free to ask any questions you want about it.
[1] https://cloud.russo79.com/public.php?service=files&t=f00d660f9ae6001dd09276e960a02e9d
Some suggestions for your PKGBUILD: - You could use $pkgver inside sources=() so you don't always have to adjust the version number in two places. - You should really quote $pkgdir and $srcdir properly: http://mywiki.wooledge.org/Quotes Your PKGBUILD will break when someone uses it inside a path with spaces in it. Florian -- http://www.the-compiler.org | me@the-compiler.org (Mail/XMPP) GPG 0xFD55A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/
Some suggestions for your PKGBUILD:
- You could use $pkgver inside sources=() so you don't always have to adjust the version number in two places.
- You should really quote $pkgdir and $srcdir properly: http://mywiki.wooledge.org/Quotes Your PKGBUILD will break when someone uses it inside a path with spaces in it.
Yeah, I wrote it a bit quickly, and didn't took the time to re-read myself. ;) I uploaded a new version with these issues fixed, just so that stef has a "good example" to refer to. Thanks for the comments. -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com
-------------------------------------------- On Tue, 9/16/14, Nuno Araujo <nuno.araujo@russo79.com> wrote: Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: aur-general@archlinux.org Date: Tuesday, September 16, 2014, 3:55 PM
- You should really quote $pkgdir and $srcdir properly: http://mywiki.wooledge.org/Quotes Your PKGBUILD will break when someone uses it inside a path with spaces in it.
Yeah, I wrote it a bit quickly, and didn't took the time to re-read myself. ;) I uploaded a new version with these issues fixed, just so that stef has a "good example" to refer to. Thanks for the comments. -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com ============================ OK, thanks to you both. Taking a look at everything and will come back with questions.
-------------------------------------------- On Tue, 9/16/14, Nuno Araujo <nuno.araujo@russo79.com> wrote: Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: aur-general@archlinux.org Date: Tuesday, September 16, 2014, 1:42 PM Try to study it and feel free to ask any questions you want about it. -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com ========================================= @nuno @list So, I have studied the PKGBUILD and install file. I have some questions, a few of which might be really newb-type but please bear with me. All is clear in your PKGBUILD: and it was certainly helpful to see how you went about it; clean and simple. 1) Questions regarding the variables $pkgdir and $srcdir . My understanding: these 2 babies are directories created by makepkg during the fetch source, configure, build and install to fakeroot process. $srcdir is where makepkg stores all source files downloaded. $pkgdir is where the build/compiled files are then stored in the fakeroot environment which gets compressed into the final .pkg.tar.xz file. THAT .tar.xz file is a compressed archive which contains the directory structure and installed files which pacman will then extract to user's root system. a) Did i get this right? b) what happens to $srcdir and $pkgdir ? I imagine those are just 'temp' dirs which makepkg deletes when it's done compressing the fakeroot? 2) in the PKGBUILD, you are dropping the data/ directory and installing both .svg and .glade files to /usr/share/masshash-launcher/ In python3/ masshash-launcher: L31 #path to glade ui file L32 glade_source_dir = '../data/masshash-launcher.glade' L33 if os.path.isfile(glade_source_dir): L34 glade = glade_source_dir L35 else: L36 glade = '/usr/share/masshash-launcher/masshash-launcher.glade' So you're using L36 as opposed to 32. This is your preference? Or for the sake of simplicity? So as not to create an additional dir "data" as unnecessary? (Sorry not trying to be "anal" but only to understand.) 3) You are copying the svg file to /usr/share/icons/hicolor/scalable/apps/ Then you are using the install file to update the gtk icon cache--which I didn't even think of and makes a lot of sense. Why the hicolor though? Your choice? Because it is the most basic/core GTK icon theme? Sorry, but this is all a bit new to me. 4) Not important; for the CLI program, the category I would use to upload to AUR would be "system" but since we also have the GUI, (which would be what X11?), which category would you recommend when uploading to AUR? X11? 5) @nuno : Do you want to be listed as contributor? Would only be (more than) fair (eeer...) but might be a bit of a PITA for you....Just let me know. Thanks.
On jeu, 2014-09-18 at 11:26 -0700, stef wrote:
-------------------------------------------- On Tue, 9/16/14, Nuno Araujo <nuno.araujo@russo79.com> wrote:
Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: aur-general@archlinux.org Date: Tuesday, September 16, 2014, 1:42 PM
Try to study it and feel free to ask any questions you want about it.
-- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com
========================================= @nuno @list
So, I have studied the PKGBUILD and install file.
I have some questions, a few of which might be really newb-type but please bear with me.
All is clear in your PKGBUILD: and it was certainly helpful to see how you went about it; clean and simple.
1) Questions regarding the variables $pkgdir and $srcdir .
My understanding: these 2 babies are directories created by makepkg during the fetch source, configure, build and install to fakeroot process. $srcdir is where makepkg stores all source files downloaded. $pkgdir is where the build/compiled files are then stored in the fakeroot environment which gets compressed into the final .pkg.tar.xz file. THAT .tar.xz file is a compressed archive which contains the directory structure and installed files which pacman will then extract to user's root system.
a) Did i get this right?
More or less, the source files are into $SRCDEST, which defaults to the current directory. (you should take a look at makepkg.conf man page). $srcdir is a folder where the source files are uncompressed (or symlinked). As for $pkgdir, you are right. It should be noted that besides the built/compiled files, the .install files as well as a .PKGINFO file providing the package metadata is also in that directory.
b) what happens to $srcdir and $pkgdir ? I imagine those are just 'temp' dirs which makepkg deletes when it's done compressing the fakeroot?
makepkg doesn't delete them. It's your work.
2) in the PKGBUILD, you are dropping the data/ directory and installing both .svg and .glade files to /usr/share/masshash-launcher/
In python3/ masshash-launcher:
L31 #path to glade ui file L32 glade_source_dir = '../data/masshash-launcher.glade' L33 if os.path.isfile(glade_source_dir): L34 glade = glade_source_dir L35 else: L36 glade = '/usr/share/masshash-launcher/masshash-launcher.glade'
So you're using L36 as opposed to 32.
This is your preference? Or for the sake of simplicity? So as not to create an additional dir "data" as unnecessary?
(Sorry not trying to be "anal" but only to understand.)
Neither. If you give it a look, you'll see that L32 refers to a relative path. This is usually used when developing a program. While coding, you would like to access your work version of the files. This allows work to be done on a "development" version of the program without affecting the one installed on your system.
3) You are copying the svg file to /usr/share/icons/hicolor/scalable/apps/ Then you are using the install file to update the gtk icon cache--which I didn't even think of and makes a lot of sense. Why the hicolor though? Your choice? Because it is the most basic/core GTK icon theme?
Sorry, but this is all a bit new to me.
If you give a look at the contents of the hicolor-icon-theme package, you will see that it has no icons by itself. Only a "standard" set of directories where icons should to be stored and a index file describing these directories. Having this theme installed ensure that you can call gtk-update-icon-cache without issues. This "theme" is usually the standard place to store your application icons (at least in a XDG "compatible" desktop).
4) Not important; for the CLI program, the category I would use to upload to AUR would be "system" but since we also have the GUI, (which would be what X11?), which category would you recommend when uploading to AUR? X11?
My personal opinion: I don't care about AUR categories. But if I had to choose between both, I would pick "system".
5) @nuno : Do you want to be listed as contributor? Would only be (more than) fair (eeer...) but might be a bit of a PITA for you....Just let me know.
Don't care. Seing the time and effort you're putting to study a single and simple PKGBUILD, you deserve to take all the credit for it.
Thanks.
As a generic advice, take the time to read the man pages of Arch Linux build system utilities and configuration files (e.g. man PKGBUILD, man makepkg, man pacman, man pacman.conf, and so on...). Also read other AUR PKGBUILDs (or even better, the one published in core and extra repositories). There you would have find answers for some of the questions you asked. -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com
-------------------------------------------- On Thu, 9/18/14, Nuno Araujo <nuno.araujo@russo79.com> wrote: Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: aur-general@archlinux.org Date: Thursday, September 18, 2014, 10:00 PM As a generic advice, take the time to read the man pages of Arch Linux build system utilities and configuration files (e.g. man PKGBUILD, man makepkg, man pacman, man pacman.conf, and so on...). Also read other AUR PKGBUILDs (or even better, the one published in core and extra repositories). There you would have find answers for some of the questions you asked. -- Nuno Araujo <nuno.araujo@russo79.com> http://www.russo79.com ================================ OK, got all of that, thanks again for feedback and guidance. Oh, and I did read the man pages of makepkg and pacman maybe not all of the data but whatever I could find that applied to my questions but it's always good to double check my understanding (or misunderstanding, as the case may be), since I'm still learning. I always do that before posting but again, not so obvious at times. I think I'm pretty much ready to go here and will proceed to upload soon. Thanks.
makepkg doesn't delete them. It's your work.
Not by default, but if you call makepkg with the -c flag, it will perform a clean-up of those temp directories. -- David Phillips GPG Key 0x7BF3D17D0884BF5B Fingerprint 2426 235A 7831 AA2F 56AF 4BC0 7BF3 D17D 0884 BF5B
-------------------------------------------- On Fri, 9/19/14, David Phillips <dbphillipsnz@gmail.com> wrote: Subject: Re: [aur-general] AUR Best Practice for New Package Upload To: "Discussion about the Arch User Repository (AUR)" <aur-general@archlinux.org> Date: Friday, September 19, 2014, 8:27 AM
makepkg doesn't delete them. It's your work.
Not by default, but if you call makepkg with the -c flag, it will perform a clean-up of those temp directories. -- David Phillips GPG Key 0x7BF3D17D0884BF5B Fingerprint 2426 235A 7831 AA2F 56AF 4BC0 7BF3 D17D 0884 BF5B ================================ OK, I see that option: -c, --clean Clean up work files after build Anyway, I should have known better than to ask that question....I have built quite a few AUR packages locally and I always see the pkg and src dirs in along with the .pkg.tar.xz file, etc.; all the time..... when I go to pacman -U to install. So the answer to that question was pretty obvious; just got confused there for a minute. I do need to read up more on how $SRCDEST , $PKGDIR and $SRCDIR are defined and work together, etc. (Just a matter of finding the right man page sections....) Thanks.
participants (5)
-
David Phillips
-
Florian Bruhin
-
Justin Dray
-
Nuno Araujo
-
stef