[arch-dev-public] Improving the package guidelines
Yo! In April there was some discussion regarding how to properly do unit testing in Python PKGBUILDs[1]. Felix had some amazing notes that was super useful. But as pointed out we don't really strive to document these things[2]. To try help and spark some interest in improving the current package guidelines, I have spent a some time restructuring and rewriting parts of the Python and Golang package guidelines. I'd appreciate reviews on them! https://wiki.archlinux.org/index.php/User:Foxboron/Python_packaging_guidelin... https://wiki.archlinux.org/index.php/User:Foxboron/Go_packaging_guidelines It really doesn't take that much time to note down what you know about packaging and what the current standards are. I think this helps both fellow maintainers and AUR maintainers in writing better PKGBUILDs and sharing knowledge. If people think editing wikis are annoying, maybe we could setup a github repository with example PKGBUILDs we can work on? [1]: https://lists.archlinux.org/pipermail/arch-dev-public/2018-April/029228.html [2]: https://lists.archlinux.org/pipermail/arch-dev-public/2018-April/029233.html -- Morten Linderud PGP: 9C02FF419FECBE16
On 05/22/2018 04:25 PM, Morten Linderud via arch-dev-public wrote:
Yo!
In April there was some discussion regarding how to properly do unit testing in Python PKGBUILDs[1]. Felix had some amazing notes that was super useful. But as pointed out we don't really strive to document these things[2]. To try help and spark some interest in improving the current package guidelines, I have spent a some time restructuring and rewriting parts of the Python and Golang package guidelines. I'd appreciate reviews on them!
https://wiki.archlinux.org/index.php/User:Foxboron/Python_packaging_guidelin... https://wiki.archlinux.org/index.php/User:Foxboron/Go_packaging_guidelines
So there's been some recent activity on the wiki pages in question. Primarily, golang seemingly needs to be copied entirely due to not supporting symlinks or some such. Also I'd like to discuss what seems to me a common misconception in python packaging -- specifically, the use of cp -r source source-py2 and building both separately. Best I can tell, this is primarily motivated by fear of py2/py3 specific bytecode being generated, but the thing is, this bytecode is all generated during package() inside of "$pkgdir" during the install step. You can check the contents of the build/ directory... it just copies the .py files and builds any ext_modules. The one case where this would make a difference is where 2to3 is being used, so depending on which version of python you use the actual source code is modified. I think the guidelines should only mention that "if 2to3 is used, you'll need to make copies of the source [...]". Honestly I consider this copying around when unnecessary to be the equivalent of inserting "sleep 5" everywhere. 2to3 being run as part of setup.py build is hardly the common case, it's not even recommended at all, really. -- Eli Schwartz Bug Wrangler and Trusted User
On 06/29/2018 05:06 AM, Eli Schwartz via arch-dev-public wrote:
Also I'd like to discuss what seems to me a common misconception in python packaging -- specifically, the use of cp -r source source-py2 and building both separately.
Best I can tell, this is primarily motivated by fear of py2/py3 specific bytecode being generated, but the thing is, this bytecode is all generated during package() inside of "$pkgdir" during the install step. You can check the contents of the build/ directory... it just copies the .py files and builds any ext_modules.
The main reason behind this (or why I started doing this) was sometimes the tests run inside the sources tree made both versions' .pyc files into the tree itself, and finally end up in the package. I had this problem for several times and end up having the separated build directories as template for new packages. This seems not the case for most packages now, so I agree that it should be avoided. -- Regards, Felix Yan
On 06/28/2018 05:59 PM, Felix Yan wrote:
The main reason behind this (or why I started doing this) was sometimes the tests run inside the sources tree made both versions' .pyc files into the tree itself, and finally end up in the package. I had this problem for several times and end up having the separated build directories as template for new packages.
This seems not the case for most packages now, so I agree that it should be avoided.
Hmm, that's quite interesting. From what I've seen, both pytest and `python setup.py test` (at least today) use the copy of the code from outside of build/ when running unittests, which I suppose probably makes sense as it is running right alongside the unittests themselves. -- Eli Schwartz Bug Wrangler and Trusted User
On 06/28/2018 05:06 PM, Eli Schwartz wrote:
The one case where this would make a difference is where 2to3 is being used, so depending on which version of python you use the actual source code is modified. I think the guidelines should only mention that "if 2to3 is used, you'll need to make copies of the source [...]". Honestly I consider this copying around when unnecessary to be the equivalent of inserting "sleep 5" everywhere.
2to3 being run as part of setup.py build is hardly the common case, it's not even recommended at all, really.
Latest revision to these guidelines clarifying the 2to3 case specifically: https://wiki.archlinux.org/index.php?title=User:Foxboron/Python_packaging_guidelines&diff=529424&oldid=528011 So far the page seems pretty good, time to consider merging it I guess. -- Eli Schwartz Bug Wrangler and Trusted User
Helluw! The rewrites have been up for a few months now and I intend to merge them soon. Feel free to still review them, either with a reply on the ML or on IRC. Whatever you prefer :) Alex Branham has also written up a page detailing some R package guidelines; https://wiki.archlinux.org/index.php/R_package_guidelines. I still have some hopes that other TUs and Devs could help fix up missing information in the guidelines. -- Morten Linderud PGP: 9C02FF419FECBE16
[2018-08-29 20:20:29 +0200] Morten Linderud via arch-dev-public:
The rewrites have been up for a few months now and I intend to merge them soon. Feel free to still review them, either with a reply on the ML or on IRC. Whatever you prefer :)
Sorry but I don't recall such a thing ever being discussed on this list. What rewrites are we talking about? Cheers. -- Gaetan
On 8/29/18 6:35 PM, Gaetan Bisson via arch-dev-public wrote:
[2018-08-29 20:20:29 +0200] Morten Linderud via arch-dev-public:
The rewrites have been up for a few months now and I intend to merge them soon. Feel free to still review them, either with a reply on the ML or on IRC. Whatever you prefer :)
Sorry but I don't recall such a thing ever being discussed on this list. What rewrites are we talking about?
The email from May 22nd that began this email thread, specifically, the ArchWiki packaging guidelines for Python and Golang. You can see the work that has been done, by looking at Foxboron's User namespace on the wiki. -- Eli Schwartz Bug Wrangler and Trusted User
[2018-08-29 18:42:39 -0400] Eli Schwartz via arch-dev-public:
On 8/29/18 6:35 PM, Gaetan Bisson via arch-dev-public wrote:
[2018-08-29 20:20:29 +0200] Morten Linderud via arch-dev-public:
The rewrites have been up for a few months now and I intend to merge them soon. Feel free to still review them, either with a reply on the ML or on IRC. Whatever you prefer :)
Sorry but I don't recall such a thing ever being discussed on this list. What rewrites are we talking about?
The email from May 22nd that began this email thread, specifically, the ArchWiki packaging guidelines for Python and Golang.
You can see the work that has been done, by looking at Foxboron's User namespace on the wiki.
Ah, my mistake. I thought you were talking about general packaging guidelines, not language specific ones. Sorry for the noise. Cheers. -- Gaetan
participants (4)
-
Eli Schwartz
-
Felix Yan
-
Gaetan Bisson
-
Morten Linderud