On Sat 26 May 2018 at 21:12, Eli Schwartz via aur-general wrote:

On 05/26/2018 08:37 PM, Ista Zahn via aur-general wrote:

...

Hi Alex,

Thanks for doing this. However, it seems your script does not correctly specify system dependencies, which IMO is the main benefit of using pacman instead of install.packages. For example, the sf package should depend on gdal, goes, and proj. Do you plan to fix all these dependencies?

Best, Ista

This is why anatolik's quarry repo whitelists the packages it will build, and includes a configuration system for teaching ruby gems how to depend on system dependencies, inject necessary patches, and do other overrides for gems which don't work using just the barebones generated PKGBUILD.

Which is sort of the point here -- it's ridiculous to autogenerate 13K PKGBUILDs, then upload them to the AUR without testing them.

Thanks for the feedback. I was under the (apparently mistaken) impression that CRAN didn't allow packages that required system dependencies other than R and a few compilers. I'll look into if there's a way to get that info. Alex

On Sat 26 May 2018 at 21:24, Eli Schwartz via aur-general wrote:

On 05/26/2018 12:04 PM, Alex Branham via aur-general wrote:

...

Hi all -

I'd like to use pacman to manage R packages, similar to how you can for other languages (e.g. the python packages are all named python-*). There are some packages available already, but not many (I count about 130 currently). I wrote a script[1] that generates PKGBUILDS for all the packages on the Comprehensive R Archive Network (CRAN). There's 12,593 packages on CRAN as of today.

If you evaluate the script, then call write_all_pkgbuilds(), R will put each PKGBUILD into PKGBUILDS/r-cran-<pkgname>/PKGBUILD. Here's an example of the output.[2] It doesn't generate .SRCINFO or setup a git repo (yet). It also doesn't take into account (yet) that there are already 130ish R packages on the AUR. I named them r-cran-* rather than r-* because 1) it's clearer where the packages are coming from and 2) it's a pain to search for r-*

Is this something people would like to have in the AUR? I figured I should ask before adding ~12.5k packages.

Any and all comments/feedback appreciated!

Thanks, Alex

If you're generating PKGBUILDs for upload to the AUR, they should be well-written. As observed, injecting "cran" into the name is confusing, I think anyone looking for an R package will know how to find it based on the rest of the name component.

Yes, I think everyone prefers r-* to r-cran-*. That's easy enough to change.

I doubt anyone is just looking randomly to see which R packages are available, but if they are, the most effective way would be to download https://aur.archlinux.org/packages.gz and use regular expressions to find packages matching the pattern ^r-

It would be nice if the web search interface supported things like this.

This would find the regardless of whether they mentioned cran, and in a highly effective manner.

...

Footnotes: [1] https://github.com/jabranham/r-cran-pkgbuilds/blob/master/gen-R-PKGBUILDS.R [2]

# Maintainer: Alex Branham <email-address-removed> _cranname=A3 _cranver=1.0.0 pkgname=r-cran-a3 pkgver=1.0.0

Why is cryanver different from pkgver?

CRAN versions can contain characters that aren't allowed in PKGBUILDS such as hyphens.

...

pkgrel=1 pkgdesc="Supplies tools for tabulating and analyzing the results of predictive models. The methods employed are applicable to virtually any predictive model and make comparisons between different methodologies straightforward."

pkgdesc should be 80 chars, not 218

Thanks. I'll work on shortening the pkgdesc to around 80 characters.

...

url="https://cran.r-project.org/web/packages/${_cranname}/index.html" arch=('x86_64') license=('GPL') depends=('r-cran-xtable' 'r-cran-pbapply')

optdepends=('r-cran-randomforest' 'r-cran-e1071')

source=("https://cran.r-project.org/src/contrib/${_cranname}_${_cranver}.tar.gz") md5sums=('027ebdd8affce8f0effaecfcd5f5ade2') package() { mkdir -p ${pkgdir}/usr/lib/R/library cd ${srcdir} R CMD INSTALL ${_cranname} -l ${pkgdir}/usr/lib/R/library }

This compiles source code into x86_64 ELF executable code, in the package() step?

Is your question here about the architecture? It automatically detects that during R CMD INSTALL.

Also ${srcdir} without quotes will guaranteedly break on at least some users who build from directories which contain spaces. You carefully quote array assignments, which do *not* contain spaces (note, I'm not saying this was the wrong decision!) then neglect to do so for the only two variables which contain content not under your control!

Fixed, thanks! Thank you very much for the feedback, Eli. I don't have a ton of experience writing PKGBUILDs so this was very helpful! Alex

On Sun 27 May 2018 at 09:17, Eli Schwartz wrote:

On 05/27/2018 10:03 AM, Alex Branham wrote:

...

...

I doubt anyone is just looking randomly to see which R packages are available, but if they are, the most effective way would be to download https://aur.archlinux.org/packages.gz and use regular expressions to find packages matching the pattern ^r-

It would be nice if the web search interface supported things like this.

https://bugs.archlinux.org/task/49099

Voted, thanks.

...

...

...

# Maintainer: Alex Branham <email-address-removed> _cranname=A3 _cranver=1.0.0 pkgname=r-cran-a3 pkgver=1.0.0

Why is cryanver different from pkgver?

CRAN versions can contain characters that aren't allowed in PKGBUILDS such as hyphens.

Hmm, I think it's more common to use e.g. pkgver=${_pkgver//-/.}, then use _pkgver everywhere where file/directory names are needed (_pkgver or _cranver, either one, obviously _pkgver will be more likely to be used in places that don't actually use cran...)

Less hardcoding, easier to grok the relationship between the two if you set the pkgver from the cranver rather than independently hardcoding it too.

This makes sense. I started out setting pkgver in the R script because I wasn't sure how much I would need to change cranver. I'll look over a few though and see if it's just as easy to set it in the PKGBUILD though.

...

...

...

pkgdesc="Supplies tools for tabulating and analyzing the results of predictive models. The methods employed are applicable to virtually any predictive model and make comparisons between different methodologies straightforward."

pkgdesc should be 80 chars, not 218

Thanks. I'll work on shortening the pkgdesc to around 80 characters.

Yeah, this one will be... problematic, if you're just using upstream descriptions in a generated script, but it's well worth doing correctly (often manually), because the alternative is some fairly ugly pacman -Si output.

Hopefully you can figure out some way that does not require writing all 13K out by hand... though this won't be as much of a problem if you use a whitelist.

Yes, I'm not looking forward to this part :-(

...

...

...

package() { mkdir -p ${pkgdir}/usr/lib/R/library cd ${srcdir} R CMD INSTALL ${_cranname} -l ${pkgdir}/usr/lib/R/library }

This compiles source code into x86_64 ELF executable code, in the package() step?

Is your question here about the architecture? It automatically detects that during R CMD INSTALL.

No, I assume that all cran packages are ELF executable code, unlike how ruby or python can be interpreted scripts marked as "any".

R packages do not usually (ever? need to check...) produce executable files. R CMD INSTALL copies over the package to a library (AKA a directory. R's term for where packages live on your system). In the latest version of R (3.5.0), they've also started byte compiling it by default.

(In those cases, well, quarry has a function to detect whether the package contains native extensions, and sets the arch=() of the generated PKGBUILD as appropriate.)

Is this something I need to worry about since Arch only supports x86_64 now? So far I've just been setting arch=('x86_64') in the PKGBUILDS.

The issue is that you're supposed to build during build(), which occurs out of fakeroot and makes the --repackage option make sense (it assumes you already built during build, and just repackages the results, which should result in completely identical contents).

I'm not sure whether this is possible or not but I'll look into it. Thanks again, Alex

Thanks for your help. I've adopted a few R packages in the AUR[1] in the meantime and updated them with the output of this automated script. I've included an example of a current one below. Note that the r-cran-* stuff isn't mine; they're packages other people have already uploaded to the AUR. I think I've managed to automate everything, the only exception being that there's no way (as far as I can see, anyway) to tell if a package has non-R dependencies (e.g. some packages need gcc-fortran to be installed, but there's no way to tell that automatically I think). Thanks again everyone for your help! Alex Footnotes: [1] https://aur.archlinux.org/packages/?K=jabranham&SeB=m # Maintainer: Alex Branham <--address@hidden--> _cranname=zoo _cranver=1.8-1 _pkgtar=${_cranname}_${_cranver}.tar.gz pkgname=r-zoo pkgver=${_cranver//[:-]/.} pkgrel=1 pkgdesc="S3 Infrastructure for Regular and Irregular Time Series (Zs Ordered Observations)" arch=('any') url="https://cran.r-project.org/web/packages/${_cranname}/index.html" license=('GPL') depends=('r' ) optdepends=('r-cran-coda' 'r-cran-chron' 'r-daag' 'r-fts' 'r-cran-ggplot2' 'r-mondate' 'r-cran-scales' 'r-strucchange' 'r-timedate' 'r-timeseries' 'r-tis' 'r-tseries' 'r-xts') source=("https://cran.r-project.org/src/contrib/${_pkgtar}") md5sums=('bf8789b3a448b5bed39e54e027ee7c6a') build(){ R CMD INSTALL ${_pkgtar} -l $srcdir } package() { install -d "$pkgdir/usr/lib/R/library" cp -r "$srcdir/$_cranname" "$pkgdir/usr/lib/R/library" }

On Wed 30 May 2018 at 14:31, Jagannathan Tiruvallur Eachambadi via aur-general wrote:

License in at least one of package is wrong. I haven't checked the others but if they are generated by your script you might want to look into it. In r-tidyverse, license is specified as "GPL-3 | file LICENSE" which doesn't exists.

Instead of "license=('GPL-3 | file LICENSE')" use "license=('GPL3' 'custom')" if there are other custom licenses involved. "license" in the PKGBUILD is an array so each license has to be specified separately.

Thanks. I've been meaning to look into this. I'm pretty sure that "GPL-3 or file LICENSE" just means GPL3 for most of them, but that they also bundle the GPL file. That's the case for r-tidyverse, for example. I just need to come up with a way to check it for the other 12,500 packages.

Also consider adopting the r-cran-* packages or ask the maitainers to convert them to r-$modulename since it will be consistant with what was discussed earlier.

I've adopted some that were very outdated and renamed them already. I'll email the other maintainers to see if they'll rename.

On Wed 30 May 2018 at 20:53, Eli Schwartz wrote:

So, I looked into this a bit.

https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#The-DESCRIPTION-f...

Looks like the rule is, if NeedsCompilation = yes, then the package contains C or Fortran code which must be compiled into an ELF library foo.so In this case, the PKGBUILD should specify arch=('x86_64') and additionally any other architectures you feel comfortable supporting in the AUR package.

In the case where NeedsCompilation = no, then the package contains only R code which is byte-compiled, but can run on any architecture and should therefore specify arch=('any')

Thanks for looking into this. I had assumed that NeedsCompilation dealt with byte compiling the R code (only in the most recent R version did byte compilation get enabled by default). I've updated the R packages I maintain to take advantage of this. Alex