Re: [aur-general] TU application: Daurnimator
On 12/11/18 8:30 PM, Alad Wenter wrote:
Since the discussion period is about to end without much discussion...
Right now the rate of new applications is very high - about 2 new applications per month. That makes a thorough review difficult.
Considering the positive experiences of the sponsor, it would be a shame to let a voting period pass. That said, I'm not sure we have sufficient information - at present - to proceed with such a voting period in a meaningful manner.
So about Foxboron's question for confirmation: "Say one or two people confirm they think the voting process should be continued after the discussion has ended?" - I don't know.
Alad
Let's try to get the ball rolling by asking some questions.
1. When I look at LUA modules, I see that most are available on "luarocks", which is apparently a package manager for LUA. Can you leverage this to make more LUA modules available on Arch?
2. You have some AUR packages for LUA modules of your own making, yet they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
3. I have no idea on what some of your more complicated packages do, or why they would require said complexity, e.g. iup. [2] Perhaps you could explain a bit on that regard.
4. Related to the above, there are no current packaging guidelines for LUA packages. [3] Do you plan on starting an effort (possibly with other LUA package maintainers) to remedy this?
[1] https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=lua-compat53 [2] https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=iup [3] https://wiki.archlinux.org/index.php/Arch_package_guidelines#Additional_guid...
Alad
On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general aur-general@archlinux.org wrote:
- When I look at LUA modules, I see that most are available on
"luarocks", which is apparently a package manager for LUA. Can you leverage this to make more LUA modules available on Arch?
Note that it's "Lua" not LUA.
I'd like to leverage luarocks to build lua packages, however it maintains its own local database of installed packages which makes things awkward. I don't want to introduce luarocks as a non-build-time dependency. Follow https://github.com/luarocks/luarocks/issues/671 for possible progress in this area.
- You have some AUR packages for LUA modules of your own making, yet
they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
The upstream packages do not ship a makefile; they "officially" only support luarocks for building. Due to the above-mentioned issue, I don't see it as feasible to use luarocks for AUR packages at the moment, so I had to hard-code the build commands into the PKGBUILD.
- I have no idea on what some of your more complicated packages do, or
why they would require said complexity, e.g. iup. [2] Perhaps you could explain a bit on that regard.
IUP is a graphical toolkit similar to gtk, qt or wxwidgets with many components. The upstream do not want to support dynamic linking for some pieces, so I need to patch out their (custom) build system 'tecmake'.
The release tarballs include lua bindings to the C IUP library; some complexity in the PKGBUILD comes from compiling necessary components for each of lua 5.1, 5.2 and 5.3.
- Related to the above, there are no current packaging guidelines for
LUA packages. [3] Do you plan on starting an effort (possibly with other LUA package maintainers) to remedy this?
Yes :) If anyone would like to help out with this I'm all ears.
On 12/11/18 9:00 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general aur-general@archlinux.org wrote:
- You have some AUR packages for LUA modules of your own making, yet
they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
The upstream packages do not ship a makefile; they "officially" only support luarocks for building.
You are upstream, you have the power to make a change for the better
On Tue, 11 Dec 2018 at 12:04, Robin Broda via aur-general aur-general@archlinux.org wrote:
On 12/11/18 9:00 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general aur-general@archlinux.org wrote:
- You have some AUR packages for LUA modules of your own making, yet
they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
The upstream packages do not ship a makefile; they "officially" only support luarocks for building.
You are upstream, you have the power to make a change for the better
I am not the upstream for the linked package (compat53), though I am for others you may be talking about.
The problem is that there is no nice cross-platform makefile structure suitable for lua libraries. Each operating system/distro calls the lua shared library something different, they all have their own set of required and conflicting flags, they all have differing install locations and search paths! I've got a back-burner project to try and homogenize things, I've documented the status quo here: https://docs.google.com/spreadsheets/d/1Z_oM8LgwyGAsfof6_FbbBJlY-wCA6DfadwZb...
On 11 Dec 2018, at 12:10 pm -0800, Daurnimator wrote:
On Tue, 11 Dec 2018 at 12:04, Robin Broda via aur-general aur-general@archlinux.org wrote:
On 12/11/18 9:00 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general aur-general@archlinux.org wrote:
- You have some AUR packages for LUA modules of your own making, yet
they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
The upstream packages do not ship a makefile; they "officially" only support luarocks for building.
You are upstream, you have the power to make a change for the better
The problem is that there is no nice cross-platform makefile structure suitable for lua libraries.
Each operating system/distro calls the lua shared library something different, they all have their own set of required and conflicting flags, they all have differing install locations and search paths!
Yikes!
One ugly but workable solution could be to conditionally set variables in a Makefile. For instance:
ifeq ($(OS),Windows_NT) # for Windows versions >= NT LUA := C:\Winders\Path\To\Lua.whatever else UNAME := $(shell uname -s) ifeq ($(UNAME),Linux) LUA := ... endif ... endif
Granted, this likely duplicates some of the magic luarocks is doing, but it would make things simpler for packaging.
Come to think of it, that seems like the sort of thing that could be done once as boilerplate in, e.g., lua.mk, which you and other luarines (lua-ites? luans? lua-ers?) could then include in future Makefiles.
Just spitballing here; lua packaging is weird (-: .
Cheers, Ivy ("escondida")
On Tue, 11 Dec 2018 at 12:47, Ivy Foster via aur-general aur-general@archlinux.org wrote:
Yikes!
One ugly but workable solution could be to conditionally set variables in a Makefile. For instance:
ifeq ($(OS),Windows_NT) # for Windows versions >= NT LUA := C:\Winders\Path\To\Lua.whatever else UNAME := $(shell uname -s) ifeq ($(UNAME),Linux) LUA := ... endif ... endifGranted, this likely duplicates some of the magic luarocks is doing, but it would make things simpler for packaging.
I've done similar before, but then the complaints from e.g. bsd make users start coming in. Not to mention that you'd be hardcoding defaults for all the different distros.... as the earlier linked spreadsheet shows: there are a lot of variations.
I think the better solution is to try and get different distros to use the same formats and try and unify it all with e.g. pkg-config. However that's a long and political process.
Come to think of it, that seems like the sort of thing that could be done once as boilerplate in, e.g., lua.mk, which you and other luarines (lua-ites? luans? lua-ers?) could then include in future Makefiles.
This has been proposed before, but I've never been quite happy in how it works out; nor has it caught on. See https://25thandclement.com/~william/projects/luapath.html for one example.
Just spitballing here; lua packaging is weird (-: .
Agreed! luarocks is the best we have, and it's not perfect!
On 12/11/18 3:51 PM, Daurnimator wrote:
I think the better solution is to try and get different distros to use the same formats and try and unify it all with e.g. pkg-config. However that's a long and political process.
lua *has* a pkg-config file already, it just exists as documentation in the Makefile apparently.
And pkg-config is not and has never been about agreeing on "formats". It's about agreeing on a query language to find out which format any given system is using. There is nothing requiring any distro to do anything other than build the pkg-config file according to their Make arguments and install the thing...
This is not hard, most software can do it properly without fuss. I fail to see why it would represent a political process for lua specifically.
Can you point me to a bug report or pull request or mailing list discussion or other form of discussion in the lua community where the topic of pkg-config has been previously discussed and rejected as too political? Because that's a startling claim.
On Tue, 11 Dec 2018 at 13:28, Eli Schwartz via aur-general aur-general@archlinux.org wrote:
Can you point me to a bug report or pull request or mailing list discussion or other form of discussion in the lua community where the topic of pkg-config has been previously discussed and rejected as too political? Because that's a startling claim.
- http://lua-users.org/lists/lua-l/2008-09/msg00186.html - http://lua-users.org/lists/lua-l/2010-03/msg00718.html - http://lua-users.org/lists/lua-l/2010-05/msg00616.html - http://lua-users.org/lists/lua-l/2011-12/msg00731.html - http://lua-users.org/lists/lua-l/2012-02/msg00814.html - http://lua-users.org/lists/lua-l/2015-03/msg00334.html
On 12/11/18 3:10 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 12:04, Robin Broda via aur-general aur-general@archlinux.org wrote:
On 12/11/18 9:00 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general aur-general@archlinux.org wrote:
- You have some AUR packages for LUA modules of your own making, yet
they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
The upstream packages do not ship a makefile; they "officially" only support luarocks for building.
You are upstream, you have the power to make a change for the better
I am not the upstream for the linked package (compat53), though I am for others you may be talking about.
The problem is that there is no nice cross-platform makefile structure suitable for lua libraries. Each operating system/distro calls the lua shared library something different, they all have their own set of required and conflicting flags, they all have differing install locations and search paths! I've got a back-burner project to try and homogenize things, I've documented the status quo here: https://docs.google.com/spreadsheets/d/1Z_oM8LgwyGAsfof6_FbbBJlY-wCA6DfadwZb...
Leaving aside the complete inadequacy of the official lua Makefile (among several issues, lua does not support DESTDIR), or the fact that the lua Makefile is not even available in their github repo (which is itself, seemingly a subdirectory of their official release tarball)...
Lua *does* provide a "make pc" target. Admittedly, this target is missing the Libs: keyword so that users can actually use the invocation `pkg-config --libs lua` to find out the right flags for linking lua. This would be helpful both to users wishing to statically link, and those wishing to dynlink. Admittedly also, this pc target echoes to stdout instead of like creating the actual file.
Note: the Arch, Debian, and Fedora maintainers have *all* fixed the pc file to provide proper Libs:, and while I wouldn't generally recommend relying on downstream pkg-config files, the official sources do quite plainly allow you to use in your Makefile:
CFLAGS += $(shell pkg-config --variable includedir lua)
since it is assumed lua distributors will run make pc > lua.pc and install this file themselves.
I don't see why this is so hard to do. Even if you somehow decided you didn't want to use pkg-config, you could use autoconf or a custom configure script to look for the locations in question and add them to a config.mk
Therefore, as the maintainer of the lua-psl project you should fix this, and submit pull requests for other packages.
On Tue, 11 Dec 2018 at 13:18, Eli Schwartz via aur-general aur-general@archlinux.org wrote:
Lua *does* provide a "make pc" target.
Only lua 5.1 does, it was removed in the lua 5.2 release. Due to disagreements between debian and fedora lua package maintainers about what it should contain.
Admittedly, this target is missing the Libs: keyword so that users can actually use the invocation `pkg-config --libs lua` to find out the right flags for linking lua. This would be helpful both to users wishing to statically link, and those wishing to dynlink. Admittedly also, this pc target echoes to stdout instead of like creating the actual file.
Note: the Arch, Debian, and Fedora maintainers have *all* fixed the pc file to provide proper Libs:, and while I wouldn't generally recommend relying on downstream pkg-config files, the official sources do quite plainly allow you to use in your Makefile:
CFLAGS += $(shell pkg-config --variable includedir lua)
since it is assumed lua distributors will run make pc > lua.pc and install this file themselves.
They do not.
The upstream lua makefile does not even support creating shared libraries; that is patched in (differently!) by each distro.
On 12/11/18 4:21 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 13:18, Eli Schwartz via aur-general aur-general@archlinux.org wrote:
Lua *does* provide a "make pc" target.
Only lua 5.1 does, it was removed in the lua 5.2 release. Due to disagreements between debian and fedora lua package maintainers about what it should contain.
I found it by downloading https://www.lua.org/ftp/lua-5.3.5.tar.gz and looking at the Makefile. Pretty sure this is not lua5.1, could be wrong though.
Do you mean they removed a pc rather than an echo target?
..
It seems bizarre that Debian and Fedora would disagree about what pkg-config should contain. There's an official spec about what it should contain. :)
Maybe link both the Debian and Fedora maintainers to the pc(5) manpage? They seem to have badly pendanticized themselves into irrelevance, which I guess is sad for the lua community. None of the mentioned debian concerns, at least, are remotely relevant to lua.
Once again, Debian is why we cannot have nice things I guess.
On 12/11/18 4:43 PM, Daurnimator wrote:
Ouch. It's a job for downstream packagers to provide the .pc in much the same way it's a job for downstream packagers to provide the Makefile.
The FHS specifies,
/usr/share : Architecture-independent data
It is thus only technically correct that it be stored in /usr/lib, and if they get this nigglingly insignificant detail wrong, then Debian, which does make use of multiarch packages, will be happy to perform a trivial downstream integration patch and likely also inform the lua developers of their typo.
If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++.
If lua only provides a static library named liblua.a, then this will conflict with any other version of lua, and thus, so will the pkg-config file. Distributions which work around this using `mv` and `ln` on the library can also `mv` and `ln` the pkg-config file.
tl;dr why is it the job of lua to provide pkg-config files for libraries that they don't provide?
More on static linking and pkg-config files below.
Huh? http://lua-users.org/lists/lua-l/2012-02/msg00748.html
If downstream lua applications wish to support versioned lua shared libraries, I would recommend using something like
LDLIBS += $(shell pkg-config --libs lua$(LUAVER))
and telling distros to compile with make LUAVER=5.2 to match their own distro versioning, if they don't want to use the one, official, canonical lua.pc that doesn't support dynamic linking at all.
If LUAVER is not defined, then no loss! It does as intended and compiles for the default version of lua, right?
That person is just deliberately trolling. :(
Admittedly, this target is missing the Libs: keyword so that users can actually use the invocation `pkg-config --libs lua` to find out the right flags for linking lua. This would be helpful both to users wishing to statically link, and those wishing to dynlink. Admittedly also, this pc target echoes to stdout instead of like creating the actual file.
Note: the Arch, Debian, and Fedora maintainers have *all* fixed the pc file to provide proper Libs:, and while I wouldn't generally recommend relying on downstream pkg-config files, the official sources do quite plainly allow you to use in your Makefile:
CFLAGS += $(shell pkg-config --variable includedir lua)
since it is assumed lua distributors will run make pc > lua.pc and install this file themselves.
They do not.
Then why do they have the target? For that matter, reading your links it appears that the lua developers are quite insistent that distributions *should* provide these pc files even if they write them themselves.
I'd guess the assumption by lua devs is therefore that "yes, they will either use: make pc > lua.pc, or they will write their own from scratch, either way assume it exists". Maybe they should document this on their installation guidelines to make it official.
The upstream lua makefile does not even support creating shared libraries; that is patched in (differently!) by each distro.
This is such a huge misunderstanding of what pkg-config is, and it's something I have zero opinion on. For the purposes of this discussion I don't care if lua uses shared libraries either officially or unofficially, and it has no bearing on anything I say.
pkg-config is not about shared libraries. pkg-config does not care about whether you use shared libraries. pkg-config is about telling you which compiler flags to use, and compiler flags don't care whether you use shared or static.
There are primarily four types of flags pkg-config provides:
Name: Version: Cflags: Libs:
1) is documented to contain a name useful for naming the thing
2) is documented to contain a version that can be used for checking to see if your system claims to support a given version of the named thing.
3) and 4) are documented to contain symbolic flags that are required by an application that expects to consume the project provided by the pkg-config file. It does not specify whether this project has any such flags, but if it does have such flags, they need to be here. The flags that the project needs are philosophically derived from whatever the project that provides the pkg-config file believes is right and just that should be used, and wish to unconditionally message to the entire world that they should use.
3) should contain any -I/usr/include/ path to the location of the headers, and any other compiler CFLAGS of note
4) should contain the -lfoo name of any compiler LDLIBS of note, paying heed to the fact that statically linked applications use -lfoo to link to libfoo.a just as surely as shared applications use -lfoo to link to libfoo.so, and your compiler will in fact always check for both.
In fact, GNU ld documents -lfoo as searching for libfoo.a, and adds as an afternote that "on systems which support shared libraries, ld may also search for files other than libnamespec.a" (and on ELF systems, it will first search for *.so)
You may also specify compiler LDLIBS like:
Libs: -l:libfoo.a
which is a pkg-config metadata specifier signaling a GNU ld flag that looks for the exact filename "libfoo.a" and thus does not give the option of auto-detecting shared libraries.
...
Finally, pkg-config explicitly supports static linking via
Libs.private: Cflags.private:
(The latter is ignored by freedesktop pkg-config, but used by pkgconf.)
The .private variant means, this is the additional flags you use when explicitly using `pkg-config --static --libs foo` to request the flags that are fine-tuned to building your application with a static version of the pkg-config symbolic system provider for "foo".
The pkg-config specification authors would like the lua developers -- and you -- to know that they are warmly welcoming to the needs of people who provide static libraries.
. On Tue, 11 Dec 2018 at 14:44, Eli Schwartz via aur-general aur-general@archlinux.org wrote:
Do you mean they removed a pc rather than an echo target?
Yes, sorry for confusion.
If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++.
Lua supports either compiling with longjmp or C++ based exceptions. See "Error-recovery functions" near https://www.lua.org/source/5.3/ldo.c.html#errorstatus
If lua only provides a static library named liblua.a, then this will conflict with any other version of lua, and thus, so will the pkg-config file. Distributions which work around this using `mv` and `ln` on the library can also `mv` and `ln` the pkg-config file.
Also to patch the pkg-config file to provide the relevant suffixes on e.g. cflags.
If downstream lua applications wish to support versioned lua shared libraries, I would recommend using something like
LDLIBS += $(shell pkg-config --libs lua$(LUAVER))
and telling distros to compile with make LUAVER=5.2 to match their own distro versioning, if they don't want to use the one, official, canonical lua.pc that doesn't support dynamic linking at all.
If LUAVER is not defined, then no loss! It does as intended and compiles for the default version of lua, right?
Admittedly, this target is missing the Libs: keyword so that users can actually use the invocation `pkg-config --libs lua` to find out the right flags for linking lua. This would be helpful both to users wishing to statically link, and those wishing to dynlink. Admittedly also, this pc target echoes to stdout instead of like creating the actual file.
Note: the Arch, Debian, and Fedora maintainers have *all* fixed the pc file to provide proper Libs:, and while I wouldn't generally recommend relying on downstream pkg-config files, the official sources do quite plainly allow you to use in your Makefile:
CFLAGS += $(shell pkg-config --variable includedir lua)
since it is assumed lua distributors will run make pc > lua.pc and install this file themselves.
They do not.
Then why do they have the target? For that matter, reading your links it appears that the lua developers are quite insistent that distributions *should* provide these pc files even if they write them themselves.
I'd guess the assumption by lua devs is therefore that "yes, they will either use: make pc > lua.pc, or they will write their own from scratch, either way assume it exists". Maybe they should document this on their installation guidelines to make it official.
The upstream lua makefile does not even support creating shared libraries; that is patched in (differently!) by each distro.
This is such a huge misunderstanding of what pkg-config is, and it's something I have zero opinion on. For the purposes of this discussion I don't care if lua uses shared libraries either officially or unofficially, and it has no bearing on anything I say.
pkg-config is not about shared libraries. pkg-config does not care about whether you use shared libraries. pkg-config is about telling you which compiler flags to use, and compiler flags don't care whether you use shared or static.
There are primarily four types of flags pkg-config provides:
Name: Version: Cflags: Libs:
is documented to contain a name useful for naming the thing
is documented to contain a version that can be used for checking to
see if your system claims to support a given version of the named thing.
- and 4) are documented to contain symbolic flags that are required by
an application that expects to consume the project provided by the pkg-config file. It does not specify whether this project has any such flags, but if it does have such flags, they need to be here. The flags that the project needs are philosophically derived from whatever the project that provides the pkg-config file believes is right and just that should be used, and wish to unconditionally message to the entire world that they should use.
- should contain any -I/usr/include/ path to the location of the
headers, and any other compiler CFLAGS of note
- should contain the -lfoo name of any compiler LDLIBS of note, paying
heed to the fact that statically linked applications use -lfoo to link to libfoo.a just as surely as shared applications use -lfoo to link to libfoo.so, and your compiler will in fact always check for both.
In fact, GNU ld documents -lfoo as searching for libfoo.a, and adds as an afternote that "on systems which support shared libraries, ld may also search for files other than libnamespec.a" (and on ELF systems, it will first search for *.so)
You may also specify compiler LDLIBS like:
Libs: -l:libfoo.a
which is a pkg-config metadata specifier signaling a GNU ld flag that looks for the exact filename "libfoo.a" and thus does not give the option of auto-detecting shared libraries.
...
Finally, pkg-config explicitly supports static linking via
Libs.private: Cflags.private:
(The latter is ignored by freedesktop pkg-config, but used by pkgconf.)
The .private variant means, this is the additional flags you use when explicitly using `pkg-config --static --libs foo` to request the flags that are fine-tuned to building your application with a static version of the pkg-config symbolic system provider for "foo".
The pkg-config specification authors would like the lua developers -- and you -- to know that they are warmly welcoming to the needs of people who provide static libraries.
So some extra context may be required here. Lua needs to be used by two different types of code: 1. code that embeds lua 2. lua libraries written in C
For most unix systems (this is *not* true on windows):
Things in category 1 include the `lua` executable itself, as well as applications like mame, powerdns, vlc, etc. If they link against lua statically, then they should expose lua C symbols via e.g. -Wl,--export-dynamic If they link against lua dynamically then they are fine.
Things in category 2 include libraries such as lua-filesystem. They expose an entrypoint of luaopen_mylibname, and use the lua C api. These should *not* link against liblua, and need to rely on symbols provided by the host application.
However note that both categories of code need to find the lua headers.
On 12/11/18 6:19 PM, Daurnimator wrote:
If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++.
Lua supports either compiling with longjmp or C++ based exceptions. See "Error-recovery functions" near https://www.lua.org/source/5.3/ldo.c.html#errorstatus
Okay, that's fine. So how does this work, then, and why does it need a separate pkg-config file? Is it just a matter of renaming liblua.a to liblua-c++.a to prevent filename clashes and thereby permit parallel installation? So... why is this any different from renaming the versioned libraries?
Once again: for anyone wishing to provide a different version of the lua library, interpreter, and/or pkg-config file should presumably modify the same names in all three as a set. This does not detract from the logic of supporting pkg-config via a pkg-config file meant to describe the project itself.
So some extra context may be required here. Lua needs to be used by two different types of code:
- code that embeds lua
- lua libraries written in C
For most unix systems (this is *not* true on windows):
Things in category 1 include the `lua` executable itself, as well as applications like mame, powerdns, vlc, etc. If they link against lua statically, then they should expose lua C symbols via e.g. -Wl,--export-dynamic If they link against lua dynamically then they are fine.
I don't get how that argues against pkg-config...
That sounds like the job of the application to use, that they know they absolutely need to expose all symbols just in case and don't want to mess with linker scripts for upstream symbols.
What's the cost of doing that in your embedded application that links against liblua and finds a system-provided dynamic lib? The failure to optimize your symbol table, when you didn't expect to be able to dynlink at all?
But let's imagine instead, lua devs who support static only, and wish to support pkg-config discovery.
This is lua.pc, no versions
Name: Lua Version: 5.3.5 Description: An Extensible Extension Language Cflags: -I${includedir} Libs: -L${libdir} -Wl,--export-dynamic -llua -lm
Applications which desire to embed lua can link using:
CFLAGS += $(shell pkg-config --cflags lua) LDLIBS += $(shell pkg-config --libs lua)
Applications which wish to build modules for lua can skip the LDLIBS since they don't link to lua, and coincidentally also don't need to --export-dynamic.
Downstream distros which patch lua to build shared libraries are also responsible to optimize their ecosystem symbol tables by patching out -Wl,--export-dynamic
Downstream distros that provide both dynamic and static libs can instead move it to Libs.private, and pkg-config will only emit it when using:
pkg-config --static --libs lua
...
In all cases, the pkg-config file by design should work as-is, with the assumption that it describes the project which is compiled using make && sudo make install. It's a madness game to even pretend to also support the modifications which people make to the build system.
Things in category 2 include libraries such as lua-filesystem. They expose an entrypoint of luaopen_mylibname, and use the lua C api. These should *not* link against liblua, and need to rely on symbols provided by the host application.
However note that both categories of code need to find the lua headers.
That's fine, it makes it even more convenient to reserve --libs for the exclusive use of -Wl,--export-dynamic
On Tue, 11 Dec 2018 at 18:50, Eli Schwartz eschwartz@archlinux.org wrote:
On 12/11/18 6:19 PM, Daurnimator wrote:
If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++.
Lua supports either compiling with longjmp or C++ based exceptions. See "Error-recovery functions" near https://www.lua.org/source/5.3/ldo.c.html#errorstatus
Okay, that's fine. So how does this work, then, and why does it need a separate pkg-config file? Is it just a matter of renaming liblua.a to liblua-c++.a to prevent filename clashes and thereby permit parallel installation? So... why is this any different from renaming the versioned libraries?
It is not essentially a different ABI? liblua.a uses longjmp for error handling, libluac++.a.so uses C++ exceptions. If you want to specifically link against e.g. the C++ one, what should a lua-using-project use?
Downstream distros which patch lua to build shared libraries are also responsible to optimize their ecosystem symbol tables by patching out -Wl,--export-dynamic
This is where upstream turns around and says "if everyone is patching it, why should we ship it at all?"
In all cases, the pkg-config file by design should work as-is, with the assumption that it describes the project which is compiled using make && sudo make install. It's a madness game to even pretend to also support the modifications which people make to the build system.
upstream lua has historically shipped with a "please create your own build system". For many years the recommended compilation instructions were "give the .c files to your compiler of choice and hit build" Such instructions are relatively timeless e.g. https://youtu.be/-jvLY5pUwic
Now I think we here agree that this isn't the best way forward, but the upstream audience is diverse and can take time to convince.
Things in category 2 include libraries such as lua-filesystem. They expose an entrypoint of luaopen_mylibname, and use the lua C api. These should *not* link against liblua, and need to rely on symbols provided by the host application.
However note that both categories of code need to find the lua headers.
That's fine, it makes it even more convenient to reserve --libs for the exclusive use of -Wl,--export-dynamic
I just know that someone would reply to me with something like the following if I proposed this, so I may as well see what your response is:
package config's --libs output is meant for use with LDLIBS. Flags for the linker should *not* be passed in LDLIBS and should only be passed in LDFLAGS
On 12/16/18 11:43 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 18:50, Eli Schwartz eschwartz@archlinux.org wrote:
On 12/11/18 6:19 PM, Daurnimator wrote:
If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++.
Lua supports either compiling with longjmp or C++ based exceptions. See "Error-recovery functions" near https://www.lua.org/source/5.3/ldo.c.html#errorstatus
Okay, that's fine. So how does this work, then, and why does it need a separate pkg-config file? Is it just a matter of renaming liblua.a to liblua-c++.a to prevent filename clashes and thereby permit parallel installation? So... why is this any different from renaming the versioned libraries?
It is not essentially a different ABI? liblua.a uses longjmp for error handling, libluac++.a.so uses C++ exceptions. If you want to specifically link against e.g. the C++ one, what should a lua-using-project use?
Downstream distros which patch lua to build shared libraries are also responsible to optimize their ecosystem symbol tables by patching out -Wl,--export-dynamic
This is where upstream turns around and says "if everyone is patching it, why should we ship it at all?"
If everyone is patching it, why should lua ship a *Makefile*?
But I didn't say anyone should patch it. Micro-optimizations for things that work either way aren't really relevant. What is relevant is that:
- people building their own unpatched liblua.a can use it without modifications - people who build distro-patched lua can *also* use it without modifications
- people who build several historically versioned copies of their distro-patched lua have a canonical file with rigidly defined linker flags, which *also* can be used without modifications due to, if I read the Makefile correctly, defining LUA_A with a versioned name. This is already needing to be defined at install time due to the installation target directory.
- people who want to both build a distro-patched shared library for lua, *and* micro-optimize their symbol table that works fine either way, can do whatever they want.
...
You're right though, having a configuration system is way too controversial if I cannot even convince you.
In all cases, the pkg-config file by design should work as-is, with the assumption that it describes the project which is compiled using make && sudo make install. It's a madness game to even pretend to also support the modifications which people make to the build system.
upstream lua has historically shipped with a "please create your own build system". For many years the recommended compilation instructions were "give the .c files to your compiler of choice and hit build" Such instructions are relatively timeless e.g. https://youtu.be/-jvLY5pUwic
Now I think we here agree that this isn't the best way forward, but the upstream audience is diverse and can take time to convince.
Things in category 2 include libraries such as lua-filesystem. They expose an entrypoint of luaopen_mylibname, and use the lua C api. These should *not* link against liblua, and need to rely on symbols provided by the host application.
However note that both categories of code need to find the lua headers.
That's fine, it makes it even more convenient to reserve --libs for the exclusive use of -Wl,--export-dynamic
I just know that someone would reply to me with something like the following if I proposed this, so I may as well see what your response is:
package config's --libs output is meant for use with LDLIBS. Flags for the linker should *not* be passed in LDLIBS and should only be passed in LDFLAGS
My response would be:
If you take a closer look at the manual page for pkg-config, it actually says the exact opposite, defining "--libs" as "linker flags". In fact, there are even three dedicated *subvariants* to get, depending on your highly customized needs,
--libs-only-L --libs-only-l --libs-only-other
This is clearly a case of libs-only-other.
On Mon, 17 Dec 2018 at 16:40, Eli Schwartz eschwartz@archlinux.org wrote:
If you take a closer look at the manual page for pkg-config, it actually says the exact opposite, defining "--libs" as "linker flags". In fact, there are even three dedicated *subvariants* to get, depending on your highly customized needs,
--libs-only-L --libs-only-l --libs-only-other
This is clearly a case of libs-only-other.
Ah ha! I didn't know about these. --libs-only-other seems to be missing from https://linux.die.net/man/1/pkg-config However according to https://bz.apache.org/bugzilla/show_bug.cgi?id=46168 they're available since 2003
------------------------------
Does this sound like a reasonable proposal then?: The .pc should contain:
``` V=5.3 R=${V}.5
prefix=/usr INSTALL_BIN=${prefix}/bin INSTALL_INC=${prefix}/include INSTALL_LIB=${prefix}/lib INSTALL_MAN=${prefix}/man/man1 INSTALL_LMOD=${prefix}/share/lua/${V} INSTALL_CMOD=${prefix}/lib/lua/${V} LIB=lua exec_prefix=${prefix} libdir=${exec_prefix}/lib includedir=${prefix}/include
Name: Lua Description: An Extensible Extension Language Version: ${R} Requires: Cflags: -I${includedir} Libs: -L${libdir} -l${LIB} Libs.private: -lm -Wl,-E ```
If you're compiling a application that dynamically embeds lua then you'd use: CFLAGS="$(pkgconf lua --cflags)" LDFLAGS="$(pkgconf lua --libs)"
If you're compiling a application that statically embeds lua then you'd use: CFLAGS="$(pkgconf lua --cflags --static)" LDFLAGS="$(pkgconf lua --libs --static)"
If you're compiling a lua library dynamically then your makefile would use CFLAGS+="$(shell pkgconf lua --cflags)" Your .so file should be installed to $(DESTDIR)/$(shell pkgconf lua --variable=INSTALL_CMOD)
If you're compiling a lua library statically then your makefile would use CFLAGS+="$(shell pkgconf lua --cflags --static)" Your .so file should be installed to $(DESTDIR)/$(libdir)
With no use of LDFLAGS for the last two cases.
Now to come up with a migration story: - existing distributions do not currently supply -Wl,--export-dynamic in their pkg-config files - existing projects do not know to use pkg-config with lua in this manner.
If upstream started shipping an adjusted pkg-config file, best case it would only be available for the 5.4 release.
On 12/18/18 3:28 AM, Daurnimator wrote:
--libs-only-other seems to be missing from https://linux.die.net/man/1/pkg-config
linux.die.net has been extremely out of date for ages, fwiw
On 12/11/18 3:00 PM, Daurnimator wrote:
On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general aur-general@archlinux.org wrote:
- When I look at LUA modules, I see that most are available on
"luarocks", which is apparently a package manager for LUA. Can you leverage this to make more LUA modules available on Arch?
Note that it's "Lua" not LUA.
I'd like to leverage luarocks to build lua packages, however it maintains its own local database of installed packages which makes things awkward. I don't want to introduce luarocks as a non-build-time dependency. Follow https://github.com/luarocks/luarocks/issues/671 for possible progress in this area.
So I looked at this link of yours and all I see is the luarocks maintainer asking in the very first response, why not just use luarocks to build the package and just discard the database.
I'm not even seeing why such a database database is awkward though. I can think of two analogues:
python setuptools packages install egg-info metadata which is hardly required to execute the python code, but does allow setuptools itself to build a runtime index of available modules
haskell's ghc-pkg stores the same sort of metadata in /usr/lib/ghc-8.6.3/package.conf.d/ but additionally has a binary cache of this database, that is updated with the contents of non-core packages by a pacman PostTransaction hook.
What harm or "awkwardness" is caused by lua packages installing optional metadata that doesn't require luarocks to be installed? Are you saying that merely building a lua package with luarocks will cause the package to import the luarocks module during initialization and error out if it isn't installed?
- You have some AUR packages for LUA modules of your own making, yet
they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
The upstream packages do not ship a makefile; they "officially" only support luarocks for building. Due to the above-mentioned issue, I don't see it as feasible to use luarocks for AUR packages at the moment, so I had to hard-code the build commands into the PKGBUILD.
- I have no idea on what some of your more complicated packages do, or
why they would require said complexity, e.g. iup. [2] Perhaps you could explain a bit on that regard.
IUP is a graphical toolkit similar to gtk, qt or wxwidgets with many components. The upstream do not want to support dynamic linking for some pieces, so I need to patch out their (custom) build system 'tecmake'.
The release tarballs include lua bindings to the C IUP library; some complexity in the PKGBUILD comes from compiling necessary components for each of lua 5.1, 5.2 and 5.3.
- Related to the above, there are no current packaging guidelines for
LUA packages. [3] Do you plan on starting an effort (possibly with other LUA package maintainers) to remedy this?
Yes :) If anyone would like to help out with this I'm all ears.
As someone who has never touched lua in my life (other than googling for their Makefile to try to figure out the public pkg-config interface it provides), I think the first step is to try to work with lua upstream, because if this discussion is a sign of what packaging guidelines would look like, then you'd need to first convince me why it is right and just, and only then tell me how to do it.
I'm now far more confused about how to package lua than I was before this thread -- and I used to not know anything!
On 12/11/18 8:45 PM, Alad Wenter via aur-general wrote:
On 12/11/18 8:30 PM, Alad Wenter wrote:
Since the discussion period is about to end without much discussion...
Right now the rate of new applications is very high - about 2 new applications per month. That makes a thorough review difficult.
Considering the positive experiences of the sponsor, it would be a shame to let a voting period pass. That said, I'm not sure we have sufficient information - at present - to proceed with such a voting period in a meaningful manner.
I agree
So about Foxboron's question for confirmation: "Say one or two people confirm they think the voting process should be continued after the discussion has ended?" - I don't know.
Let's try to get the ball rolling by asking some questions.
.. snip ..
- I have no idea on what some of your more complicated packages do, or
why they would require said complexity, e.g. iup. [2] Perhaps you could explain a bit on that regard.
Taking a closer look at iup, what's with the `sed`ding in prepare()? - those should be patches, as sed will silently fail when they stop applying - why do you explicitly link iupview statically?
Also, the url is reachable via https - you should maybe update that.
wt., 11.12.2018, 20:45: Alad Wenter via aur-general < aur-general@archlinux.org> napisał(a):
On 12/11/18 8:30 PM, Alad Wenter wrote:
Since the discussion period is about to end without much discussion...
Right now the rate of new applications is very high - about 2 new applications per month. That makes a thorough review difficult.
Considering the positive experiences of the sponsor, it would be a shame to let a voting period pass. That said, I'm not sure we have sufficient information - at present - to proceed with such a voting period in a meaningful manner.
So about Foxboron's question for confirmation: "Say one or two people confirm they think the voting process should be continued after the discussion has ended?" - I don't know.
Alad
Let's try to get the ball rolling by asking some questions.
- When I look at LUA modules, I see that most are available on
"luarocks", which is apparently a package manager for LUA. Can you leverage this to make more LUA modules available on Arch?
- You have some AUR packages for LUA modules of your own making, yet
they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?
- I have no idea on what some of your more complicated packages do, or
why they would require said complexity, e.g. iup. [2] Perhaps you could explain a bit on that regard.
- Related to the above, there are no current packaging guidelines for
LUA packages. [3] Do you plan on starting an effort (possibly with other LUA package maintainers) to remedy this?
[1] https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=lua-compat53 [2] https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=iup [3]
https://wiki.archlinux.org/index.php/Arch_package_guidelines#Additional_guid...
Alad
participants (6)
-
Alad Wenter
-
Daurnimator
-
Eli Schwartz
-
Ivy Foster
-
Robin Broda
-
Sebastian Pokora