On 12/11/18 9:00 PM, Daurnimator wrote:

On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general <aur-general@archlinux.org> wrote:

...

2. You have some AUR packages for LUA modules of your own making, yet they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?

The upstream packages do not ship a makefile; they "officially" only support luarocks for building.

You are upstream, you have the power to make a change for the better -- Rob (coderobe) O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

On 11 Dec 2018, at 12:10 pm -0800, Daurnimator wrote:

On Tue, 11 Dec 2018 at 12:04, Robin Broda via aur-general <aur-general@archlinux.org> wrote:

...

On 12/11/18 9:00 PM, Daurnimator wrote:

...

On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general <aur-general@archlinux.org> wrote:

...

2. You have some AUR packages for LUA modules of your own making, yet they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?

...

...

The upstream packages do not ship a makefile; they "officially" only support luarocks for building.

...

You are upstream, you have the power to make a change for the better

The problem is that there is no nice cross-platform makefile structure suitable for lua libraries.

Each operating system/distro calls the lua shared library something different, they all have their own set of required and conflicting flags, they all have differing install locations and search paths!

Yikes! One ugly but workable solution could be to conditionally set variables in a Makefile. For instance: ifeq ($(OS),Windows_NT) # for Windows versions >= NT LUA := C:\Winders\Path\To\Lua.whatever else UNAME := $(shell uname -s) ifeq ($(UNAME),Linux) LUA := ... endif ... endif Granted, this likely duplicates some of the magic luarocks is doing, but it would make things simpler for packaging. Come to think of it, that seems like the sort of thing that could be done once as boilerplate in, e.g., lua.mk, which you and other luarines (lua-ites? luans? lua-ers?) could then include in future Makefiles. Just spitballing here; lua packaging is weird (-: . Cheers, Ivy ("escondida")

On 12/11/18 3:51 PM, Daurnimator wrote:

I think the better solution is to try and get different distros to use the same formats and try and unify it all with e.g. pkg-config. However that's a long and political process.

lua *has* a pkg-config file already, it just exists as documentation in the Makefile apparently. And pkg-config is not and has never been about agreeing on "formats". It's about agreeing on a query language to find out which format any given system is using. There is nothing requiring any distro to do anything other than build the pkg-config file according to their Make arguments and install the thing... This is not hard, most software can do it properly without fuss. I fail to see why it would represent a political process for lua specifically. Can you point me to a bug report or pull request or mailing list discussion or other form of discussion in the lua community where the topic of pkg-config has been previously discussed and rejected as too political? Because that's a startling claim. -- Eli Schwartz Bug Wrangler and Trusted User

On 12/11/18 3:10 PM, Daurnimator wrote:

On Tue, 11 Dec 2018 at 12:04, Robin Broda via aur-general <aur-general@archlinux.org> wrote:

...

On 12/11/18 9:00 PM, Daurnimator wrote:

...

On Tue, 11 Dec 2018 at 11:45, Alad Wenter via aur-general <aur-general@archlinux.org> wrote:

...

2. You have some AUR packages for LUA modules of your own making, yet they hardcode gcc lines instead of using a Makefile. [1] (At least they respect $CFLAGS and $LDFLAGS, I guess.) Why?

The upstream packages do not ship a makefile; they "officially" only support luarocks for building.

You are upstream, you have the power to make a change for the better

I am not the upstream for the linked package (compat53), though I am for others you may be talking about.

The problem is that there is no nice cross-platform makefile structure suitable for lua libraries. Each operating system/distro calls the lua shared library something different, they all have their own set of required and conflicting flags, they all have differing install locations and search paths! I've got a back-burner project to try and homogenize things, I've documented the status quo here: https://docs.google.com/spreadsheets/d/1Z_oM8LgwyGAsfof6_FbbBJlY-wCA6DfadwZb...

Leaving aside the complete inadequacy of the official lua Makefile (among several issues, lua does not support DESTDIR), or the fact that the lua Makefile is not even available in their github repo (which is itself, seemingly a subdirectory of their official release tarball)... Lua *does* provide a "make pc" target. Admittedly, this target is missing the Libs: keyword so that users can actually use the invocation `pkg-config --libs lua` to find out the right flags for linking lua. This would be helpful both to users wishing to statically link, and those wishing to dynlink. Admittedly also, this pc target echoes to stdout instead of like creating the actual file. Note: the Arch, Debian, and Fedora maintainers have *all* fixed the pc file to provide proper Libs:, and while I wouldn't generally recommend relying on downstream pkg-config files, the official sources do quite plainly allow you to use in your Makefile: CFLAGS += $(shell pkg-config --variable includedir lua) since it is assumed lua distributors will run make pc > lua.pc and install this file themselves. I don't see why this is so hard to do. Even if you somehow decided you didn't want to use pkg-config, you could use autoconf or a custom configure script to look for the locations in question and add them to a config.mk Therefore, as the maintainer of the lua-psl project you should fix this, and submit pull requests for other packages. -- Eli Schwartz Bug Wrangler and Trusted User

On 12/11/18 4:21 PM, Daurnimator wrote:

On Tue, 11 Dec 2018 at 13:18, Eli Schwartz via aur-general <aur-general@archlinux.org> wrote:

...

Lua *does* provide a "make pc" target.

Only lua 5.1 does, it was removed in the lua 5.2 release. Due to disagreements between debian and fedora lua package maintainers about what it should contain.

I found it by downloading https://www.lua.org/ftp/lua-5.3.5.tar.gz and looking at the Makefile. Pretty sure this is not lua5.1, could be wrong though. Do you mean they removed a pc rather than an echo target? .. It seems bizarre that Debian and Fedora would disagree about what pkg-config should contain. There's an official spec about what it should contain. :) Maybe link both the Debian and Fedora maintainers to the pc(5) manpage? They seem to have badly pendanticized themselves into irrelevance, which I guess is sad for the lua community. None of the mentioned debian concerns, at least, are remotely relevant to lua. Once again, Debian is why we cannot have nice things I guess. On 12/11/18 4:43 PM, Daurnimator wrote:

- http://lua-users.org/lists/lua-l/2008-09/msg00186.html

Ouch. It's a job for downstream packagers to provide the .pc in much the same way it's a job for downstream packagers to provide the Makefile.

- http://lua-users.org/lists/lua-l/2010-03/msg00718.html - http://lua-users.org/lists/lua-l/2010-05/msg00616.html

The FHS specifies, /usr/share : Architecture-independent data It is thus only technically correct that it be stored in /usr/lib, and if they get this nigglingly insignificant detail wrong, then Debian, which does make use of multiarch packages, will be happy to perform a trivial downstream integration patch and likely also inform the lua developers of their typo.

- http://lua-users.org/lists/lua-l/2011-12/msg00731.html

If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++. If lua only provides a static library named liblua.a, then this will conflict with any other version of lua, and thus, so will the pkg-config file. Distributions which work around this using `mv` and `ln` on the library can also `mv` and `ln` the pkg-config file. tl;dr why is it the job of lua to provide pkg-config files for libraries that they don't provide? More on static linking and pkg-config files below.

- http://lua-users.org/lists/lua-l/2012-02/msg00814.html

Huh? http://lua-users.org/lists/lua-l/2012-02/msg00748.html If downstream lua applications wish to support versioned lua shared libraries, I would recommend using something like LDLIBS += $(shell pkg-config --libs lua$(LUAVER)) and telling distros to compile with make LUAVER=5.2 to match their own distro versioning, if they don't want to use the one, official, canonical lua.pc that doesn't support dynamic linking at all. If LUAVER is not defined, then no loss! It does as intended and compiles for the default version of lua, right?

- http://lua-users.org/lists/lua-l/2015-03/msg00334.html

That person is just deliberately trolling. :(

...

Admittedly, this target is missing the Libs: keyword so that users can actually use the invocation `pkg-config --libs lua` to find out the right flags for linking lua. This would be helpful both to users wishing to statically link, and those wishing to dynlink. Admittedly also, this pc target echoes to stdout instead of like creating the actual file.

Note: the Arch, Debian, and Fedora maintainers have *all* fixed the pc file to provide proper Libs:, and while I wouldn't generally recommend relying on downstream pkg-config files, the official sources do quite plainly allow you to use in your Makefile:

CFLAGS += $(shell pkg-config --variable includedir lua)

since it is assumed lua distributors will run make pc > lua.pc and install this file themselves.

They do not.

Then why do they have the target? For that matter, reading your links it appears that the lua developers are quite insistent that distributions *should* provide these pc files even if they write them themselves. I'd guess the assumption by lua devs is therefore that "yes, they will either use: make pc > lua.pc, or they will write their own from scratch, either way assume it exists". Maybe they should document this on their installation guidelines to make it official.

The upstream lua makefile does not even support creating shared libraries; that is patched in (differently!) by each distro.

This is such a huge misunderstanding of what pkg-config is, and it's something I have zero opinion on. For the purposes of this discussion I don't care if lua uses shared libraries either officially or unofficially, and it has no bearing on anything I say. pkg-config is not about shared libraries. pkg-config does not care about whether you use shared libraries. pkg-config is about telling you which compiler flags to use, and compiler flags don't care whether you use shared or static. There are primarily four types of flags pkg-config provides: Name: Version: Cflags: Libs: 1) is documented to contain a name useful for naming the thing 2) is documented to contain a version that can be used for checking to see if your system claims to support a given version of the named thing. 3) and 4) are documented to contain symbolic flags that are required by an application that expects to consume the project provided by the pkg-config file. It does not specify whether this project has any such flags, but if it does have such flags, they need to be here. The flags that the project needs are philosophically derived from whatever the project that provides the pkg-config file believes is right and just that should be used, and wish to unconditionally message to the entire world that they should use. 3) should contain any -I/usr/include/ path to the location of the headers, and any other compiler CFLAGS of note 4) should contain the -lfoo name of any compiler LDLIBS of note, paying heed to the fact that statically linked applications use -lfoo to link to libfoo.a just as surely as shared applications use -lfoo to link to libfoo.so, and your compiler will in fact always check for both. In fact, GNU ld documents -lfoo as searching for libfoo.a, and adds as an afternote that "on systems which support shared libraries, ld may also search for files other than libnamespec.a" (and on ELF systems, it will first search for *.so) You may also specify compiler LDLIBS like: Libs: -l:libfoo.a which is a pkg-config metadata specifier signaling a GNU ld flag that looks for the exact filename "libfoo.a" and thus does not give the option of auto-detecting shared libraries. ... Finally, pkg-config explicitly supports static linking via Libs.private: Cflags.private: (The latter is ignored by freedesktop pkg-config, but used by pkgconf.) The .private variant means, this is the additional flags you use when explicitly using `pkg-config --static --libs foo` to request the flags that are fine-tuned to building your application with a static version of the pkg-config symbolic system provider for "foo". The pkg-config specification authors would like the lua developers -- and you -- to know that they are warmly welcoming to the needs of people who provide static libraries. -- Eli Schwartz Bug Wrangler and Trusted User

On 12/11/18 6:19 PM, Daurnimator wrote:

...

If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++.

Lua supports either compiling with longjmp or C++ based exceptions. See "Error-recovery functions" near https://www.lua.org/source/5.3/ldo.c.html#errorstatus

Okay, that's fine. So how does this work, then, and why does it need a separate pkg-config file? Is it just a matter of renaming liblua.a to liblua-c++.a to prevent filename clashes and thereby permit parallel installation? So... why is this any different from renaming the versioned libraries? Once again: for anyone wishing to provide a different version of the lua library, interpreter, and/or pkg-config file should presumably modify the same names in all three as a set. This does not detract from the logic of supporting pkg-config via a pkg-config file meant to describe the project itself.

So some extra context may be required here. Lua needs to be used by two different types of code: 1. code that embeds lua 2. lua libraries written in C

For most unix systems (this is *not* true on windows):

Things in category 1 include the `lua` executable itself, as well as applications like mame, powerdns, vlc, etc. If they link against lua statically, then they should expose lua C symbols via e.g. -Wl,--export-dynamic If they link against lua dynamically then they are fine.

I don't get how that argues against pkg-config... That sounds like the job of the application to use, that they know they absolutely need to expose all symbols just in case and don't want to mess with linker scripts for upstream symbols. What's the cost of doing that in your embedded application that links against liblua and finds a system-provided dynamic lib? The failure to optimize your symbol table, when you didn't expect to be able to dynlink at all? But let's imagine instead, lua devs who support static only, and wish to support pkg-config discovery. This is lua.pc, no versions Name: Lua Version: 5.3.5 Description: An Extensible Extension Language Cflags: -I${includedir} Libs: -L${libdir} -Wl,--export-dynamic -llua -lm Applications which desire to embed lua can link using: CFLAGS += $(shell pkg-config --cflags lua) LDLIBS += $(shell pkg-config --libs lua) Applications which wish to build modules for lua can skip the LDLIBS since they don't link to lua, and coincidentally also don't need to --export-dynamic. Downstream distros which patch lua to build shared libraries are also responsible to optimize their ecosystem symbol tables by patching out -Wl,--export-dynamic Downstream distros that provide both dynamic and static libs can instead move it to Libs.private, and pkg-config will only emit it when using: pkg-config --static --libs lua ... In all cases, the pkg-config file by design should work as-is, with the assumption that it describes the project which is compiled using make && sudo make install. It's a madness game to even pretend to also support the modifications which people make to the build system.

Things in category 2 include libraries such as lua-filesystem. They expose an entrypoint of luaopen_mylibname, and use the lua C api. These should *not* link against liblua, and need to rely on symbols provided by the host application.

However note that both categories of code need to find the lua headers.

That's fine, it makes it even more convenient to reserve --libs for the exclusive use of -Wl,--export-dynamic -- Eli Schwartz Bug Wrangler and Trusted User

On 12/16/18 11:43 PM, Daurnimator wrote:

On Tue, 11 Dec 2018 at 18:50, Eli Schwartz <eschwartz@archlinux.org> wrote:

...

On 12/11/18 6:19 PM, Daurnimator wrote:

...

...

If lua does not officially compile a C++ version, it is the job of Debian to both provide their own pkg-config files, and modify lua to build using C++.

Lua supports either compiling with longjmp or C++ based exceptions. See "Error-recovery functions" near https://www.lua.org/source/5.3/ldo.c.html#errorstatus

Okay, that's fine. So how does this work, then, and why does it need a separate pkg-config file? Is it just a matter of renaming liblua.a to liblua-c++.a to prevent filename clashes and thereby permit parallel installation? So... why is this any different from renaming the versioned libraries?

It is not essentially a different ABI? liblua.a uses longjmp for error handling, libluac++.a.so uses C++ exceptions. If you want to specifically link against e.g. the C++ one, what should a lua-using-project use?

...

Downstream distros which patch lua to build shared libraries are also responsible to optimize their ecosystem symbol tables by patching out -Wl,--export-dynamic

This is where upstream turns around and says "if everyone is patching it, why should we ship it at all?"

If everyone is patching it, why should lua ship a *Makefile*? But I didn't say anyone should patch it. Micro-optimizations for things that work either way aren't really relevant. What is relevant is that: - people building their own unpatched liblua.a can use it without modifications - people who build distro-patched lua can *also* use it without modifications - people who build several historically versioned copies of their distro-patched lua have a canonical file with rigidly defined linker flags, which *also* can be used without modifications due to, if I read the Makefile correctly, defining LUA_A with a versioned name. This is already needing to be defined at install time due to the installation target directory. - people who want to both build a distro-patched shared library for lua, *and* micro-optimize their symbol table that works fine either way, can do whatever they want. ... You're right though, having a configuration system is way too controversial if I cannot even convince you.

...

In all cases, the pkg-config file by design should work as-is, with the assumption that it describes the project which is compiled using make && sudo make install. It's a madness game to even pretend to also support the modifications which people make to the build system.

upstream lua has historically shipped with a "please create your own build system". For many years the recommended compilation instructions were "give the .c files to your compiler of choice and hit build" Such instructions are relatively timeless e.g. https://youtu.be/-jvLY5pUwic

Now I think we here agree that this isn't the best way forward, but the upstream audience is diverse and can take time to convince.

...

...

Things in category 2 include libraries such as lua-filesystem. They expose an entrypoint of luaopen_mylibname, and use the lua C api. These should *not* link against liblua, and need to rely on symbols provided by the host application.

However note that both categories of code need to find the lua headers.

That's fine, it makes it even more convenient to reserve --libs for the exclusive use of -Wl,--export-dynamic

I just know that someone would reply to me with something like the following if I proposed this, so I may as well see what your response is:

package config's --libs output is meant for use with LDLIBS. Flags for the linker should *not* be passed in LDLIBS and should only be passed in LDFLAGS

My response would be: If you take a closer look at the manual page for pkg-config, it actually says the exact opposite, defining "--libs" as "linker flags". In fact, there are even three dedicated *subvariants* to get, depending on your highly customized needs, --libs-only-L --libs-only-l --libs-only-other This is clearly a case of libs-only-other. -- Eli Schwartz Bug Wrangler and Trusted User

On 12/18/18 3:28 AM, Daurnimator wrote:

--libs-only-other seems to be missing from https://linux.die.net/man/1/pkg-config

linux.die.net has been extremely out of date for ages, fwiw -- Rob (coderobe) O< ascii ribbon campaign - stop html mail - www.asciiribbon.org