[arch-general] The OpenCL ICD problem

Ionut Biru

7 Jul 2011 7 Jul '11

1:57 p.m.

Forwarding discussion to arch-general mailing list. -------- Original Message -------- Subject: The OpenCL ICD problem Date: Thu, 7 Jul 2011 15:48:52 +0200 From: Vojtěch Král Hello, I'm writing to you because of a problem which has arisen with the OpenCL Arch packages. To introduce myself shortly: My name is Vojtech "kralyk" Kral (Czech Rep., EU) and I currently maintain AMD APP SDK (aka amdstream) and Intel OpenCL SDK in AUR. I'm sending this to maintainers who maintain involved pkgs. People that might be also interested are in Cc, please let me know should you find this mail too spammy/annoying :) The problem in question is a conflicting file, namely /usr/lib/libOpenCL.so (and possibly so version symlinks too). This file is currently provided by these packages (that I know of): nvidia-utils [extra], lib32-nvidia-utils [multilib], amdstream [AUR], intel-opencl-sdk [AUR], lib32-nvidia-utils-173xx [AUR] and lib32-nvidia-utils-beta [AUR]. The thing is this conflict should be resolved so that it would be possible to install OpenCL implementations from multiple vendors at the same time on the same system (as requested by the standard from Khronos) while still providing this so file to satistfy dependencies for other either existing pkgs (such as pyopencl) or possible funture ones. This should be done in compliance with the ICD standard, see http://www.khronos.org/registry/cl/extensions/khr/cl_khr_icd.txt However, it should hopefully not be that difficult, since most of the packages already comply to the standard by providing appropriate *.icd file in /etc/OpenCL/vendors. And those that don't should be updated accordingly (lib32-nvidia-utils-beta maybe? Not sure.) Also, the standard requests the ICD loader - which is in fact usually the libOpenCL.so library - to be able to enumerate all the ICDs on its own. So this so file is pretty much independent on the vendor it is supplied by, providing the vendor a) follows the standard correctly b) is sane :-) Therefore, a sollution might be to simply pick one of available ICD loaders (a thin one preferably) and make it a dependency for other pkgs. I'm not saying this is an optimal sollution, I'm merely leaving it to your consideration. Feel free to suggest other possibilities. Let me know what you think... Best Regards, Vojtech "kralyk" Kral

Show replies by date

Florian Pritz

7 Jul 7 Jul

2:29 p.m.

...

I'm sending this to maintainers who maintain involved pkgs. People that might be also interested are in Cc, please let me know should you find this mail too spammy/annoying :)

Next time please send this to the appropriate mailing list (arch-general or aur-general in most cases) and if you feel it's necessary you can still CC the maintainers.

...

The problem in question is a conflicting file, namely /usr/lib/libOpenCL.so (and possibly so version symlinks too). This file is currently provided by these packages (that I know of): nvidia-utils [extra], lib32-nvidia-utils [multilib],

multilib/lib32-nvidia-utils (275.09.07-1) : /usr/lib32/libOpenCL.so extra/nvidia-utils (275.09.07-1) : /usr/lib/libOpenCL.so There is no conflict.

...

Therefore, a sollution might be to simply pick one of available ICD loaders (a thin one preferably) and make it a dependency for other pkgs.

We only have one in our repos so there's nothing to do as far as I can tell. In case I misunderstood something, please clarify. -- Florian Pritz

Damjan

2:46 p.m.

...

There is no conflict.

...
Therefore, a sollution might be to simply pick one of available ICD loaders (a thin one preferably) and make it a dependency for other pkgs.

We only have one in our repos so there's nothing to do as far as I can tell.

In case I misunderstood something, please clarify.

He is saying that some people might want to have both Nvidia, intel or AMD OpenCL installed, like this guy here (comment by Void-995, 4th from top) http://aur.archlinux.org/packages.php?ID=21933 -- дамјан

Nicolas Bigaouette

2:46 p.m.

I've found somebody who tried to implement his own ICD loader: https://github.com/Max-E/libclicd It was not updated for many months, and may be missing a lot of feature. It might be easier to create a package that just provides /usr/lib/libOpenCL.so taken from one of the different implementation. I suggest taking one that supports OpenCL 1.1. I think nvidia support only 1.0, while intel and amd support 1.1. As it looks, /usr/lib/libOpenCL.so does not provide much: it's just a wrapper (the ICD Loader). Intel's is 27 kB while AMD's and nvidia's are 21 kB each. Here's each one's ldd: # AMD $ ldd /opt/amdstream/lib/x86_64/libOpenCL.so.1 linux-vdso.so.1 => (0x00007fffab8e7000) libpthread.so.0 => /lib/libpthread.so.0 (0x00007fa4b6d4b000) libdl.so.2 => /lib/libdl.so.2 (0x00007fa4b6b47000) libc.so.6 => /lib/libc.so.6 (0x00007fa4b67e5000) /lib/ld-linux-x86-64.so.2 (0x00007fa4b71af000) # Nvidia (on Gentoo) ldd /usr/lib64/libOpenCL.so.1.0.0 linux-vdso.so.1 => (0x00007fffa25ff000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f68d4981000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f68d477c000) libc.so.6 => /lib64/libc.so.6 (0x00007f68d4416000) /lib64/ld-linux-x86-64.so.2 (0x00007f68d4dcf000) # Intel ldd /opt/intel/opencl-sdk/usr/lib64/libOpenCL.so linux-vdso.so.1 => (0x00007fff991ff000) libdl.so.2 => /lib/libdl.so.2 (0x00007f198f4b9000) libnuma.so.1 => /usr/lib/libnuma.so.1 (0x00007f198f2b1000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f198efa6000) libm.so.6 => /lib/libm.so.6 (0x00007f198ed24000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f198eb0e000) libc.so.6 => /lib/libc.so.6 (0x00007f198e7ac000) /lib/ld-linux-x86-64.so.2 (0x00007f198f906000) On the license side, Intel is not clear. There is an EULA here: http://software.intel.com/en-us/articles/opencl-sdk-EULA/ but it does not say anything about OpenCL... It seems to be a generic EULA from Intel's compiler. But there is an llvm licence file in the package (llvm_release_license.txt). Their compiler is probably based on it. But that's not the wrapper. AMD has a license in /opt/amdstream/docs/opencl/LICENSES but I did not had the courage to read it. Section 2 b) seems to talk about "you may not [...] modify, network, rent, lend, loan, distribute or create derivative works based upon the Software in whole or in part;" What's is forbidden? Modify or distribute the package or modify/distribute a derivative work? I'm just lost. I can't find a license for nvidia, but it's probably as cryptic... I think it's a good idea to have an ICD wrapper. Actually I need that for myself.

...

Therefore, a sollution might be to simply pick one of available ICD

...
loaders (a thin one preferably) and make it a dependency for other pkgs.

We only have one in our repos so there's nothing to do as far as I can tell.

In case I misunderstood something, please clarify.

There might be just one right now, but OpenCL standard says many different implementation should be able to install side by side, and the choice between them is taken by the ICD loader.

Vojtěch Král

2:50 p.m.

Thankyou Nicolas for the research, that's exactly what we need. And yes Damjan, you are right.

Vojtěch Král

3:34 p.m.

2011/7/7 Nicolas Bigaouette :

...

I've found somebody who tried to implement his own ICD loader: https://github.com/Max-E/libclicd It was not updated for many months, and may be missing a lot of feature.

It might be easier to create a package that just provides /usr/lib/libOpenCL.so taken from one of the different implementation. I suggest taking one that supports OpenCL 1.1. I think nvidia support only 1.0, while intel and amd support 1.1.

Although that libclicd thingy looks outdated and/or incomplete, I like the testapps/list_platforms.c utility. It would be nice to have something like that in (a possible) wrapper ICD loader package (the /usr/lib/libOpenCL.so). AMD has a similar (more verbose) utility called 'clinfo'. I have experimentally removed the /usr/lib/libOpenCL.so file from the Intel sdk, installed it alongisde the AMD sdk and tried what these utils would output. The result looks quite nice, see http://codepad.org/NGaPsYbr ~kralyk

Nicolas Bigaouette

3:50 p.m.

...

Although that libclicd thingy looks outdated and/or incomplete, I like the testapps/list_platforms.c utility. It would be nice to have something like that in (a possible) wrapper ICD loader package (the /usr/lib/libOpenCL.so).

AMD has a similar (more verbose) utility called 'clinfo'. I have experimentally removed the /usr/lib/libOpenCL.so file from the Intel sdk, installed it alongisde the AMD sdk and tried what these utils would output. The result looks quite nice, see http://codepad.org/NGaPsYbr

I've just written such a library to print available platforms and their devices. Here's the output of it on my Q6600 cpu: https://gist.github.com/1069813 This is "just" a library, but a simple main() that calls its .Initialize() could print this easily. I'd wish to release that library as opensource at some point. Maybe this could be a nice time for it ;)

Ionut Biru

4:34 p.m.

On 07/07/2011 05:46 PM, Nicolas Bigaouette wrote: <snip>

...

There might be just one right now, but OpenCL standard says many different implementation should be able to install side by side, and the choice between them is taken by the ICD loader.

OKKKKKKK after talking with Lukas i now have an idea about what you guys talking about. Right now, what can i do is to take care about nvidia's opencl implementation and loader. What I understand from Nicolas is that we can use any loader, either form nvidia, ati, intel. It will work with any implementation. My propose is to split opencl loader and implementation out from nvidia-utils like this: libcl - containing /usr/lib/libOpenCL.so*. This can be easy replaced with an opensource loader once is available in mesa or other that is better. libcl-nvidia - depending on libcl containing /etc/OpenCL/vendors/nvidia.icd and libcuda.so and maybe nvidia module(need confirmation) for you guys: libcl-intel - depending on libcl libcl-amd - idem How does it sound for you? -- Ionuț

Lukáš Jirkovský

4:58 p.m.

On 7 July 2011 18:34, Ionut Biru wrote:

...

On 07/07/2011 05:46 PM, Nicolas Bigaouette wrote:

<snip>

OKKKKKKK after talking with Lukas i now have an idea about what you guys talking about.

Right now, what can i do is to take care about nvidia's opencl implementation and loader.

What I understand from Nicolas is that we can use any loader, either form nvidia, ati, intel. It will work with any implementation.

My propose is to split opencl loader and implementation out from nvidia-utils like this:

libcl - containing /usr/lib/libOpenCL.so*. This can be easy replaced with an opensource loader once is available in mesa or other that is better. libcl-nvidia - depending on libcl containing /etc/OpenCL/vendors/nvidia.icd and libcuda.so and maybe nvidia module(need confirmation)

for you guys: libcl-intel - depending on libcl libcl-amd - idem

How does it sound for you?

-- Ionuț

I support this proposal. First I was not convinced that splitting nvidia-utils to three packages is necessary and that two packages would be enough. However OpenCL implementation is independent on OpenGL and it's possible to use multiple devices at once it actually makes sense. Lukas

Vojtěch Král

5:36 p.m.

@Ionut, Lukas: I support this proposal too and I'd agree that only the libOpenCL.so (the ICD loader) needs to be split off nvidia-utils. Otherwise it could just stay intact as far as I can tell... Also, it would probably be wise to settle for dependency names once for all, by dependency names I mean what the pkgs provide - right now, all OpenCL pkgs provide both 'opencl' and 'libcl'. So following your proposal, 'libcl' could be provided by the loader, while 'opencl' could be provided by opencl implementations. Or something along those lines... @Nicolas: That looks great. If you released that as opensource, it'd probably be a better choice over the clinfo by Amd. ~kralyk Dne 7. července 2011 18:58 Lukáš Jirkovský napsal(a):

...

On 7 July 2011 18:34, Ionut Biru wrote:

...
On 07/07/2011 05:46 PM, Nicolas Bigaouette wrote:

<snip>

OKKKKKKK after talking with Lukas i now have an idea about what you guys talking about.

Right now, what can i do is to take care about nvidia's opencl implementation and loader.

What I understand from Nicolas is that we can use any loader, either form nvidia, ati, intel. It will work with any implementation.

My propose is to split opencl loader and implementation out from nvidia-utils like this:

libcl - containing /usr/lib/libOpenCL.so*. This can be easy replaced with an opensource loader once is available in mesa or other that is better. libcl-nvidia - depending on libcl containing /etc/OpenCL/vendors/nvidia.icd and libcuda.so and maybe nvidia module(need confirmation)

for you guys: libcl-intel - depending on libcl libcl-amd - idem

How does it sound for you?

-- Ionuț

I support this proposal.

First I was not convinced that splitting nvidia-utils to three packages is necessary and that two packages would be enough. However OpenCL implementation is independent on OpenGL and it's possible to use multiple devices at once it actually makes sense.

Lukas

Vojtěch Král

8 Jul 8 Jul

6:49 p.m.

So, what is going on? Don't you guys quit on me... @Nicolas: You pointed out that the nVidia ICD loader (libOpenCL.so) is only 1.0 (outdated), that is correct, but the ICD specs only require 1.0, so 1.1 possibly should not be needed. See specs: http://www.khronos.org/registry/cl/extensions/khr/cl_khr_icd.txt Although this would require testing. I'll do it later if possible... ~kralyk

Nicolas Bigaouette

7:11 p.m.

...

@Nicolas: You pointed out that the nVidia ICD loader (libOpenCL.so) is only 1.0 (outdated), that is correct, but the ICD specs only require 1.0, so 1.1 possibly should not be needed. See specs: http://www.khronos.org/registry/cl/extensions/khr/cl_khr_icd.txt

I think it requires 1.0 to provide, well, 1.0... Version 1.1 adds many things. For example, there is no type3 (float3, int3, etc.) in 1.0, just in 1.1. So if your code uses float3, then you'll probably get some undefined symbols when linking to a 1.0 libOpenCL.so. This is a guess though. If this is true, then using nvidia's loader would prevent any 1.1 implementation from being compiled/loaded. Not good. As for the library I wrote, I think I will release it, probably GPL3. Bear in mind that it was written to abstract the OpenCL initialization out of my other programs. In that sense, it might still contain some details specific to my projets. For example, it uses "std_cout" instead of "std::cout", which allows logging to a file every printed statement. I might add an #ifndef std_cout #define std_cout std::cout #endif, I think that should fix this. Also, the makefile are really targetted toward scientific simulations. But since it's only two files (one .hpp and one .cpp) it should be trivial to write a makefile... I think a package providing a libOpenCL.so and all others depending on it is the way to go too. About the name, I don't really care, as long as it's consistant. libcl is weird though, as the library is not "cl" but "opencl"... Also kralyk , regarding the intel ocl package, I think we should install it in /opt/intel/opencl-sdk, not /opt/intel-opencl-sdk. The reason is that other intel's product are installed in /opt/intel. For example, the compiler goes into /opt/intel/composerxe-<version>. Putting all intel product into its own folder make sense I think... At least it's consistant. N

Vojtěch Král

10:39 p.m.

2011/7/8 Nicolas Bigaouette :

...

I think it requires 1.0 to provide, well, 1.0... Version 1.1 adds many things. For example, there is no type3 (float3, int3, etc.) in 1.0, just in 1.1. So if your code uses float3, then you'll probably get some undefined symbols when linking to a 1.0 libOpenCL.so. This is a guess though.

Ok, I'll try to put together some 1.1-specific code to test this. If you have such a code already, let me know...

...

Also kralyk , regarding the intel ocl package, I think we should install it in /opt/intel/opencl-sdk, not /opt/intel-opencl-sdk.

Alright, I'll update the package. ~kralyk

Vojtěch Král

11:20 p.m.

I think that the "libcl" pkg might very well be based off of amd's loader. I made a simple PKGBUILD just to demonstrate: http://codepad.org/iK2oBo0W It contains the shared object and OpenCL info utility, it is up to date, has no dependencies and is only 88kB in size ;-) ~kralyk

Nicolas Bigaouette

9 Jul 9 Jul

2:38 p.m.

...

I made a simple PKGBUILD just to demonstrate: http://codepad.org/iK2oBo0W

Looks fine to me! I'll try it later on.

Vojtěch Král

5:16 p.m.

@Nicolas I've put it up on AUR: https://aur.archlinux.org/packages.php?ID=50587 (named 'libopencl' cause 'libcl' is not allowed on aur) Both Intel and AMD sdk now depend on it, so you can install both of them at the same time. The problem with this libopencl pkg is the god damned license - - it probably couldn't go into repo because of that. On the other hand, would someone really be worried about that? :) ~kralyk 2011/7/9 Nicolas Bigaouette :

...

...
I made a simple PKGBUILD just to demonstrate: http://codepad.org/iK2oBo0W

Looks fine to me! I'll try it later on.

Ionut Biru

15 Jul 15 Jul

2:25 p.m.

On 07/08/2011 09:49 PM, Vojtěch Král wrote:

...

So, what is going on? Don't you guys quit on me...

@Nicolas: You pointed out that the nVidia ICD loader (libOpenCL.so) is only 1.0 (outdated), that is correct, but the ICD specs only require 1.0, so 1.1 possibly should not be needed. See specs: http://www.khronos.org/registry/cl/extensions/khr/cl_khr_icd.txt Although this would require testing. I'll do it later if possible...

~kralyk

i've been waiting a new nvidia version and now is time to push something into extra. in the end, is it settled that the name of the package for loader should be _libcl_ and the implementation _opencl-nvidia_ ? -- Ionuț

Ionut Biru

17 Jul 17 Jul

3:37 p.m.

On 07/15/2011 05:25 PM, Ionut Biru wrote:

...

On 07/08/2011 09:49 PM, Vojtěch Král wrote:

...
So, what is going on? Don't you guys quit on me...

@Nicolas: You pointed out that the nVidia ICD loader (libOpenCL.so) is only 1.0 (outdated), that is correct, but the ICD specs only require 1.0, so 1.1 possibly should not be needed. See specs: http://www.khronos.org/registry/cl/extensions/khr/cl_khr_icd.txt Although this would require testing. I'll do it later if possible...

~kralyk

i've been waiting a new nvidia version and now is time to push something into extra.

in the end, is it settled that the name of the package for loader should be _libcl_ and the implementation _opencl-nvidia_ ?

Those are now in extra. I added couples of sonames into opencl-nvidia. Let me know what is bloat to remove and add it back into nvidia-utils. /etc/ /etc/OpenCL/ /etc/OpenCL/vendors/ /etc/OpenCL/vendors/nvidia.icd /usr/ /usr/lib/ /usr/lib/libcuda.so /usr/lib/libcuda.so.1 /usr/lib/libcuda.so.275.19 /usr/lib/libnvcuvid.so /usr/lib/libnvcuvid.so.1 /usr/lib/libnvcuvid.so.275.19 /usr/lib/libnvidia-compiler.so.275.19 -- Ionuț

ianux

4:33 p.m.

New subject: [arch-general] sdlmame post-install message

Hi archers, The sdlmame post-install script still echoes a warning about upgrading from 0.117 or earlier. Since this version was released in july 2007, we can assume that no current arch install still runs it. So, it should be safe to remove this warning, don't you think? -- radio ianux - http://ianux.fr/

Florian Pritz

4:35 p.m.

New subject: [arch-general] sdlmame post-install message

On 17.07.2011 18:33, ianux wrote:

...

Hi archers,

The sdlmame post-install script still echoes a warning about upgrading from 0.117 or earlier. Since this version was released in july 2007, we can assume that no current arch install still runs it. So, it should be safe to remove this warning, don't you think?

This doesn't seem to be related to "[arch-general] The OpenCL ICD problem". Next time please start a new thread instead of replying to and old one and changing the subject. -- Florian Pritz

ianux

5:53 p.m.

New subject: [arch-general] sdlmame post-install message

Le 17/07/11, Florian Pritz a écrit :

...

On 17.07.2011 18:33, ianux wrote:

...
Hi archers,

The sdlmame post-install script still echoes a warning about upgrading from 0.117 or earlier. Since this version was released in july 2007, we can assume that no current arch install still runs it. So, it should be safe to remove this warning, don't you think?

This doesn't seem to be related to "[arch-general] The OpenCL ICD problem".

Next time please start a new thread instead of replying to and old one and changing the subject.

It's true that I did it that way. Thanks for pointing this out, I won't do it again. -- radio ianux - http://ianux.fr/

Vojtěch Král

7:13 p.m.

Thanks Ionut! So far it looks ok, I think there's no bloat. I'll do some testing hopefully tonight. And with Nicolas' code too... Thanks for helping out, both of you... ~kralyk 2011/7/17 Ionut Biru :

...

...
i've been waiting a new nvidia version and now is time to push something into extra.

in the end, is it settled that the name of the package for loader should be _libcl_ and the implementation _opencl-nvidia_ ?

Those are now in extra. I added couples of sonames into opencl-nvidia. Let me know what is bloat to remove and add it back into nvidia-utils.

/etc/ /etc/OpenCL/ /etc/OpenCL/vendors/ /etc/OpenCL/vendors/nvidia.icd /usr/ /usr/lib/ /usr/lib/libcuda.so /usr/lib/libcuda.so.1 /usr/lib/libcuda.so.275.19 /usr/lib/libnvcuvid.so /usr/lib/libnvcuvid.so.1 /usr/lib/libnvcuvid.so.275.19 /usr/lib/libnvidia-compiler.so.275.19

-- Ionuț

Vojtěch Král

8 Aug 8 Aug

7:04 p.m.

Hi, today I just randomly checked clinfo and it says: FATAL: Module nvidia not found. I have opencl-nvidia installed. This happens regardless to which libcl I install. I tried putting full path in /etc/OpenCL/vendors/nvidia.icd but it's still the same... Is it just me or is there something wrong with opnecl-nvidia? ~kralyk Dne 17. července 2011 21:13 Vojtěch Král napsal(a):

...

Thanks Ionut! So far it looks ok, I think there's no bloat. I'll do some testing hopefully tonight. And with Nicolas' code too...

Thanks for helping out, both of you...

~kralyk

2011/7/17 Ionut Biru :

...
...
i've been waiting a new nvidia version and now is time to push something into extra.

in the end, is it settled that the name of the package for loader should be _libcl_ and the implementation _opencl-nvidia_ ?

Those are now in extra. I added couples of sonames into opencl-nvidia. Let me know what is bloat to remove and add it back into nvidia-utils.

/etc/ /etc/OpenCL/ /etc/OpenCL/vendors/ /etc/OpenCL/vendors/nvidia.icd /usr/ /usr/lib/ /usr/lib/libcuda.so /usr/lib/libcuda.so.1 /usr/lib/libcuda.so.275.19 /usr/lib/libnvcuvid.so /usr/lib/libnvcuvid.so.1 /usr/lib/libnvcuvid.so.275.19 /usr/lib/libnvidia-compiler.so.275.19

-- Ionuț

Ionut Biru

7:21 p.m.

On 08/08/2011 10:04 PM, Vojtěch Král wrote:

...

Hi, today I just randomly checked clinfo and it says: FATAL: Module nvidia not found. I have opencl-nvidia installed. This happens regardless to which libcl I install. I tried putting full path in /etc/OpenCL/vendors/nvidia.icd but it's still the same...

Is it just me or is there something wrong with opnecl-nvidia?

~kralyk

i guess you need to install nvidia package as well and load the kernel module -- Ionuț

4659

Age (days ago)

4691

Last active (days ago)

List overview

Download

23 comments

7 participants

participants (7)

Damjan
Florian Pritz
ianux
Ionut Biru
Lukáš Jirkovský
Nicolas Bigaouette
Vojtěch Král