[arch-dev-public] linux-firmware package, some thoughts

Tobias Powalowski

27 Sep 2021 27 Sep '21

6:30 p.m.

Hi guys, lately did a lot of archboot hacking and stumbled over our firmware package. It needs installed 750 MB, download size is 130 MB. I tried to get it smaller and uploaded the archboot-linux-firmware package for my purposes. Our linux kernel needs only 180 MB installed size, download size is about 37 MB. Wouldn't it make sense to provide for each kernel we ship, a separate firmware package? Just my 2 cwnts, greetings tpowa -- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org

Show replies by date

Jan Alexander Steffens (heftig)

27 Sep 27 Sep

7:13 p.m.

On Mon, Sep 27, 2021 at 8:30 PM Tobias Powalowski via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:

...

Hi guys, lately did a lot of archboot hacking and stumbled over our firmware package. It needs installed 750 MB, download size is 130 MB. I tried to get it smaller and uploaded the archboot-linux-firmware package for my purposes. Our linux kernel needs only 180 MB installed size, download size is about 37 MB. Wouldn't it make sense to provide for each kernel we ship, a separate firmware package? Just my 2 cwnts, greetings tpowa

The method used there wouldn't produce a usable firmware package for me. For example, `modinfo -F firmware iwlwifi` lists `iwlwifi-Qu-c0-hr-b0-64.ucode` but linux-firmware does not contain that file. The driver will fall back to the next lower version, `iwlwifi-Qu-c0-hr-b0-63.ucode`, which is in linux-firmware but not in the modinfo output. I think the best we can do is split some of the more obscure vendors that have large firmware sets, like Netronome, Qualcomm, Mellanox, Marvell, Cavium, QLogic and NXP.

Jelle van der Waa

7:46 p.m.

On 27/09/2021 21:13, Jan Alexander Steffens (heftig) via arch-dev-public wrote:

...

On Mon, Sep 27, 2021 at 8:30 PM Tobias Powalowski via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:

...
Hi guys, lately did a lot of archboot hacking and stumbled over our firmware package. It needs installed 750 MB, download size is 130 MB. I tried to get it smaller and uploaded the archboot-linux-firmware package for my purposes. Our linux kernel needs only 180 MB installed size, download size is about 37 MB. Wouldn't it make sense to provide for each kernel we ship, a separate firmware package? Just my 2 cwnts, greetings tpowa

The method used there wouldn't produce a usable firmware package for me. For example, `modinfo -F firmware iwlwifi` lists `iwlwifi-Qu-c0-hr-b0-64.ucode` but linux-firmware does not contain that file. The driver will fall back to the next lower version, `iwlwifi-Qu-c0-hr-b0-63.ucode`, which is in linux-firmware but not in the modinfo output.

I think the best we can do is split some of the more obscure vendors that have large firmware sets, like Netronome, Qualcomm, Mellanox, Marvell, Cavium, QLogic and NXP.

I second this, the biggest firmware sizes are from these fancynetwork equipemnt folks such as netronome. An alternative is enabling xz compression for the firmware itself this should save space as well, but it depends on support in the LTS kernel. Greetings, Jelle van der Waa

Giancarlo Razzolini

7:54 p.m.

Em setembro 27, 2021 16:46 Jelle van der Waa via arch-dev-public escreveu:

...

On 27/09/2021 21:13, Jan Alexander Steffens (heftig) via arch-dev-public wrote:

...
On Mon, Sep 27, 2021 at 8:30 PM Tobias Powalowski via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:

...
Hi guys, lately did a lot of archboot hacking and stumbled over our firmware package. It needs installed 750 MB, download size is 130 MB. I tried to get it smaller and uploaded the archboot-linux-firmware package for my purposes. Our linux kernel needs only 180 MB installed size, download size is about 37 MB. Wouldn't it make sense to provide for each kernel we ship, a separate firmware package? Just my 2 cwnts, greetings tpowa

The method used there wouldn't produce a usable firmware package for me. For example, `modinfo -F firmware iwlwifi` lists `iwlwifi-Qu-c0-hr-b0-64.ucode` but linux-firmware does not contain that file. The driver will fall back to the next lower version, `iwlwifi-Qu-c0-hr-b0-63.ucode`, which is in linux-firmware but not in the modinfo output.

I think the best we can do is split some of the more obscure vendors that have large firmware sets, like Netronome, Qualcomm, Mellanox, Marvell, Cavium, QLogic and NXP.

I second this, the biggest firmware sizes are from these fancynetwork equipemnt folks such as netronome. An alternative is enabling xz compression for the firmware itself this should save space as well, but it depends on support in the LTS kernel.

Can't we do both? Enable xz and split the larger firmwares? Or LTS don't have plans to support xz for firmware? Regards, Giancarlo Razzolini

Tobias Powalowski

28 Sep 28 Sep

4:55 a.m.

Hi, according to google in 5.3 the xz compression was introduced so all kernels we ship should support it. greetings tpowa Am Mo., 27. Sept. 2021 um 21:54 Uhr schrieb Giancarlo Razzolini via arch-dev-public :

...

...
On 27/09/2021 21:13, Jan Alexander Steffens (heftig) via arch-dev-public wrote:

...
On Mon, Sep 27, 2021 at 8:30 PM Tobias Powalowski via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:

...
Hi guys, lately did a lot of archboot hacking and stumbled over our firmware package. It needs installed 750 MB, download size is 130 MB. I tried to get it smaller and uploaded the archboot-linux-firmware

...
...
...
for my purposes. Our linux kernel needs only 180 MB installed size, download size is about 37 MB. Wouldn't it make sense to provide for each kernel we ship, a separate firmware package? Just my 2 cwnts, greetings tpowa

The method used there wouldn't produce a usable firmware package for me. For example, `modinfo -F firmware iwlwifi` lists `iwlwifi-Qu-c0-hr-b0-64.ucode` but linux-firmware does not contain that file. The driver will fall back to the next lower version, `iwlwifi-Qu-c0-hr-b0-63.ucode`, which is in linux-firmware but not in

...
...
modinfo output.

I think the best we can do is split some of the more obscure vendors

Em setembro 27, 2021 16:46 Jelle van der Waa via arch-dev-public escreveu: package the that

...
...
have large firmware sets, like Netronome, Qualcomm, Mellanox, Marvell, Cavium, QLogic and NXP.

I second this, the biggest firmware sizes are from these fancynetwork equipemnt folks such as netronome. An alternative is enabling xz compression for the firmware itself this should save space as well, but it depends on support in the LTS kernel.

Can't we do both? Enable xz and split the larger firmwares? Or LTS don't have plans to support xz for firmware?

Regards, Giancarlo Razzolini

-- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org

Tobias Powalowski

6:34 a.m.

Hi, short summarize from irc: https://fedoraproject.org/wiki/Changes/CompressKernelFirmware xz -C crc32 is the only supported mode mkinitcpio needs to be patched: https://bugs.archlinux.org/task/72263

...

From my side unknown, if dracut and booster can handle compressed firmware files.

greetings tpowa Am Di., 28. Sept. 2021 um 06:55 Uhr schrieb Tobias Powalowski < tobias.powalowski@googlemail.com>:

...

Hi, according to google in 5.3 the xz compression was introduced so all kernels we ship should support it. greetings tpowa

Am Mo., 27. Sept. 2021 um 21:54 Uhr schrieb Giancarlo Razzolini via arch-dev-public :

...
...
On 27/09/2021 21:13, Jan Alexander Steffens (heftig) via arch-dev-public wrote:

...
On Mon, Sep 27, 2021 at 8:30 PM Tobias Powalowski via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:

...
Hi guys, lately did a lot of archboot hacking and stumbled over our firmware package. It needs installed 750 MB, download size is 130 MB. I tried to get it smaller and uploaded the archboot-linux-firmware

...
...
...
for my purposes. Our linux kernel needs only 180 MB installed size, download size is about 37 MB. Wouldn't it make sense to provide for each kernel we ship, a separate firmware package? Just my 2 cwnts, greetings tpowa

The method used there wouldn't produce a usable firmware package for me. For example, `modinfo -F firmware iwlwifi` lists `iwlwifi-Qu-c0-hr-b0-64.ucode` but linux-firmware does not contain that file. The driver will fall back to the next lower version, `iwlwifi-Qu-c0-hr-b0-63.ucode`, which is in linux-firmware but not in

...
...
modinfo output.

I think the best we can do is split some of the more obscure vendors

Em setembro 27, 2021 16:46 Jelle van der Waa via arch-dev-public escreveu: package the that

...
...
have large firmware sets, like Netronome, Qualcomm, Mellanox, Marvell, Cavium, QLogic and NXP.

I second this, the biggest firmware sizes are from these fancynetwork equipemnt folks such as netronome. An alternative is enabling xz compression for the firmware itself this should save space as well, but it depends on support in the LTS kernel.

Can't we do both? Enable xz and split the larger firmwares? Or LTS don't have plans to support xz for firmware?

Regards, Giancarlo Razzolini

-- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org

-- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org

Morten Linderud

7:10 a.m.

On Tue, Sep 28, 2021 at 08:34:45AM +0200, Tobias Powalowski via arch-dev-public wrote:

...

Hi, short summarize from irc: https://fedoraproject.org/wiki/Changes/CompressKernelFirmware xz -C crc32 is the only supported mode

mkinitcpio needs to be patched: https://bugs.archlinux.org/task/72263

From my side unknown, if dracut and booster can handle compressed firmware files.

Namarrgon submitted a WIP patch for this in August as a side-note. https://github.com/archlinux/mkinitcpio/pull/65 -- Morten Linderud PGP: 9C02FF419FECBE16

Jonas Witschel

8:35 a.m.

On 2021-09-28 08:34, Tobias Powalowski via arch-dev-public wrote:

...

From my side unknown, if dracut and booster can handle compressed firmware files.

For dracut this is supposed to work [1], though note that I haven't tested this myself on Arch. Booster seems to read the name of the firmware file from the kernel module [2], then use that name to find the file in the firmware directory [3,4]. It seems as if the appendFirmwareFiles() function might need to get patched to look for `fwPath + ".xz"` in addition to `fwPath`, though I could be totally wrong here. Best, Jonas [1] https://github.com/dracutdevs/dracut/commit/999cfa84582ab4ce4cc602242cb71d0a... [2] https://github.com/anatol/booster/blob/d8caf32a34527085f85764f262151bcb2cc07... [3] https://github.com/anatol/booster/blob/d8caf32a34527085f85764f262151bcb2cc07... [4] https://github.com/anatol/booster/blob/8c3f6fee7604170229ed0dfb1fdba02f5944c...

941

Age (days ago)

942

Last active (days ago)

List overview

Download

7 comments

6 participants

participants (6)

Giancarlo Razzolini
Jan Alexander Steffens (heftig)
Jelle van der Waa
Jonas Witschel
Morten Linderud
Tobias Powalowski

[arch-dev-public] linux-firmware package, some thoughts

tags

participants (6)