[pacman-dev] [PATCH 2/3] Use OpenSSL MD5 crypto functions if available

[Dan McGee]

On Mon, Sep 6, 2010 at 10:04 AM, Allan McRae <allan at archlinux.org> wrote:
> On 07/09/10 00:25, Bryce Gibson wrote:
>>
>> ----- Original message -----
>>>
>>> On 04/09/10 19:16, Jürgen Hötzel wrote:
>>>>
>>>> Hi Dan,
>>>>
>>>> 2010/9/2 Dan McGee<dan at archlinux.org>:
>>>>>
>>>>> This does not remove the MD5 code from our codebase, but it does
>>>>> enable linking against OpenSSL to get their much faster
>>>>> implementation if it is available on whatever platform you are
>>>>> using. At configure-time, we will default to using it if it is
>>>>> available, but this can be easily changed by using the
>>>>> `--with-openssl` or `--without-openssl` arguments to configure.
>>>>
>>>> What about just replacing the current MD5 implementation with the
>>>> OpenSSL implementation?
>>>>
>>>> This would prevent conditional compilation and a direct OpenSSL
>>>> dependency in libalpm.
>>>
>>> Can we do that?   Openssl is BSD code.
>>>
>>>
>>> Anyway, I have concerns...     Think of an openssl upgrade.   pacman is
>>> in
>>> SyncFirst and it pulls in all its deps.   If that pulls in openssl with a
>>> soname bump, things may get interesting.   I have not check, but I do not
>>> think --as-needed saves us there.
>>>
>>> Allan
>>>
>> The licenses that openssl is released under aren't compatible with the gpl
>> (according to the fsf) because of the advertising clause... So openssl code
>> can't really be used like that... (Unfortunately)
>
> How does the coreutils md5sum speed compare?  That is GPL code so could be
> included directly.  Though I guess it is now GPL3 being a GNU project...

Quite a bit slower. Test results (for one platform, please test on
whatever you are running on, it is very machine-dependent) and the
test harness included below, run using ./md5-speed.sh <bigfile>

Keep in mind that "including directly" more than likely is a drawback.
I wouldn't be surprised if there are assembly routines involved in
both of these codebases which just makes the build process that much
more involved and we have to work it into our code. Not that I think
these routines change much upstream, but it is one more thing to keep
up to date.

-Dan



dmcgee at dublin /tmp
$ ./md5-speed.sh ~/downloads/Fireworks.amb
path: /home/dmcgee/downloads/Fireworks.amb
md5(/home/dmcgee/downloads/Fireworks.amb) = 987fef0281e0924977fc8d14f78f4cd4
Version: i686
real    0m9.019s
user    0m7.623s
sys     0m1.143s

path: /home/dmcgee/downloads/Fireworks.amb
md5(/home/dmcgee/downloads/Fireworks.amb) = 987fef0281e0924977fc8d14f78f4cd4
Version: atom
real    0m9.082s
user    0m7.736s
sys     0m1.120s

path: /home/dmcgee/downloads/Fireworks.amb
md5(/home/dmcgee/downloads/Fireworks.amb) = 987fef0281e0924977fc8d14f78f4cd4
Version: native
real    0m9.078s
user    0m7.739s
sys     0m1.103s

987fef0281e0924977fc8d14f78f4cd4  /home/dmcgee/downloads/Fireworks.amb
Program: md5sum
real    0m8.034s
user    0m6.963s
sys     0m0.870s

MD5(/home/dmcgee/downloads/Fireworks.amb)= 987fef0281e0924977fc8d14f78f4cd4
Program: openssl dgst -md5
real    0m6.511s
user    0m5.356s
sys     0m0.980s
-------------- next part --------------
A non-text attachment was scrubbed...
Name: md5.c
Type: text/x-csrc
Size: 10314 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/pacman-dev/attachments/20100906/ee09d8ad/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: md5-speed.sh
Type: application/x-sh
Size: 744 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/pacman-dev/attachments/20100906/ee09d8ad/attachment.sh>