[pacman-dev] [PATCH 2/3] Use OpenSSL MD5 crypto functions if available

[Allan McRae]

On 07/09/10 01:15, Dan McGee wrote:
> On Mon, Sep 6, 2010 at 10:04 AM, Allan McRae<allan at archlinux.org>  wrote:
>> On 07/09/10 00:25, Bryce Gibson wrote:
>>>
>>> ----- Original message -----
>>>>
>>>> On 04/09/10 19:16, Jürgen Hötzel wrote:
>>>>>
>>>>> Hi Dan,
>>>>>
>>>>> 2010/9/2 Dan McGee<dan at archlinux.org>:
>>>>>>
>>>>>> This does not remove the MD5 code from our codebase, but it does
>>>>>> enable linking against OpenSSL to get their much faster
>>>>>> implementation if it is available on whatever platform you are
>>>>>> using. At configure-time, we will default to using it if it is
>>>>>> available, but this can be easily changed by using the
>>>>>> `--with-openssl` or `--without-openssl` arguments to configure.
>>>>>
>>>>> What about just replacing the current MD5 implementation with the
>>>>> OpenSSL implementation?
>>>>>
>>>>> This would prevent conditional compilation and a direct OpenSSL
>>>>> dependency in libalpm.
>>>>
>>>> Can we do that?   Openssl is BSD code.
>>>>
>>>>
>>>> Anyway, I have concerns...     Think of an openssl upgrade.   pacman is
>>>> in
>>>> SyncFirst and it pulls in all its deps.   If that pulls in openssl with a
>>>> soname bump, things may get interesting.   I have not check, but I do not
>>>> think --as-needed saves us there.
>>>>
>>>> Allan
>>>>
>>> The licenses that openssl is released under aren't compatible with the gpl
>>> (according to the fsf) because of the advertising clause... So openssl code
>>> can't really be used like that... (Unfortunately)
>>
>> How does the coreutils md5sum speed compare?  That is GPL code so could be
>> included directly.  Though I guess it is now GPL3 being a GNU project...
>
> Quite a bit slower. Test results (for one platform, please test on
> whatever you are running on, it is very machine-dependent) and the
> test harness included below, run using ./md5-speed.sh<bigfile>


 > . md5-speed.sh 
/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz
path: /home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz
md5(/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz) = 
40a30098649adf29ba79fcd6699d5f67
Version: i686
real	0m28.441s
user	0m25.692s
sys	0m2.643s

path: /home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz
md5(/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz) = 
40a30098649adf29ba79fcd6699d5f67
Version: native
real	0m28.527s
user	0m25.455s
sys	0m2.970s

40a30098649adf29ba79fcd6699d5f67 
/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz
Program: md5sum
real	0m22.553s
user	0m20.552s
sys	0m1.923s

MD5(/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz)= 
40a30098649adf29ba79fcd6699d5f67
Program: openssl dgst -md5
real	0m22.436s
user	0m19.745s
sys	0m2.650s

So not much difference here.

> Keep in mind that "including directly" more than likely is a drawback.
> I wouldn't be surprised if there are assembly routines involved in
> both of these codebases which just makes the build process that much
> more involved and we have to work it into our code. Not that I think
> these routines change much upstream, but it is one more thing to keep
> up to date.

Well,  how often do we keep the current md5 code up to date?  Has anyone 
checked the upstream source lately?   It may have gotten faster (and it 
is still GPL2) making the whole thing moot, which it really is anyway...  :P

Allan