And there seems to be, but, this test is very bad in multiple ways. Different versions of just about everything was used (glibc for 32-bit, musl for 64-bit, different gcc versions for building, etc) and also because raspbian is built for armv6 it will loose from the start.
These are just preliminary finds so take them with a grain of salt. Still, interesting...
|OpenSSL hmac(md5) speed test comparing 32-bit and 64-bit build|
|OpenSSL sha512 speed test comparing 32-bit and 64-bit builds|
32-bit: MWIPS 1038.823
64-bit: MWIPS 1379.241