Back in January, I made a tool to manipulate the CPUID of VIA processors and published
the code here in the hope that somebody would test a lot of programs to see if
the performance depends on the CPU vendor. The research staff of a Russian IT
webzine iXBT.com offered to help with this. I
gave them some equipment that they couldn't get in Russia, and they have tested
a lot of programs. Their results are published in
Russian, and later also in English.
The Russian researchers found several programs where the performance depended
on the vendor name on the CPU. While this is ground for suspicion, it does not
necessarily mean that Intel software is involved. It is necessary to make a
deeper investigation in order to see if the programs are compiled with an Intel
compiler or an Intel function library.
I noticed from the screening results that Matlab and Mathematica were among
the programs where the vendor name effect was highest. I decided therefore to
make further investigations of mathematical programs and found that Mathcad was
also affected. Matlab, Mathematica and Mathcad are the most commonly used math
programs at universities and colleges.
Below are the results of my investigations so far on a VIA Nano L3050, 1.8
GHz.
Mathematica
Mathematica version 7.0.1 was tested using the BenchmarkReport function that is included with the package.
The overall benchmark result for different (faked) CPU vendors was as follows (average of 5 tests):
Faked vendor and family number |
Benchmark
(higher is better) |
VIA |
1.078 |
AMD |
1.102 |
Intel, (nonexisting) family 7 |
1.102 |
Intel, family 6 |
1.114 |
A further investigation shows that Mathematica uses the Intel Math Kernel Library (MKL version 10.1 beta, 2008),
including the Vector Math Library (VML) which contains optimized code paths used exclusively for Intel processors, and the Gnu Multiple Precision Arithmetic Library (GMP,
no version info), which contains Intel-specific and AMD-specific code paths but no VIA-specific code paths. Another executable file (mathdll.dll) with Mathematica's own kernel code contains a check for the Intel vendor string, but I could not find out what the purpose of this check is, and I found no evidence that it originates from Intel software.
The benchmark is based on 15 different tests. Some of these tests were influenced by the CPU
ID, others were not. The benchmark for elementary mathematical functions was 26.9% better when the CPU was identified as an Intel
than when it was identified as anything else, probably due to the VML. The tests
for high precision math was better for both AMD and Intel, probably due to the
GMP.
Mathcad
Mathcad version 15.0 was tested with some simple benchmarks made by myself.
Matrix algebra was among the types of calculations that were highly affected by
the CPU ID. The calculation time for a series of matrix inversions was as
follows:
Faked CPU |
Computation time, s |
MKL version loaded |
Instruction set used |
VIA Nano |
69.6 |
default |
386 |
AMD Opteron |
68.7 |
default |
386 |
Intel Core 2 |
44.7 |
Pentium 3 |
SSE |
Intel Atom |
73.9 |
Pentium 3 |
SSE |
Intel Pentium 4 |
33.2 |
Pentium 4 w. SSE3 |
SSE3 |
Intel nonexisting fam. 7 |
69.5 |
default |
386 |
Using a debugger, I could verify that it uses an old version of Intel MKL (version 7.2.0,
2004), and that it loads different versions of the MKL depending on the CPU ID
as indicated in the table above. The speed is more than doubled when the CPU
fakes to be an Intel Pentium 4.
It is interesting that this version of MKL doesn't choose the optimal code path for an Intel Core 2. This proves my
point that dispatching by CPU model number rather than by instruction set is not sure to be optimal on future processors,
and that it sometimes takes years before a new library makes it to the end product. Any processor-specific optimization is likely to be obsolete at that time. In this case the library is
six years behind the software it is used in.
Matlab
I haven't got a Matlab package for testing yet, so the detailed results will
have to wait. However, it is known that Matlab uses Intel's MKL library. The
Russians report that Matlab runs 28% slower when the CPU identifies as a VIA
compared to an Intel Core 2.
Apparently, the Matlab people are aware of the problem because they have announced
that they are now using Intel's MKL library on Intel machines, and AMD's ACML
library on AMD machines for basic linear algebra calculations. However, this is
probably no improvement. Our Russian friends reported
two years ago that Matlab runs faster with MKL than with ACML on an AMD
machine!
It may sound like a fair solution that each CPU vendor makes its own function
libraries, but this can soon be a nightmare for the producers of application
software. This goes against the very principle of having a standardized
instruction set. And apparently, only Intel can afford the costs of optimizing
large function libraries on the detailed instruction level. They don't make much
money on these function libraries, and it is surely very costly to develop, test
and optimize such a big library of complicated mathematical functions, let alone
the costs of making a different version for every new instruction set extension.
The AMD libraries cannot match this level of optimization, and VIA can hardly
afford to make any function libraries at all.
This is the core of the problem. By investing in the development of large,
comprehensive and highly optimized math libraries, Intel have obtained a
dominating market position in mathematical software, but in a very subtle way.
They are not making the application software for the end user; but they are
making some of the tools and building blocks for making such software. This
enables them to manipulate the performance of this software on the CPUs produced
by their competitors. And this manipulation is completely invisible to the end
user and perhaps even to the application programmer. In fact, Intel have managed
to put their CPU dispatcher into an AMD function library, as revealed in a
previous post here.
Intel are putting themselves into an advantageous position by making better
function libraries than everybody else, and they are taking advantage of this
position by lowering the performance of common mathematical software on the CPUs
of their competitors relative to their own. We have probably not seen the end of the legal battles yet. |