Agner`s CPU blog

Software optimization resources | E-mail subscription to this blog | www.agner.org

Common math programs are affected
Author: Agner Fog Date: 2010-08-20 11:05

Back in January, I made a tool to manipulate the CPUID of VIA processors and published the code here in the hope that somebody would test a lot of programs to see if the performance depends on the CPU vendor. The research staff of a Russian IT webzine iXBT.com offered to help with this. I gave them some equipment that they couldn't get in Russia, and they have tested a lot of programs. Their results are published in Russian, and later also in English.

The Russian researchers found several programs where the performance depended on the vendor name on the CPU. While this is ground for suspicion, it does not necessarily mean that Intel software is involved. It is necessary to make a deeper investigation in order to see if the programs are compiled with an Intel compiler or an Intel function library.

I noticed from the screening results that Matlab and Mathematica were among the programs where the vendor name effect was highest. I decided therefore to make further investigations of mathematical programs and found that Mathcad was also affected. Matlab, Mathematica and Mathcad are the most commonly used math programs at universities and colleges.

Below are the results of my investigations so far on a VIA Nano L3050, 1.8 GHz.

Mathematica

Mathematica version 7.0.1 was tested using the BenchmarkReport function that is included with the package. The overall benchmark result for different (faked) CPU vendors was as follows (average of 5 tests):

Faked vendor and family number Benchmark
(higher is better)
VIA 1.078
AMD 1.102
Intel, (nonexisting) family 7 1.102
Intel, family 6 1.114

A further investigation shows that Mathematica uses the Intel Math Kernel Library (MKL version 10.1 beta, 2008), including the Vector Math Library (VML) which contains optimized code paths used exclusively for Intel processors, and the Gnu Multiple Precision Arithmetic Library (GMP, no version info), which contains Intel-specific and AMD-specific code paths but no VIA-specific code paths. Another executable file (mathdll.dll) with Mathematica's own kernel code contains a check for the Intel vendor string, but I could not find out what the purpose of this check is, and I found no evidence that it originates from Intel software.

The benchmark is based on 15 different tests. Some of these tests were influenced by the CPU ID, others were not. The benchmark for elementary mathematical functions was 26.9% better when the CPU was identified as an Intel than when it was identified as anything else, probably due to the VML. The tests for high precision math was better for both AMD and Intel, probably due to the GMP.

Mathcad

Mathcad version 15.0 was tested with some simple benchmarks made by myself. Matrix algebra was among the types of calculations that were highly affected by the CPU ID. The calculation time for a series of matrix inversions was as follows:

Faked CPU Computation time, s MKL version loaded Instruction set used
VIA Nano 69.6 default 386
AMD Opteron 68.7 default 386
Intel Core 2 44.7 Pentium 3 SSE
Intel Atom 73.9 Pentium 3 SSE
Intel Pentium 4 33.2 Pentium 4 w. SSE3 SSE3
Intel nonexisting fam. 7 69.5 default 386

Using a debugger, I could verify that it uses an old version of Intel MKL (version 7.2.0, 2004), and that it loads different versions of the MKL depending on the CPU ID as indicated in the table above. The speed is more than doubled when the CPU fakes to be an Intel Pentium 4.

It is interesting that this version of MKL doesn't choose the optimal code path for an Intel Core 2. This proves my point that dispatching by CPU model number rather than by instruction set is not sure to be optimal on future processors, and that it sometimes takes years before a new library makes it to the end product. Any processor-specific optimization is likely to be obsolete at that time. In this case the library is six years behind the software it is used in.

Matlab

I haven't got a Matlab package for testing yet, so the detailed results will have to wait. However, it is known that Matlab uses Intel's MKL library. The Russians report that Matlab runs 28% slower when the CPU identifies as a VIA compared to an Intel Core 2.

Apparently, the Matlab people are aware of the problem because they have announced that they are now using Intel's MKL library on Intel machines, and AMD's ACML library on AMD machines for basic linear algebra calculations. However, this is probably no improvement. Our Russian friends reported two years ago that Matlab runs faster with MKL than with ACML on an AMD machine!

It may sound like a fair solution that each CPU vendor makes its own function libraries, but this can soon be a nightmare for the producers of application software. This goes against the very principle of having a standardized instruction set. And apparently, only Intel can afford the costs of optimizing large function libraries on the detailed instruction level. They don't make much money on these function libraries, and it is surely very costly to develop, test and optimize such a big library of complicated mathematical functions, let alone the costs of making a different version for every new instruction set extension. The AMD libraries cannot match this level of optimization, and VIA can hardly afford to make any function libraries at all.

This is the core of the problem. By investing in the development of large, comprehensive and highly optimized math libraries, Intel have obtained a dominating market position in mathematical software, but in a very subtle way. They are not making the application software for the end user; but they are making some of the tools and building blocks for making such software. This enables them to manipulate the performance of this software on the CPUs produced by their competitors. And this manipulation is completely invisible to the end user and perhaps even to the application programmer. In fact, Intel have managed to put their CPU dispatcher into an AMD function library, as revealed in a previous post here.

Intel are putting themselves into an advantageous position by making better function libraries than everybody else, and they are taking advantage of this position by lowering the performance of common mathematical software on the CPUs of their competitors relative to their own. We have probably not seen the end of the legal battles yet.

 
thread Intel's "cripple AMD" function new - Agner Fog - 2009-12-30
reply Intel's new - Felid - 2010-01-01
replythread Intel's "cripple AMD" function new - inhahe - 2010-01-03
last replythread Intel's new - Agner Fog - 2010-01-04
replythread Intel's compiler is the best? new - Weber - 2010-01-04
last reply Intel's compiler is the best? new - Agner Fog - 2010-01-09
reply Intel article new - Agner Fog - 2010-01-22
last replythread Intel's new - Deng - 2016-12-11
last replythread Intel's "cripple AMD" function new - Biplab Raut - 2019-12-20
last reply Intel's cripple AMD function new - Agner - 2019-12-29
replythread Web Parallels new - Jeff Craig - 2010-01-04
last replythread More Parallels new - Agner Fog - 2010-01-23
reply Early Examples new - Yuhong Bao - 2010-02-01
last reply More Parallels new - Yuhong Bao - 2010-02-20
replythread New CPUID manipulation program new - Agner Fog - 2010-01-22
replythread CPUID manipulation through virtualization new - Andrew Lofthouse - 2010-08-16
reply CPUID manipulation through virtualization new - Agner Fog - 2010-08-16
replythread CPUID manipulation program for AMD new - Agner - 2010-10-01
last replythread CPUID manipulation program for AMD new - Ralf - 2012-01-30
last reply CPUID manipulation program for AMD new - Agner - 2012-01-31
last reply CPUID manipulation through virtualization new - akshay - 2015-07-08
last replythread New CPUID manipulation program new - AVK - 2011-02-09
last reply New CPUID manipulation program new - Agner - 2011-02-09
reply AMD Blog on compilers/benchmarch new - margaret lewis - 2010-02-01
replythread New version is still crippling Intel's competitors new - Agner Fog - 2010-06-29
last reply New version is still crippling Intel's competitors new - granyte - 2014-09-16
reply Out of court settlement with FTC new - Agner Fog - 2010-08-05
reply AMD library contains Intel's cripple-AMD function! new - Agner Fog - 2010-08-11
replythread Common math programs are affected - Agner Fog - 2010-08-20
last reply Preliminary test results for Matlab new - Agner Fog - 2010-09-16
replythread Overview of CPU dispatching in Intel software new - Agner Fog - 2010-08-23
last reply Overview of CPU dispatching in Intel software new - Mingye Wang - 2020-08-31
replythread New Intel compiler version - still the same! new - Agner Fog - 2010-09-22
reply GCC now has support for function dispatch new - Jean-Luc - 2010-09-27
replythread Intel compiler question new - James Russell - 2010-10-11
last reply Intel compiler question new - Agner - 2010-10-12
reply New Intel compiler version - still the same! new - Don Kretsch - 2010-11-29
last replythread New Intel compiler version - still the same! new - Daniel - 2011-12-23
last replythread New Intel compiler version - still the same! new - Agner - 2011-12-25
last replythread New Intel compiler version - still the same! new - Stanley Theamer - 2012-02-12
last reply New Intel compiler version - still the same! new - Stretcho - 2012-03-14
replythread Still no library that is optimal on all processors new - Agner - 2012-04-18
replythread Still no library that is optimal on all processors new - Guest - 2012-05-17
last replythread Still no library that is optimal on all processors new - Agner - 2012-05-17
last replythread Still no library that is optimal on all processors new - David - 2012-05-19
last replythread Still no library that is optimal on all processors new - Agner - 2012-05-20
last reply Still no library that is optimal on all processors new - Bubba_Hotepp - 2012-06-16
last replythread Still no library that is optimal on all processors new - Marat Dukhan - 2013-05-20
last replythread Still no library that is optimal on all processors new - Agner - 2013-05-21
last replythread This is still going on, wow just wow new - Vuurdraak - 2016-11-10
last replythread This is still going on, wow just wow new - Agner - 2016-11-10
last replythread This is still going on, wow just wow new - Vuurdraak - 2016-11-11
last replythread This is still going on, wow just wow new - Denis - 2017-01-02
last replythread This is still going on, wow just wow new - Agner - 2017-01-02
replythread RYZEN thoughts? new - Noob programmer - 2017-03-10
last replythread RYZEN thoughts? new - Chromatix - 2017-03-16
last replythread RYZEN thoughts? new - Peter - 2017-04-11
replythread RYZEN thoughts? new - Agner - 2017-04-12
last replythread RYZEN thoughts? new - Ballsystemlord - 2019-02-12
last reply RYZEN thoughts? new - Agner - 2019-02-13
last reply RYZEN thoughts? new - itsmydamnation - 2017-04-21
last reply This is still going on, wow just wow new - Naoki Shibata - 2017-07-19
replythread A long history of legal antitrust battles new - Agner - 2017-07-27
last replythread A long history of legal antitrust battles new - Jorcy Neto - 2017-07-27
last replythread A long history of legal antitrust battles new - Royi - 2018-02-19
last reply A long history of legal antitrust battles new - Agner - 2018-05-15
reply Intel's "cripple AMD" function new - PCPMD - 2019-02-27
replythread Patches and workarounds new - Neville C - 2019-11-21
last reply Patches and workarounds new - Mingye Wang - 2020-09-01
replythread Intel's "cripple AMD" function new - Walker - 2020-06-29
last replythread Intel's "cripple AMD" function new - Forsen - 2020-09-16
reply Intel's new - Agner - 2020-09-16
last replythread Intel's "cripple AMD" function new - ETERNALBLUEbullrun - 2021-11-28
last reply Intel's new - Agner - 2021-11-28
last replythread New Intel compiler. Latest update new - Agner - 2022-08-08
last replythread MKL performance on AMD with the new compiler new - Gil Moses - 2022-08-22
last reply MKL performance on AMD with the new compiler new - Agner - 2022-08-22