04:34 AM PST
October 21, 2017

Linux™ Test Project Results

The Linux Test Project (LTP) is a joint project with SGI™, IBM®, and OSDL with a goal to deliver test suites to the open source community that validate the reliability, robustness, and stability of Linux. Interested open source contributors are encouraged to join the project. The following results are based on testing completed by the Linux Test Project team.

Initial test efforts focused on commercially available distributions. The LTP is continually evaluating it's test environment and may add additional distributions in the future. Different distributions were used for installation and configuration of the workload environment, however the focus of the test runs was on the Linux kernel. The distributions and releases for the test effort were as follows:

 Debian® 2.2
 Red Hat® 6.2, 7.0, 7.1, 7.2
 SuSE® 7.0, 7.1, 7.2, 7.3
 Turbolinux® 6.5, 7.0

The following kernels were downloaded from kernel.org.

  2.4.3 2.4.8
  2.4.4 2.4.9
  2.4.5 2.4.14
  2.4.6 2.4.16
  2.4.7 2.4.17

Memory utilization varied during the LTP runs and ranged from ~10 - 90% utilization depending on how the stress workloads were configured.

All test results are provided "AS IS" and no warranties or guarantees are expressed or implied. Actual system performance may vary and is dependent on many factors including system hardware configuration and software design and configuration.


LTP Tests
Kernel Test Results
Expected errors for LTP
LTP test on the 2.4.17 Kernel
LTP test on the 2.4.16 Kernel
LTP test on the 2.4.14 Kernel
LTP test on the 2.4.9 Kernel
LTP test on the 2.4.7 Kernel
LTP test on the 2.4.5 Kernel
LTP test on a JFS partition with 2.4.5 Kernel
LTP test with diskio and network tests on the 2.4.5 Kernel
LTP test on the 2.4.4 Kernel
DB2 ® Tests
DB2 tests on the 2.4.4 Kernel
DB2 tests on the 2.4.5 Kernel
DB2 tests on the 2.4.6 Kernel
DB2 tests on the 2.4.7 Kernel
DB2 tests on the 2.4.7 Kernel
DB2 tests on Reiser Filesystem
Websphere Tests
OSDL Results
JFS Tests
JFS on LVM Tests
Next Generation Posix™ Threads Tests
Lab Topology
Current Lab Topology
Forecasted Lab Topology
Test Team

LTP Testing Results
Parameters: runalltests.sh, kern.sh, and network.sh
                   Set runalltests.sh to run 4 concurrent runs on the 4-way system.
                   Set runalltests.sh to run 8 concurrent runs on the 8-way system.
                   Network test requires second machine to connect to.

LTP test running on the 2.4.4 Kernel

Kernel 2.4.4

Red Hat 7.1 with 2.4.4 kernel

SuSE 7.1 with 2.4.4 kernel

Turbolinux 6.5 with 2.4.4 kernel

Hardware UP - P3-866 Mhz, 256MB RAM, 30GB HDD
UP(AMD®) - 1.2 Ghz, 512MB RAM, 30GB HDD
4-way - P3-700 Mhz, 2GB RAM, 108GB HDD
UP - P3-866 Mhz, 256MB RAM, 30GB HDD
4-way P3-700 Mhz, 2GB RAM, 108GB HDD
8-way P3-700 Mhz, 10GB RAM, 300GB HDD
UP - P3-866 Mhz, 256MB RAM, 30GB HDD
4-way - P3-700 Mhz, 2GB RAM, 108GB HDD
Observations UPUP (AMD)4-way UP4-way8-way UP4-way
Start date 05/25/
2001
06/07/
2001
05/25/
2001
06/04/
2001
06/04/
2001
06/08/
2001
06/07/
2001
06/08/
2001
Test duration 96 hrs.96 hrs.96 hrs. 96 hrs.96 hrs.72 hrs. 96 hrs.96 hrs.
SGI success rate 99.22%98.83%99.22% 98.83%98.83%98.83% 99.22%99.22%
Kernel success rate 100%100%100% 100%100%100% 100%100%
Network success rate 95.56%99.34%91.89% 96.67%89.83%n/a 95.55%82.17%
CPU utilization 99.66%99.98%83.62% 99.98%90.58%51.50% 99.98%90.54%
Memory utilization 46.72%54.56%48.02% 47.25%41.92%22.69% 38.08%41.04%
Comments All tests ran as expected. Few problems noticed. 8-way the network test hung during the first iteration.
On 4-way after about 75 hours the openfile kernel test hungup and had to be killed manually for the kernel tests to continue.
The following network commands do not exist in Turbolinux 6.5: rup, rusers, rwho, and rdist.

Network issues with our workloads/lab topology occured with all distributions:
      The network tests hung up a few times. We believe it is a random occurence due to the stress on the system.  When running the network tests alone, we have not seen the problem.  The hangups occur at different times and on different parts of the network test.

LTP test running on the 2.4.5 Kernel

Kernel 2.4.5

Turbolinux 6.5 with 2.4.5 kernel

HardwareUP - P3-866 Mhz, 256MB RAM, 30GB HDD
Observations UP
Start date 06/15/2001
Test duration 96 hrs.
SGI success rate 99.22%
Kernel success rate 100%
Network success rate 95.80%
CPU utilization 99.97%
Memory utilization 38.49%
CommentsAll tests ran as expected.

LTP test running on a JFS partition

Kernel 2.4.5
JFS 0.3.4

Red Hat 7.1 with 2.4.5 kernel

Hardware UP(AMD) - 1.2 Ghz, 512MB RAM, 30GB HDD
Observations UP (AMD)
Start date 06/13/2001
Test duration 118 hrs.
SGI success rate 98.83%
Kernel success rate 100%
Network success rate 92.59%
CPU utilization 99.98%
Memory utilization 53.34%
Comments Problem with network tests: telnet and rlogin tests failed due to ptys creation limit error.  We believe it is due to the way Linux was configured for this box.  It only happens during the running of the test; if you type in the command manually, it works fine. Other tests ran as expected.

LTP test running with diskio and network tests on the 2.4.5 Kernel

Parameters: runalltests.sh, ltctest.sh(mtest01 and mtest01w set to -p70), diskio.sh, and network.sh
                   Set runalltests.sh to run 2 concurrent runs on the 2-way system.
                   Set runalltests.sh to run 4 concurrent runs on the 4-way system.
                   Set runalltests.sh to run 8 concurrent runs on the 8-way system.
                   Network test requires second machine to connect to.
 

Kernel 2.4.5

Red Hat 7.1 with 2.4.5 kernel

SuSE 7.1 with
2.4.5 kernel

SuSE 7.2 with
2.4.5 kernel

TurboLinux 6.5
with 2.4.5 kernel

Hardware UP - P3-866 Mhz, 256MB ram, 30GB HDD
2-way AMD 1.2 Ghz, 512MB ram, 40GB HDD
4-way - P3-700 Mhz, 2GB ram, 108GB HDD
8-way - P3-700 Mhz, 10GB ram, 300GB HDD
UP - P3-866 Mhz, 256MB ram, 30GB HDD
4-way - P3-700 Mhz, 2GB ram, 108GB HDD
UP - P3-866 Mhz, 256MB ram, 30GB HDD UP - P3-866 Mhz, 256MB ram, 30GB HDD
 
Observations UP UP (AMD) 2-way (AMD) 4-way 8-way UP 4-way UP UP
Start date 06/29/
2001
06/29/
2001
07/12/
2001
06/29/
2001
06/29/
2001
06/29/
2001
07/12/
2001
06/20/
2001
06/20/
2001
Test duration 96 hrs. 96 hrs. 96 hrs. 96 hrs. 96 hrs. 96 hrs. 26 hrs. 96 hrs. 118 hrs.
SGI success rate 99.22% 99.22% 99.22% 99.25% * 99.22% 99.22% 99.22%* 99.22% 99.22%
LTC tests success rate 100% 100% 92.67% 97.32% * 100% * 100% 91.09%* 96.89% 100%
Network success rate 88.21% 87.49% 47.33% 90.57% * N/A 88.92% 80.10%* 41.71% 91.18%
Diskio success rate 100% 100% 95.23% 100% * 100% 100% 50%* 50% N/A
CPU utilization 97.23% 99.06% 83.61% N/A * 95.16% 99.67% 76.06% 99.71% 99.22%
Memory utilization 94.70% 84.04% 48.33% N/A * 97.54% 93.61% 82.82% 77.04% 95.14%
Comments The UP boxes ran as expected.

2-way (AMD) - LTCtests and the diskio tests stopped after 2.5 hours, the network tests had a lower percentage due to some services not being turned on during the run.

4-way - the ltctests.sh stopped on the openfile test after 30 minutes and had some errors with the mtest01 and mtests01w; and the diskio.sh and networktests.sh stopped after 2 hours; and the runalltests.sh hung on the sync01 tests.  * None of the tests completed correctly so we do not have a good CPU utilization or Memory utilization.

8-way - the ltctests.sh stopped on the openfile test after 5 minutes; and the diskio.sh and runalltests.sh ran fine; we did not run networktests.sh.
The UP boxes ran as expected.

4-way - All the floppy stress tests failed in the diskio.sh and all 4 scripts stopped running and locked the machine up after 26.5 hours.
All floppy stress tests failed, but we have identified a fix.   The network tests had a lower percentage due to some services not being turned on during the run. Everything else ran fine; all tests ran for 96 hours. The following network commands do not exist in TurboLinux 6.5: rup, rusers, rwho, and rdist.

Everything ran as expected.

LTP test running on the 2.4.7 Kernel

Parameters: runalltests.sh, ltctest.sh(mtest01 and mtest01w set to -p80), diskio.sh, and network.sh
                  Network test requires second machine to connect to.
 
Kernel
2.4.7
RedHat 7.1 with
2.4.7 kernel
SuSE 7.1 with 2.4.7 kernel TurboLinux 6.5 with 2.4.7 kernel
Hardware UP - P3-600 Mhz, 512MB ram,  18GB HDD UP - P3-600 Mhz, 512MB ram, 18GB HDD UP - P3-600 Mhz, 512MB ram,  18GB HDD
Observations UP UP UP
Start date 08/09/2001 08/09/2001 08/09/2001
Test duration 96 hrs. 96 hrs. 96 hrs.
SGI success rate 99.61% 99.61% 99.61%
LTC tests success rate 97.01% 97.27% 94.81%
Network success rate 87.03% 78.33% 48.89%
Diskio success rate 100% 95.44% 99.70
CPU utilization 80.23% 98.81% 82.27%
Memory utilization 82.40% 48.61% 80.96%
Comments The UP box ran as expected. The UP network tests hung after 24hours on the MC_member test.
Everything else ran as expected.
The following network commands do not exist in TurboLinux 6.5: rup, rusers, rwho, and rdist.
UP - Everything ran as expected.

LTP test running on the 2.4.9 Kernel

Parameters: runalltests.sh, ltctest.sh(mtest01 and mtest01w set to -p80), diskio.sh, and network.sh
                  Network test requires second machine to connect to.
 
Kernel
2.4.9
RedHat 7.1 with
2.4.9 kernel
SuSE 7.1 with 2.4.9 kernel TurboLinux 6.5 with 2.4.9 kernel
Hardware UP - P3-600 Mhz, 512MB ram,  18GB HDD UP - P3-600 Mhz, 512MB ram, 18GB HDD UP - P3-600 Mhz, 512MB ram,  18GB HDD
Observations UP UP UP
Start date 08/28/2001 08/28/2001 08/28/2001
Test duration 72 hrs. 72 hrs. 72 hrs.
SGI success rate 99.61% 99.61% 99.61%
LTC tests success rate 97.41% 97.13% 95.44%
Network success rate 83.62% 95% 48.42%
Diskio success rate 100% 95.16% 100%
CPU utilization 69.22% 90.63% 73.91%
Memory utilization 80.92% 41.05% 79.31%
Comments The UP box ran as expected. The UP network tests stopped after 20 minutes.  The diskio test stopped after 4 1/2 hours.
Everything else ran as expected.
The following network commands do not exist in TurboLinux 6.5: rup, rusers, rwho, and rdist.
UP - Everything ran as expected.

LTP test running on the 2.4.9 Kernel

Parameters: New version of LTP running the runalltests.sh, diskio.sh, and network.sh
                  Network test requires second machine to connect to.
 
Kernel
2.4.9
RedHat 7.1 with
2.4.9 kernel
SuSE 7.1 with 2.4.9 kernel TurboLinux 6.5 with 2.4.9 kernel
Hardware UP - P3-866 Mhz, 256MB ram,  30GB HDD UP - P3-866 Mhz, 256MB ram, 30GB HDD UP - P3-866 Mhz, 256MB ram,  30GB HDD
Observations UP UP UP
Start Date 09/20/2001 09/20/2001 09/20/2001
Test duration 96 hrs. 96 hrs. 96 hrs.
Runalltests success rate 99.22% 98.74% 98.56%
Network success rate 77.63% 75.73% 41.24%
Diskio success rate 96.68% 99.10% 98.95%
CPU utilization 71.75% 81.52% 73.95%
Memory utilization 50.95% 59.28% 50.85%
Comments The diskio test stopped after 15 hours.  Everything else ran as expected. Everything ran as expected. The diskio test stopped after 15 hours.  Everything else ran as expected

LTP test running on the 2.4.14 Kernel

Parameters: LTP-20011107 running the runalltests.sh and network.sh
                  Network test requires second machine to connect to.
                  Set runalltests.sh to run 8 concurrent runs on the 8-way system.
 
Kernel
2.4.14
RedHat 7.2 with
2.4.14 kernel
and ext2 FS
SuSE 7.3 with
2.4.14 kernel 
and Reiser FS
Hardware UP - P3-866 Mhz, 256MB RAM,  30GB HDD 8-way - P3-700 Mhz, 10GB RAM, 75GB HDD UP - (AMD)Athlon 1200 Mhz, 512MB RAM, 30GB HDD
Observations UP 8-way UP
Start Date 11/26/2001 11/26/2001 11/26/2001
Test duration 140 hrs. 140 hrs. 140 hrs.
Runalltests success rate 99.12% 98.98% 98.96%
Network success rate 93.01% 93.89% 86.59%
CPU utilization 74.92% 40% 90.85%
Memory utilization 45.73% 89.87% 42.97%
Comments Everything ran as expected. Everything ran as expected.
Commented out the growfiles tests.
Which resulted in a lower CPU utilization than the UP's but a much higher Memory utilization.
Everything ran as expected.

LTP test running on the 2.4.16 Kernel

Parameters: latest version of LTP from cvs as of 11/30/2001 running the runalltests.sh and network.sh
                  Network test requires second machine to connect to.
                  Set runalltests.sh to run 7 concurrent runs on the 8-way system.  Ran the growfiles and shmget tests with only 1 instance.
 
Kernel
2.4.16
RedHat 7.2 with
2.4.16 kernel
and ext2 FS
SuSE 7.3 with
2.4.16 kernel 
and Reiser FS
Hardware UP - P3-866 Mhz, 256MB ram,  30GB HDD 8-way - P3-700 Mhz, 10GB ram, 75GB HDD UP - (AMD)Athlon 1200 Mhz, 512MB ram, 30GB HDD
Observations UP 8-way UP
Start Date 11/30/2001 12/03/2001 01/04/2002 11/30/2001
Test duration 72 hrs. 24 hrs. 72 hrs. 72 hrs.
Runalltests success rate 99.01% 99.09% 99.00% 99.05%
Network success rate 95.74% 96.38% N/A 94.96%
CPU utilization 74.77% 21.40% 56.45% 91.45%
Memory utilization 42.96% 61.78% 90.91% 45.33%
Comments Everything ran as expected. Everything ran as expected.
Ran growfiles and shmget tests in only 1 instance.  And ran the rest of runall in 7 instances.
Everything ran as expected.
Ran growfiles and shmget tests in only 1 instance.  And ran the rest of runall in 7 instances.  Did run the networktests.sh.
Everything ran as expected.

LTP test running on the 2.4.17 Kernel

Parameters: latest version of LTP from cvs as of 01/04/2002 running the runalltests.sh and network.sh
                  Network test requires second machine to connect to.
                  Set runalltests.sh to run 7 concurrent runs on the 8-way system.  Ran the growfiles and shmget tests with only 1 instance.
 
Kernel
2.4.17
RedHat 7.2 with
2.4.17 kernel
and ext2 FS
SuSE 7.3 with
2.4.17 kernel 
and Reiser FS
Hardware UP - P3-866 Mhz, 256MB ram,  30GB HDD UP - (AMD)Athlon 1200 Mhz, 512MB ram, 30GB HDD
Observations UP UP
Start Date 01/04/2002 01/04/2002
Test duration 72 hrs. 72 hrs.
Runalltests success rate 99.22% 99.42%
Network success rate 92.70% 87.21%
CPU utilization 79.13% 94.10%
Memory utilization 38.73% 39.60%
Comments Everything ran as expected. Everything ran as expected.

LTP test running on the 2.4.17 Kernel

Parameters: LTP-20020108 running the runalltests.sh and network.sh
                  Network test requires second machine to connect to.
                  Set runalltests.sh to run 7 concurrent runs on the 8-way system.
 
Kernel
2.4.17
RedHat 7.2 with
2.4.17 kernel
and ext3 FS
 RedHat 7.2 with
2.4.17 kernel
and ext3 FS
RedHat 7.2 with
2.4.17 kernel
and ext2 FS
SuSE 7.3 with 
2.4.17 kernel
and resier FS
TurboLinux 7.0 with
2.4.17 kernel
and ext2 FS
Hardware UP - P3-866 Mhz, 256MB RAM,  30GB HDD UP - (AMD)Athlon 1200 Mhz, 512MB RAM, 30GB HDD 8-way - P3-700 Mhz, 10GB RAM, 75GB HDD UP - (AMD)Athlon 1200 Mhz, 512MB RAM, 30GB HDD UP - P3-866 Mhz, 256MB RAM,  30GB HDD
Observations UP UP 8-way UP UP
Start Date 01/17/2002 01/17/2002 01/17/2002 01/17/2002 01/17/2002
Test duration 96 hrs. 96 hrs. 96 hrs. 96 hrs. 96 hrs.
Runalltests success rate 99.26% 99.25% 98.96% 99.37% 99.38%
Network success rate 91.26% 90.31% 92.53% 46.08% 81.78%
CPU utilization 77.60% 93.86% 53.76% 93.09% 74.81%
USER 56.04% 36.71% 32.34% 34.12% 52.72%
SYSTEM 21.56% 57.15% 21.42% 58.97% 22.09%
Memory utilization 47.04% 38.06% 45.00% 39.42% 36.58%
Comments Everything ran as expected. Everything ran as expected. Everything ran as expected.
Ran 1 iteration of growfiles and 7 iterations of runalltests.sh
Everything ran as expected. Everything else ran as expected

DB2 Testing Results

DB2 UDB EE V7.1 tests running on the 2.4.4 Kernel

Kernel 2.4.4

Red Hat 7.1 with 2.4.4 kernel

Hardware 8-way P3-700 Mhz, 10GB RAM, 300GB HDD
Observations 8-way
Start date 05/24/2001
Test duration 96 hrs.
Number of clients 32
IPCS limits 4096
Number of random executables 1629
CPU utilization 75%
Memory utilization 50%
CommentsThe test ran successfully.

DB2 UDB EE V7.2 tests running on the 2.4.5 Kernel

Kernel 2.4.5

Red Hat 7.1 with 2.4.5 kernel

SuSE 7.1 with 2.4.5 kernel

Turbolinux 6.5 with 2.4.5 kernel

Hardware UP - P3-866 Mhz, 256MB RAM, 30GB HDD
4-way - P3-700 Mhz, 2GB RAM, 108GB HDD
4-way P3-700 Mhz, 2GB RAM, 108GB HDD 4-way - P3-700 Mhz, 2GB RAM, 108GB HDD
Observations UP4-way 2 instances 4-way 4-way
Start date 06/12/200106/14/2001 06/01/2001 06/06/2001
Test duration 98 hrs.96+hrs 96 hrs. 116 hrs.
Number of clients 832 32 32
IPCS limits 5124096 4096 4096
Number of Random Executables 16292326 1629 1629
CPU utilization 95-100%95-100% 95-100% 95-100%
Memory utilization 100%100% 100% 100%
Comments The test ran successfully. The test ran successfully. The test ran successfully.

DB2 UDB EE V7.2 tests running on the 2.4.6 Kernel

Kernel 2.4.6

Red Hat 7.1 with 2.4.6 kernel

SuSE 7.2 with 2.4.6 kernel

Turbolinux 6.5 with 2.4.6 kernel

Hardware 2-way - P3-666 Mhz, 1024M RAM, 8G int, 4x RAID HDD 2-way - P3-666 Mhz, 1024M RAM, 8G int, 4x RAID HDD
8-way P3-700 Mhz, 10GB RAM, 300GB HDD
2-way - P3-666 Mhz, 1024M RAM, 8G int, 4x RAID HDD
Observations 2-way 2-way8-way 2-way
Start date 07/09/2001 07/17/200107/16/01 07/24/2001
Test duration 98 hrs. 96 hrs.96 hrs. 96 hrs.
Number of clients 16 1632 16
IPCS limits 1024 10244096 1024
Number of Random Executables 1629 16291629 1629
CPU utilization 95-100% 95-100%85-100% 95-100%
Memory utilization 100% 100%90-100% 100%
Comments The test ran successfully. The test ran successfully. The test ran successfully.

DB2 UDB EE V7.2 tests running on the 2.4.7 Kernel

Kernel 2.4.7

Turbolinux 6.5 with 2.4.7 kernel

Hardware 8-way P3-700 Mhz, 10GB RAM, 300GB HDD
Observations 8-way
Start date 07/26/01
Test duration 72
Number of clients 32
IPCS limits 4096
Number of random executables 1629
CPU utilization 95-100%
Memory utilization 100%
CommentsThe test ran successfully.

DB2 UDB EE V7.2 tests running on the 2.4.8 Kernel

Kernel 2.4.8

Debian 2.2 with 2.4.8 kernel

Hardware 2-way - P3-666 Mhz, 1024M RAM, 8G int, 4x RAID HDD
Observations 2-way
Start date 08/10/01
Test duration 72
Number of clients 16
IPCS limits 1024
Number of random executables 1629
CPU utilization 75-95%
Memory utilization 55-98%
CommentsThe test ran successfully.

DB2 UDB EE V7.2 tests running on Reiser Filesystem

 

Red Hat 7.1 with 2.4.9 kernel

Red Hat 7.1 with 2.4.7 kernel

Hardware 2-way - P3-666 Mhz, 1024M RAM, 8G int, 4x RAID HDD4-way P3-700 Mhz, 2GB RAM, 108GB HDD
Observations 2-way4-way
Start date 08/24/200108/16/2001
Test duration 96 hrs.96 hrs.
Number of clients 1632
IPCS limits 10244096
Number of Random Executables 16291629
CPU utilization 95-100%75-95%
Memory utilization 100%55-99%
Comments The test ran successfully.The test ran successfully.

OSDL Results

Kernel 2.4.0

Red Hat 6.2 with 2.4.0 kernel

Hardware 8-way - P3-Xeon 700Mhz, 8GB RAM, 36.4GB SCSI Hot-Swappable HDD
Observations 8-Way Netfinity® 8500
Start date 02/23/2001
Test duration 94hrs
SGI success rate 98.06%
Bonnie results 337 iterations on a 2GB file at 100% usage of 1 CPU
CPU utilization 99.97% (averaged across all 8 processors)
Comments The SGI tests were from the most recent drop at the time.  One of the SGI tests (sync01) was omitted from the run because it conflicted with Bonnie.  Memory utilization was not recorded, but was low due to the large amount available (8GB).

Test Team:
Ginny Black, Mgr.
Linda Scott, Lead
Casey Abell
David Barrera
Paul Edgar
Li Ge
Stephanie Glass
Jay Inman
  
Manoj Iyer
James Kenefick
Paul Larson
Jeff Martin
Vernon Mauery
Lisa Sarabia
Robbie Williamson


IBM, DB2, Netfinity, and their logos are registered trademarks of International Business Machines. Linux is a trademark of Linus Torvalds. SGI is a trademark of Silicon Graphics, Inc. POSIX is a trademark of the Institute of Electrical and Electronic Engineers (IEEE). Debian is a registered trademark of Software in the Public Interest. Red Hat and its logo are registered trademarks of Red Hat, Inc. SuSE and its logo are registered trademarks of SuSE AG. Turbolinux and its logo are trademarks of Turbolinux, Inc. AMD and its logo are registered trademarks of Advanced Micro Devices, Inc.

Other company, product, and service names may be trademarks or service marks of others.