The Linux Test Project (LTP) is an open source project with a goal to deliver test suites to the open source community that validate the reliability, robustness, and stability of Linux. The LTP test suite is a collection of automated and semi-automated tests for testing various aspects of the Linux operating system. The goal of LTP is to deliver a suite of automated testing tools for Linux as well as publishing the results of tests we run. LTP invites Internal & External community to contribute to new horizons. For more detailed information about "Linux Test Project"(LTP), visit the following websites http://ltp.sourceforge.net/ & http://sourceforge.net/projects/ltp/, and , IRC on irc.freenode.org #ltp {i}.
This wiki has been mainly designed to get more feedback from LTP users regarding their feel about LTP. Anybody can update this page about various ways by which we can improve this project and also post their willingness to contribute to this project. We as an integrated LTP team will make sure that your contribution(s) become(s) part of LTP test suite with little wastage of time. As with every project we promise to evolve on better lines and seek suggestions from you in this regard. For any issue(s) regarding LTP, kindly contact subrata@linux.vnet.ibm.com.
LTP for the month of APRIL 2008 has been released. You can pick your copy from here
MARCH 2008 release can be found here
FEBRUARY 2008 release can be found here
Task Title |
Task Description |
LTP-DEVEL RPM |
Addition of SPEC File for creation of LTP-DEVEL RPMs on various Architectures for Standalone development of LTP Unit Testcase. Contributed by George Kraft <gk4@us.ibm.com> |
RT Linux Test update |
This patch should solve the problem of RT Linux Compilation Failure on various Architectures. It just avoids the compilation of realtime tests on a non (x86,ppc) arch. Assumption: uname -m doesn't contain 86/ppc on any arch except the x86 and ppc. Signed-Off-By: Chirag <chirag@linux.vnet.ibm.com> |
Se-linux Test suite Update |
Previously I was under the impression that selinux test was not cleaning up properly. I don't believe that to be the case. I must have been mistaken. Here is a patch for the allow_domain_fd_use boolean issue in 5.2, by, Jeff Burke <jburke@redhat.com> |
Se-linux Test suite Update |
This patch, which is independent of Jeff's patch, updates the selinux testsuite to run under Fedora 9, and does no harm on Fedora 8. While creating this, I noticed two other things that ultimately need fixing: 1) The sbin_deprecated.patch adds domain_dyntrans_type() to all the test domains. If that was truly desired, we should just put it into unconfined_runs_test(). But it shouldn't be necessary -only the test_dyntrans.te and test_dyntrace.te domains should require permissions for dynamic transitions. I'll let Serge confirm that. 2) The test scripts are presently relabeling /tmp to test_file_t for the duration of the test. That's insane - it could break any other running process that tries to access /tmp during the test. That was not part of our original selinux testsuite and seems to have been introduced when IBM ported it to LTP. If you are worried about lacking search permission to /tmp in the test domains, then create your own private /test directory or something. Or just give all test domains permission to search tmp either via unconfined_runs_test() or in test_global.te using the testdomain attribute. Those don't need to be fixed though for this patch to be merged. Also, I noticed some new failures in the tests due to latest Fedora policies giving all domains search permission to all directories. Not sure if that was intentional - sent a separate note to Dan about that. |
Se-linux Test suite Update |
Ok here is a first small patch to stop relabeling /tmp as Stephen suggested. It should be no more complicated to get rid of the unneeded dyntrans_types, but I messed up somewhere generating the patch and subsequent test bombed. So I'll just do that next week or whenever this patch hits cvs (for simplicity). Subject: selinux testsuite: don't relabel /tmp. There's no need for the selinux testsuite to relabel /tmp for the duration of the test. It uses /tmp/selinux anyway. Just need to be sure to have search perms to tmp_t. Signed-off-by: Serge Hallyn <serue@us.ibm.com> |
INOTIFY Fix |
dont build things statically, by, Mike Frysinger <vapier@users.sourceforge.net> |
FCNTL Fix |
anal fix: add whitespace between arguments, by, Mike Frysinger <vapier@users.sourceforge.net> |
CONCURRENCY Fix for getipckey() |
This patch remove the useless random function used in getipckey. This random value has been replaced by a static int incremented at each call. This allow to remove the ugly whiles around the getipckey calls when we need several keys, by, Renaud Lottiaux Renaud.Lottiaux@kerlabs.com> |
CONCURRENCY Fix for msgget02() |
This patch fixes a concurrency issue in msgget02. The second key used for this test was sometime conflicting with the key from another task due to a bad way of creating keys. This patch generate a valid second key through getipckey to avoid conflicts. Signed-off-by: Renaud Lottiaux <Renaud.Lottiaux@kerlabs.com> |
CONCURRENCY Fix for msgget03() |
These patch removes a IPC leak in case of test failure. When the msgget function succeed instead of returning ENOSPC, the returned IPC was not released, by, Renaud Lottiaux <Renaud.Lottiaux@kerlabs.com> |
CONCURRENCY Fix for shmctl01() |
This patch fixes a concurrency issue un shmctl01. Synchronization between processes was done using sleeps, which is not enough when multiple instances of the test are running or when we are running on loaded CPUs, by, Renaud Lottiaux <Renaud.Lottiaux@kerlabs.com> |
CONCURRENCY Fix for msgctl04 |
Fix concurrency issue in msgctl04. The second key used for this test could conflict with the key from another task. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for msgctl06 |
Fix concurrency issue in msgctl06. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for msgctl07 |
This patch fix a concurrency issue in msgctl07, by, Renaud Lottiaux <Renaud.Lottiaux@kerlabs.com> |
CONCURRENCY Fix for msgrcv02 |
Fix concurrency issue. The second key used for this test could conflict with the key from another task. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for msgrcv05 |
Fix concurrency issue in msgrcv05. Due to the use of usleep function to synchronize processes, synchronization issues can occur on a loaded system. Fix this by using pipes to synchronize processes. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for msgrcv06 |
Fix concurrency issue in msgrcv06. Due to the use of usleep function to synchronize processes, synchronization issues can occur on a loaded system. Fix this by using pipes to synchronize processes. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for msgsnd05 |
Fix concurrency issue in msgsnd05. Due to the use of usleep function to synchronize processes, synchronization issues can occur on a loaded system. Fix this by using pipes to synchronize processes. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for msgsnd06 |
Fix concurrency issue in msgsnd06. Due to the use of usleep function to synchronize processes, synchronization issues can occur on a loaded system. Fix this by using pipes to synchronize processes. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for semget05 |
Fix concurrency issue in semget05. Create private semaphores to avoid conflict with concurrent processes. Signed-off-by:Matthieu Fertré <mfertre@irisa.fr> |
CONCURRENCY Fix for semop05 |
Fix concurrency issue in semop05. Due to the use of usleep function to synchronize processes, synchronization issues can occur on a loaded system. Fix this by using pipes to synchronize processes. Signed-off-by: Matthieu Fertré <mfertre@irisa.fr> |
networkstress.sh Fix |
This patch reviews the networkstress.sh script and adds a menu to select the network stress tests. Hoping that could be useful, by, Giuseppe Cavallaro <peppe.cavallaro@gmail.com> |
SYSCONF01 Fix |
Updated the test case as per the man page *********** RETURN VALUE If name is invalid, -1 is returned, and errno is set to EINVAL. Otherwise, the value returned is the value of the system resource and errno is not changed. In the case of options, a positive value is returned if a queried option is available, and -1 if it is not. In the case of limits, -1 means that there is no definite limit. *********** Return value of sysconf needs to be checked as well and not only depend on errno value. Have made the changes by looking at the code from http://www.comptechdoc.org/os/linux/programming/c/linux_pgcpostest.html. Max Stirling <vicky.irobot@gmail.com> |
LTP-DEVEL RPM update |
A patch to include all the headers for the ltp-devel package, by, George Kraft <gk4@austin.ibm.com> |
INOTIFY Fix |
Fix from Vivi Li for stack overflows on no-mmu systems: declare large buffers in .bss rather than on the stack |
Se-linux Test suite Update |
Here is the patch to remove the unneeded dyntrans lines. Causes no new failures on my f8 test image. Subject: selinux testsuite: don't give away dyntrans domain_dyntrans_type() only needs to be used for domains actually needing to do dynamic transitions. Don't grant it to most domains. Signed-off-by: Serge Hallyn <serue@us.ibm.com> |
DEFAULT Run Update |
Making these tests run as default with LTP run, by, Subrata Modak <subrata@linux.vnet.ibm.com> |
RT Linux Build Fix |
split CFLAGS/CPPFLAGS properly and remove inappropriate flags, by, Mike Frysinger <vapier@users.sourceforge.net> |
RT Linux Build Fix |
dont stick ar flags into AR, and dont set AR/RANLIB by default, by, Mike Frysinger <vapier@users.sourceforge.net> |
RT Linux Build Fix |
setup default RANLIB, by, Mike Frysinger <vapier@users.sourceforge.net> |
RT Linux Fix |
checks return values of asprintf calls else gcc-4.3.0 fails like following; ... libstats.c:308: error: ignoring return value of 'asprintf', declared with attribute warn_unused_result libstats.c:312: error: ignoring return value of 'asprintf', declared with attribute warn_unused_result ... Signed-off-by: S.Çağlar Onur <caglar@pardus.org.tr>, Acked-by: Chirag <chirag@linux.vnet.ibm.com> |
faccessat01 and fchmodat01 fixes |
faccessat01 and fchmodat01 did not test the AT_FDCWD, following patch makes them test AT_FDCWD and pass. Signed-off-by: S.Çağlar Onur <caglar@pardus.org.tr> |
times02 removal |
According to Ulrich Drepper, times02 test is bogus (at least for archs which report return value and error in the same value) and times has no reserved error code. So following patch removes times02 from testsuite and moves times03 test to times02. For more information please see ttp://sources.redhat.com/ml/libc-alpha/2008-04/msg00054.html and http://sources.redhat.com/ml/libc-alpha/2008-04/msg00055.html. Signed-off-by: S.Çağlar Onur <caglar@pardus.org.tr> |
TCPDUMP01 Fix |
This turned out to be a test case problem, where it fails to identify the correct backbone network interface to use for the test. But there is an option already provided for the interface to be specified while invoking the test. Here, eth1 is the backbone interface and hence if the test is invoked like 'IFNAME=eth1 ./tcpdump01' it will pass. But it relies on DNS lookup and requires the IP to be configured accordingly. Signed-off-by: Anoop Vijayan <anoop.vijayan@in.ibm.com> |
RT Linux Update |
Now that the realtime tests are part of the LTP, there is no need to have any longer specific compilation directives comments in the source files. And in most cases they are incorrect. Signed-off-by: Sebastien Dugue <sebastien.dugue@bull.net>. Cc: Darren Hart <dvhltc@us.ibm.com> |
RT Linux Update |
pi_perf is not built by default, so add it to SUBDIRS in testcases/realtime/Makefile. Signed-off-by: Sebastien Dugue sebastien.dugue@bull.net>. Acked-by: Chirag <chirag@linux.vnet.ibm.com> |
RT Linux Update |
The rttest, stats and jvmsim library archives used by the realtime tests only contain a single object file each. Therefore we can skip the "ar" and "ranlib" steps and directly link with the objects. Signed-off-by: Sebastien Dugue <sebastien.dugue@bull.net>. Acked-by: Chirag <chirag@linux.vnet.ibm.com>. |
RT Linux Update |
periodic_cpu_load has a '-i iterations' command line argument without it being listed in the help. Add its description to the usage() function. Signed-off-by: Sebastien Dugue <sebastien.dugue@bull.net>. Acked-by: Chirag <chirag@linux.vnet.ibm.com>. |
RT Linux Update |
Some tests have a configurable number of iterations. However those same tests which do calculate quantiles need a number of iterations higher than 100. Signed-off-by: Sebastien Dugue <sebastien.dugue@bull.net>. Acked-by: Chirag <chirag@linux.vnet.ibm.com>. |
RT Linux Update |
Remove the last 2 files missed in the migration,namely:testcases/realtime/stress/pi-tests/GNUmakefile.am,testcases/realtime/stress/pi-tests/GNUmakefile.in, Signed-off-by: Sebastien Dugue<sebastien.dugue@bull.net> |
execve05 concurrency fix |
In the expected scenario, it attempts to execve(2) a file which is being opened by another process for writing fails with ETXTBS. On a loaded system, however, the child which does execve might get access to the file before the other child which opens it and result in a unexpected opening fail. By Roy Lee <roylee17@gmail.com> |
SGI Common Criteria CAPP/LSPP EAL4+ certification test suite Release |
|
Release of gcov-kernel patches for Linux 2.6.25 to LTP and LKML |
|
execve05 concurrency fix |
I think there is a problem with the previous patch. Since you call the wait_son_startup() function after the fork() and before the test on the PID variable, the father AND the son will wait on the pipe, leading to a dead-lock. I suggest to move the wait_son_startup() calls after the pid test. Renaud Lottiaux <Renaud.Lottiaux@kerlabs.com>. Changes by Roy Lee <roylee17@gmail.com>. |
Execve02 concurrency fix |
This patch fix a concurrency issue in execve02. In case of concurrent executions, all tasks was using the same file, changing its mode and leading to invalid mode for some of them. This patch creates a private tmp directory for each task, copies the test file in it and performs all the tests using this private file. Regards Renaud Lottiaux <Renaud.Lottiaux@kerlabs.com> |
Ftruncate04 concurrency fix |
This patch fixes a concurrency issue in ftruncate04. In the expected scenario, child should hold the lock of the file before parent access it. On a loaded system, however, the parent might access the file before the child, hence resulting in a unexpected 'open fail'. By, Roy Lee <roylee17@gmail.com>. |
POSIXFADVISE fix |
The following patch and all others that were submitted in Feb, 2008 pertaining to "NR_fadvise64" seem to break ARM builds of ltp as the arm does not have NR_fadvise64 defined. I suspect there should be #ifndef NR_fadvise64 #define NR_fadvise64 0 in each of the files that the patches were added. By, Shane Volpe <shanevolpe@gmail.com> |
SYSFS fix |
The Linux port I run on LTP has a minimal set of system calls. It currently does not implement the sysfs system call so I'd like to disable the tests for this system call - at the moment they will fail to compile because NR_sysfs is undefined. Is there an idiomatic way of doing this? I can either prevent the test compiling completely somehow in the Makefile (e.g. as modify_ldt does) or make the test compile but return BROK perhaps. Attached is a patch to implement a strategy like this one. By Will Newton <will.newton@gmail.com> |
F00f fix |
Executing f00f testcase in x86 Xeon machines, it failed returning SIGSEGV: # ./f00f f00f 0 INFO : Testing for proper f00f instruction handling. Segmentation fault. Searching for this issue, I found an old thread in ltp-list discussing broken test-cases, where Jeff Burke raise this problem with f00f failing on systems that have the NX bit. But, according to the flags reported in /proc/cpuinfo, the machines where i'm running this testcase seems not to have this NX bit enabled. On /proc/cpuinfo, we also have this line "f00f_bug: no". Would it be useful to select if this testcase should be executed or not? These machines are running a kernel compiled with processor family "Pentium-Pro" (CONFIG_M686=y), which, in according to the kernel config help, "disables the init-time guard against the f00f bug found in earlier Pentiums". Here i have created a patch that seems to fix the testcase using an inline assembly. By, Edjunior Barbosa Machado <emachado@linux.vnet.ibm.com> |
Nptl01 fix |
nptl01 looks to have an operator precedence bug in it that prevents the output being done at appropriate times. With this patch you should get 10 lines of output, one every 10000 loops, rather than 10 lines of output, one for each of the first 10 loops. By, Will Newton <will.newton@gmail.com>. |
RT Linux Update |
I believe we had written this test to debug the other test with a lower overhead clocksource. Unfortunately, the tsc is x86 specific, isn't consistent across CPUs on certain systems, and can have it's frequency changed depending on by power states. For all those reasons this test can simply be removed from the LTP. Raied by Daniel Gollub <dgollub@suse.de>, endorsed by Sebastien Dugue <sebastien.dugue@bull.net>, seconded by Darren Hart <dvhltc@us.ibm.com>. |
Sockioctl Update |
This patch removes the dependency of existing tty* devices for this test to pass, and also ups the invalid file descriptor value from 400 to 1025 which is used to confirm the EBADF errno. By, Henry Yei <hyei@mvista.com> |
LTP-LIB-Makefile Update |
Attached is a patch to ltp/lib/Makefile that fixes Solaris cross-building. For some reason the variable CC was overridden with gcc -Wall which caused it to use the system compiler. By Henry Yei <hyei@mvista.com> |
Idcheck.sh Update |
overhaul script to make things much easier to manage by Garrett Cooper <yanegomi@gmail.com> |
fs_perms Update |
simplify targets, by, Mike Frysinger <vapier@users.sourceforge.net> |
fs_perms Update |
cleanup code and add error checking, by, Mike Frysinger <vapier@users.sourceforge.net> |
Filecaps Update |
Attached is a patch to ltp/lib/Makefile that fixes Solaris cross-building. For some reason the variable CC was overridden with gcc -Wall which caused it to use the system compiler. Also fixes cross-building for the filecaps tests. By, Henry Yei <hyei@mvista.com>. |
FLOAT_BESSEL Update |
The float_bessel testcases, as a matter of fact, all float_* testcases seem to fail when multiple instances are run concurrently. The failures occur because file sizes don't match or because the number of bytes read don't match the file size. This can be attributed to the parallel instance reading a file before the write to it (by another instance) has completed. In such situations, either the file size has not been updated in the inode header or the file size has been updated but the file's write operation has not been updated completely. To fix this concurrency problem, my suggestion is to check for an existing instance and wait for it to finish before beginning the current instance. Any other concurrency resolution technique may complicate matters. A message to the console indicating such a decision might be added. By Sridhar Vinay <vinaysridhar@in.ibm.com> & Renaud Lottiaux <Renaud.Lottiaux@kerlabs.com>. |
CPU Controllers Update |
Some Cleanups for CPU Controller Test Cases by Sudhir Kumar. Signed-off-by: Sudhir Kumar <skumar@linux.vnet.ibm.com>, Acked-by: Dhaval Giani <dhaval@linux.vnet.ibm.com>. |
Addition of Initial Set of Memory Controller Test Cases in LTP |
|
Work Archives of LTP for Previous Releases can be found here
Following are the various initiatives been taken within IBM regarding making the LTP testsuite better. We are in touch with more people within IBM and with LTP/Kernel community to integrate more action plan(s) in this regard. By the way, we are able to make a modest begining and chalked out the following plans that we intend to do in the coming months. We appreciate you to add you name inthe following list as an Willing LTP contributor
|
Action Item |
AIM |
Target Completion Date |
Willing Contributors |
Important Points |
|
Investigating LTP Logs Library |
To make LTP Logs more parsible & meaningful and get output/logs in different formats (XML,HTML,etc) |
Nov 2007 |
||
|
Investigate all types of LTP Stress Tests |
To find out whether they are relevant against the latest Distros and Kernel Version(s) testing, and then improve them to the required level |
Jan 2008 |
subrata.modak@in.ibm.com, risrajak@in.ibm.com, maknayak@in.ibm.com, poornima.nayak@in.ibm.com, dkalaker@in.ibm.com, mohd.omar@in.ibm.com, rfolco@br.ibm.com |
IPC_STRESS, Hyperthreading, Process and Pthreads=> "Subrata Modak", Network Stress(TCP,UDP,IPv4)=>"Rishikesh", (TCP,ftp,IPv6)=>Manas, (ICMP)=>"Poornima", File System Stress=>"Mohammed Omar,Deepti", (Scheduler Stress,Crash Stress,NPTL Stress,CD/Floppy Stress,System Call Stress)=>[], (GCOV/LTP, logical coverage)=>"Rafael Folco" |
|
Investigate all new features (new features which are there in the latest kernel) that are not part of LTP testsuite |
To Atleast contribute/add/integrate them in to LTP so that LTPsuite covers functionality testing for all those new features |
CPU Sets by Aug2007, RT Linux by Sep2007 |
subrata.modak@in.ibm.com, rsalveti@br.ibm.com and his team |
Investigators will concentrate on the following points:
Investigate and fix all LTP False Positives
Investigate & understand what the testcase is supposed to do, and increase documentation content/comments on the same testcase,
Make the test case completely full proof to error/exception handling,
See whether more Log library calls(existing) need to be made to communicate more info to the Log Library for reporting errors/exceptions etc,
See whether functionality testing is properly adhered to, if not, how and where to close that GAP !!
Is there any need to change the code to improve the stress testing the testcase is doing ?? or do we need to add a new testcase for doing a separate stress, but, for the same functionality ??
Following Testcases are to be added in the coming Releases
RT(Real Time) Linux Testcases (from sudhanshusingh@in.ibm.com)
CPU Sets Testcases (from balbir@linux.vnet.ibm.com)
NUMA Testcases (from sivakumar.c@in.ibm.com) [This has been successfully Integrated to LTP]
Kindly update this wiki(or mail to subrata@linux.vnet.ibm.com) with any missing new features, and, also features from Kernel 2.6.18 onwards
|
Feature Name |
Added in Kernel Version |
Willing contributor |
|
Execute-in-place support |
2.6.13 |
(You can add your name here) |
|
CPU hotplug for i386 |
do |
|
|
suid_dumpable |
do |
|
|
PCI code rewritten to ignore the BIOS |
do |
|
|
Voluntary preemption |
do |
|
|
HZ can be changed at build time |
do |
|
|
XTensa arch |
do |
|
|
I/O priorities added to the CFQ scheduler |
do |
|
|
kexec |
do |
|
|
runtime selectable class="topAlignContent data" TCP congestion algorithm |
do |
|
|
Lots of TCP congestion modules added=>H-TCP,TCP Hybla,High Speed TCP,TCP Westwood,TCP BIC |
do |
|
|
DRM support for VIA Unichrome |
do |
|
|
DRM 32/64bit ioctl compatibility |
do |
|
|
ACLs for NFSv3 |
do |
|
|
"Trusted computing" driver added |
do |
|
|
Numa aware scheduler |
2.6.14 |
(You can add your name here) |
|
smap file in proc to show true memory usage of processes |
do |
|
|
nmap_maps file in proc |
do |
|
|
lock-free file descriptor lookup, benchmarks are even faster now |
do |
|
|
SELinux memory improvements |
do |
|
|
spinlock cleanups |
do |
|
|
accounting subsystem can now handle files bigger than 2Gb |
do |
|
|
auto-detect kernel soft lockups |
do |
|
|
kzalloc() added |
do |
|
|
ppc64 adds 4 level page support, increasing the ammount of supported memory to 64 Terrabytes |
do |
|
|
32bit ppc gets cpu hotplug support |
do |
|
|
ISA DMA suspend support |
do |
|
|
PPTP protocol added for support with windows machines |
do |
|
|
DCCP protocol added |
do |
|
|
SKB fast clone added |
do |
|
|
netlink connector added |
do |
|
|
Lots of new IPV6 options added |
do |
|
|
FUSE added (userspace filesystems) |
do |
|
|
Plan9 filesystem added |
do |
|
|
NTFS fixes to allow Windows to work better |
do |
|
|
relayfs added |
do |
|
|
securityfs added |
do |
|
|
ext3 and ext2 get security labeling |
do |
|
|
SCSI SAS transport class added |
do |
|
|
removed ACPI S4 BIOS support |
do |
|
|
ipw2100 and ipw2200 (centrino wireless) drivers added |
do |
|
|
HostAP support added |
do |
|
|
radeon r300 3d support added |
do |
|
|
PCI PM CAP v3 support added |
do |
|
|
possessor permission added to key support |
do |
|
|
hdaps driver added |
do |
|
|
apple usb touchpad driver added |
do |
|
|
Shared subtrees support added |
2.6.15 |
(You can add your Name here) |
|
page table class="topAlignContent data" scalability improvements (more benchmark fun) |
do |
|
|
SMP scalability optimizing for large systems |
do |
|
|
reordered files_struct |
do |
|
|
swaptoken tuning |
do |
|
|
demand faulting for huge pages |
do |
|
|
add generic memory add and remove support (memory hotplug!) |
do |
|
|
i/o scheduler improvements |
do |
|
|
process event connector added |
do |
|
|
rcu torture testing kernel module |
do |
|
|
class device structures can now be nested |
do |
|
|
/proc/mounts is now pollable |
do |
|
|
LSM hooks for key management facilities added |
do |
|
|
i386 added cmpxchg function |
do |
|
|
i386 physical hotplug cpu added |
do |
|
|
NTFS write support |
do |
|
|
CIFS update for more interoperability |
do |
|
|
UFO networking feature added |
do |
|
|
randomize the bind port for security protection |
do |
|
|
netfilter support for ipv6 |
do |
|
|
generic netlink |
do |
|
|
PPP MPPE protocol added for connections to windows machines |
do |
|
|
ABC network congestion control support added |
do |
|
|
IPV6 RFC 3484 support added |
do |
|
|
loads of wireless updates |
do |
|
|
SAK processing speedups |
do |
|
|
SATA drivers for SI, Marvel and Promise devices added |
do |
|
|
IDE support for AMD Geode, CS5535 and M3A-2170 devices |
do |
|
|
IDE support for BCM1250 dvices |
do |
|
|
IDE via82cxx driver |
do |
|
|
input wistron driver support added |
do |
|
|
e1000 multi-queue support added |
do |
|
|
sysfs support for ide tape, raid5 and md |
do |
|
|
frame console rotation can be done on the fly |
do |
|
|
Artemis and ATIK USB cameras support added |
do |
|
|
bluetty USB driver deleted |
do |
|
|
nvidia geforce 7800 series frame buffer support added |
do |
|
|
radeon pci express support added |
do |
|
|
matrox mystique AGP support added |
do |
|
|
Rapid I/O subsystem added |
do |
|
|
Omnikey cardman 4040 and 4000 smartcard reader support added |
do |
|
|
13 new *at syscalls |
2.6.16 |
(You can add your Name here) |
|
pselect / ppoll syscalls added |
do |
|
|
unshare syscall added |
do |
|
|
mutex primitive added |
do |
|
|
hi resolution timers added |
do |
|
|
swap migration for NUMA systems added |
do |
|
|
allow all caches to be flushed by a userspace trigger |
do |
|
|
per mountpoint noatime/nodiratime flags |
do |
|
|
EDAC support added |
do |
|
|
Lots of performance fixes |
do |
|
|
dentry structure shrunk |
do |
|
|
struct page structure shrunk |
do |
|
|
tty layer buffering rework |
do |
|
|
gcc 2.9x support dropped |
do |
|
|
swsusp image encryption support removed |
do |
|
|
MD got sysfs support |
do |
|
|
SPI subsystem added |
do |
|
|
CPU topology support added to sysfs |
do |
|
|
AMD Geode GX and LX processor support added |
do |
|
|
sparsemem support for single processor |
do |
|
|
Intel ICH8 support added |
do |
|
|
SPU file system added for cell processors |
do |
|
|
CPU frequency support added for G5 processors |
do |
|
|
thermal control for G5s |
do |
|
|
new G5 processor support added |
do |
|
|
kexec for PPC added |
do |
|
|
OCFS2 cluster filesystem added |
do |
|
|
configfs filesystem added |
do |
|
|
FUSE limits removed |
do |
|
|
NFSv4 fixes |
do |
|
|
NFS large reads and writes added |
do |
|
|
readpage support for v9fs added |
do |
|
|
external journal device for ext3 added |
do |
|
|
direct I/O for FAT fs added |
do |
|
|
relayfs support for global relay buffers added |
do |
|
|
XFS support for SELinux |
do |
|
|
Kerberos and ACL support for CIFS added |
do |
|
|
TIPC network protocol added |
do |
|
|
IFB network device added |
do |
|
|
IPSec LSM hooks added |
do |
|
|
TCP BIC support updated |
do |
|
|
numa aware netfilter changes |
do |
|
|
DCCPv6 protocol added |
do |
|
|
add ability to corrupt network packets randomly |
do |
|
|
ata_piix suspend support added |
do |
|
|
modalias support added for ide devices |
do |
|
|
modalias support added for input devices |
do |
|
|
geyser2 touchpad driver added |
do |
|
|
USB ADSL modems for ADI eagle devices added |
do |
|
|
Lots of USB storage quirks added |
do |
|
|
ATI/Phillips USB RF driver support added |
do |
|
|
Posiflex PP-7000 USB printer driver added |
do |
|
|
ftdi_sio devices added |
do |
|
|
ATIK USB astronomical CCD support added |
do |
|
|
hostap firmware flashing support |
do |
|
|
Marvell Yukon2 Gigabit ethernet support added |
do |
|
|
vlan acceleration support added to sky2 driver |
do |
|
|
ipx2000 Intel network driver added |
do |
|
|
power management support for lots of sound devices added |
do |
|
|
Lots of video for linux driver updates |
do |
|
|
splice, tee, and vmsplice syscall added |
2.6.17 |
(You can add your Name here) |
|
sync_file_range syscall added |
do |
|
|
new schedular domain for multi-core processors added |
do |
|
|
lightweight robust futexes added |
do |
|
|
block queue io tracing support added |
do |
|
|
userspace interface to swsusp added |
do |
|
|
/proc/slab_allocators added |
do |
|
|
mountstats information added to procfs |
do |
|
|
mprotect now enabled for huge pages |
do |
|
|
EXPORT_SYMBOL_GPL_FUTURE added |
do |
|
|
CFQ performance tuning |
do |
|
|
FMODE_EXEC file flag support added |
do |
|
|
improved module version support for external modules added |
do |
|
|
SMP alternatices for i386 added to patch instructions in the kernel on the fly |
do |
|
|
CONFIG_REGPARM enabled by the default |
do |
|
|
1Gb process stack randomiziation added (used to be 8Mb) |
do |
|
|
make isoimage support added |
do |
|
|
memory hotadd without sparsemem added |
do |
|
|
lots of Cell processor updates |
do |
|
|
ext3 performance improvements |
do |
|
|
xfs tweaks |
do |
|
|
jfs mount options added |
do |
|
|
ext2 attributes added to jfs |
do |
|
|
jfs support for splice added |
do |
|
|
FUSE O_ASYNC and O_NONBLOCK support added |
do |
|
|
NFS I/O performance counters added |
do |
|
|
NFS client metrics added |
do |
|
|
RPC I/O stats added |
do |
|
|
relayfs support made generic |
do |
|
|
debugs blob support added |
do |
|
|
sysfs attributes are now pollable |
do |
|
|
syscall audit records added to SELinux |
do |
|
|
RFC 4191 IPv6 support added |
do |
|
|
DCCP sysctls added |
do |
|
|
softmac wireless driver layer added |
do |
|
|
lots of new wireless drivers added (broadcom included) |
do |
|
|
PCI legacy proc support removed |
do |
|
|
IPMI driver model support added |
do |
|
|
new device ids and drivers for video added |
do |
|
|
big libata update with new devices and fixes |
do |
|
|
SCSI cache settings added to sysfs |
do |
|
|
braille device support for all input devices added |
do |
|
|
SNES mouse support added |
do |
|
|
unified the USB touchscreen driver for all touchscreens |
do |
|
|
loads of new USB device support added |
do |
|
|
Huge network driver updates |
do |
|
|
large sound driver updates |
do |
|
|
acpi dock support added |
do |
|
|
i2c support for new controllers added |
do |
|
|
LED class support added, along with a lot of diferent LED drivers |
do |
|
|
Secure digital driver support added |
do |
|
|
Niagara multicore CPU processor support added |
do |
|
|
Feedback |
User |
Your Comment |
|
(1) Use the same coding style for every tests because i saw many tests that is coded in different ways. We could use the linux kernel Coding Style to make the tests(with this itll be easier to maintain the LTP code). (2) Work more with the linux kernel regression bugs. The community general report all regression that was found at every kernel release. You can see the regression list at < http://kernelnewbies.org/known_regressions . We have more detailed stuff at the kernel bugzilla. It'd be nice to have regression test cases, to be sure that these bugs are not in kernel anymore. (3) Work closely with the kernel community, making tests as soon we have new features (this is because the kernel community uses the ABAT + LTP to test every kernel release, at < http://test.kernel.org/functional/index.html we have the results). For example, at the newest kernel release, 2.6.22-rc1, we have a new system call, named eventfd (< http://lwn.net/Articles/234123/rss), and it'd be nice to have a test case that could test this system call. (4) Result graphics, using tools such as gnuplot. It'd be nice to have all results in an output that could be used with gnuplot to make some result graphics. With this we could see all pass/failed tests thought the time, watching the results with different kernels. (5) Improve LTP Website |
||
|
The biggest pain of using LTP is the number of false positives it generates. There are so many false positives that we need to keep a list of the "known errors" in order to decide whether any real error messages have popped out. Fixing bugs is exactly where the focus should remain until it becomes the norm for LTP to declare "pass" for a run where there have been no true failures.Here's the list of LTP errors that we generally ignore as false positives. (1)fcntl17 attempt to signal child failed, (2)fsync took too long, (3)gettimeofday is going backwards, (4)gf18, (5)mlockall* (2944?), (6)msg* call failed - erro=28 (note: after you get error 28 or 'device out of space, you'll get tons and tons of msg* error msgs that are safe to ignore. I think this tends to happen only when swap is less than 2*ram, and perhaps only on RHEL), (7)nanosleep remaining time doesn't match foo (17215?) (8)pselect sleep time was incorrect, (9)syslog* failed to log msgs of all levels, (10)capset02 BROK Unexpected signal 15 received(matrix issue 1234 - if you see this, please mark as PASS, but attach the issue to help establish trends), (11)getsid02 (25542), (12)ioperm* (21070, and 21619), (13)setpgid* expect EACCES got 1 (21134 mention, ltp SF bug 1114033), (14)shmctl01 FAIL : # of attaches is incorrect, (15)shmget02 FAIL: call succeeded unexpectedly (21896), (16)socket* (21065), (17)syslogd no such command (21068 - diff failure than the historical), (18)vmsplice01 1 FAIL : vmsplice() Failed, errno=38 (bz28685) |
||
|
I understand, most of the tests generates two kinds of results file. Generally, we will look throu the .log file & if any failures, go through .out file to find out cause for failure & rerun individual test, if required to see it passes or not. I haven't come across such tests which says test failed in log file & end result showing all passed. During those initial days(2000-01), LTP used to yield many bugs. Now a days not much bugs through LTP could mean kernel is getting stronger day by day & hence very hard to find regression defects. Also it is understood that the kernel developers will test their patch for functionality to get ack or nack. LTP is meant for both functional & regression tests, it may not keep track of developers patch whether it has fixed particular issue or not, but if any regressions out of that patch can be found through introduction of new bug. There is a scope for improvement in this area, filter out logs, collect in precise format. As we own LTP maintainership, our job is to make sure each LTP release has latest good patches from the community. And definitely we can contribute on the improvement other areas of LTP (like improving log files, etc), but ultimately it is also the responsibility of kernel developers across the community to contribute to the addition new effective test cases to yield more no. of regression defects to get the glory back. A code coverage analysis on LTP could help us in finding out where the bugs are lacking. |
||
|
(1)The networktests and ltpstress workloads use rsh, rcp, and rlogin. These out of date tools should be replaced with more current tools. A good replacement candidate would be ssh based tools. (2)I use ltprun to initiate the LTP base workload. This allows me to get a summary of the test status from the system that we use as a master by running ltp_check. There is no equivalent to ltprun or ltp_check available for the other LTP workloads. It would be nice to have some sort of tool to obtain a summary when running the other workloads. (3)It would be nice to have tests for CIFS/Samba in addition to ftp and nfs. (4)I would like to see long running stress tests for the various network file systems in addition to the test provided for ftp and nfs. Also a more diverse set of sample files for the test cases to operate on |
||
|
I found that some of the system calls like bdflush, clock_getres, clock_gettime, epoll_create, getdents64, get_mempolicy, mmap2, pivot_root, remap_file_pages, rt_sigaction, endfile64, sched_getaffinity, set_thread_area, set_tid_address, stat64, statfs64, tgkill, waitid, add_key... are not tested. |
||
|
One of the nasty things about LTP, is I don't think we pick up any of the test failures and cascade them through to the final test run status. That makes it harder to tell if there's any regression or not, but I guess it's doable by hand (Picked up from his comment in autotest@test.kernel.org) |