Linux Test Project - Wiki

The Linux Test Project (LTP) is an open source project with a goal to deliver test suites to the open source community that validate the reliability, robustness, and stability of Linux. The LTP test suite is a collection of automated and semi-automated tests for testing various aspects of the Linux operating system. The goal of LTP is to deliver a suite of automated testing tools for Linux as well as publishing the results of tests we run. LTP invites Internal & External community to contribute to new horizons. For more detailed information about "Linux Test Project"(LTP), visit the following websites http://ltp.sourceforge.net/ and http://sourceforge.net/projects/ltp/, and , IRC on irc.freenode.org #ltp .

This wiki has been mainly designed to get more feedback from LTP users regarding their feel about LTP. Anybody can update this page about various ways by which we can improve this project and also post their willingness to contribute to this project. We as an integrated LTP team will make sure that your contribution(s) become(s) part of LTP test suite with little wastage of time . As with every project we promise to evolve on better lines and seek suggestions from you in this regard. For any issue(s) regarding LTP, kindly contact shubham@linux.vnet.ibm.com.

Latest News on LTP

LTP for the month of JANUARY 2012 has been released. You can pick your copy from here

LTP Changes during JANUARY 2009

ChangeLog	JANUARY 2010 Changelog

Work Archives of LTP for Previous Releases can be found here

Upcoming LTP works

Recently we have embarked on a Major Collaboration initiative, where we would be porting major chunk of existing System Call test cases from the Crackerjack Project to Linux Test Project. Leading this initiative is Masatake Yamato<yamato@redhat.com> from Red Hat. He has already ported some of the test cases to LTP. The following table will depict the status of our work in this regard, which we plan to finish by 2009 end. We would like to welcome other volunteers, who can help us in this porting effort. We appreciate you to add you name inthe following list as an Willing LTP contributor .

CRACKERJACK to LTP Porting status

62 of the syscall tests require direct porting and creation of new directories in LTP. Pink Background test cases will require immediate attention and may be taken up with priority. However, you may decide to do as per your comfort,
Once this is over, 29 of testcases needs to be ported for their 16-bit or 64-bit versions. This will not involve creation of new directories in LTP, but will be placed in the existing directories. These may either add new source files or update the existing ones similar to the sendfile64 and fadvise64,
Remaining test cases like shmget, shmat, msgsnd, shmctl & shmdt can be investigated later, and once all the above i) and ii) are done. We can see whether there is something additional functionality testing in Crackerjack for them. If yes, we can then include 1 or 2 tests for each of them. But that is at the end.

Points to Follow for writing NEW SYSCALL tests(Masatake Yamato)

Here is a general rule for writing test cases for newer system calls. If you find something wrong please modify.

(1) If glibc provides interface, we SHOULD write a test case on it.

We can test kernel code through glibc.

(2) If we have more time, we will add a test case on header files

provided by kernel. So we can test the system call even if glibc has not provided a header file for it yet.

(3) If no header file neither sys/foo.h and linux/foo.h, We SHOULD

add a fallback to report it like:

testcases/kernel/syscalls/inotify/inotify01.c int main() {
#ifndef NR_inotify_init
- tst_resm(TWARN, "This test needs a kernel that has inotify syscall."); tst_resm(TWARN, "Inotify syscall can be found at kernel 2.6.13 or higher.");
#endif #ifndef HAS_SYS_INOTIFY:
- tst_resm(TBROK, "can't find header sys/inotify.h"); return 1;
#endif
- return 0;
} #endif In the report, the test case should tell from which version the system call may be available.

I'm not sure I should check NR_inotify_init and HAS_SYS_INOTIFY. But for a while I'd like to ignore these details.

One thing we have to decide now is which test status I should use when the header files is not found. inotify01.c uses TBROK. signalfd uses TCONF like:

int main(int argc, char** argv) {
- tst_resm(TCONF,
  - "System doesn't support execution of the test");
  return 0;
}

Some Points to Follow during Porting (Masatake Yamato)

See Makefile and *.h file of setgid in LTP to work on 16bit related system calls. That may help you.
As Subrata wrote, See Makefile of fadvise64 in LTP to work on 64bit related system calls.
It seems that test cases for foo16, foo and foo64 prints the result in the same format like: # ./posix_fadvise03
- posix_fadvise03 1 PASS : expected failure - returned value = 22, advise = 6 : Invalid argument posix_fadvise03 2 PASS : expected failure - returned value = 22, advise = 7 : Invalid argument posix_fadvise03 3 PASS : expected failure - returned value = 22, advise = 8 : Invalid argument
# ./posix_fadvise03_64
- posix_fadvise03 1 PASS : expected failure - returned value = 22, advise = 6 : Invalid argument posix_fadvise03 2 PASS : expected failure - returned value = 22, advise = 7 : Invalid argument posix_fadvise03 3 PASS : expected failure - returned value = 22, advise = 8 : Invalid argument

I think this should be something like:

# ./posix_fadvise03
- posix_fadvise03 1 PASS : expected failure - returned value = 22, advise = 6 : Invalid argument posix_fadvise03 2 PASS : expected failure - returned value = 22, advise = 7 : Invalid argument posix_fadvise03 3 PASS : expected failure - returned value = 22, advise = 8 : Invalid argument
# ./posix_fadvise03_64
- posix_fadvise03_64 1 PASS : expected failure - returned value = 22, advise = 6 : Invalid argument posix_fadvise03_64 2 PASS : expected failure - returned value = 22, advise = 7 : Invalid argument posix_fadvise03_64 3 PASS : expected failure - returned value = 22, advise = 8 : Invalid argument
Some test cases crackerjack project don't have copyright notice. So we cannot port. I will have a chance to have a presentation in front of crackerjack at Beijing next month. I'd like to explain this copyright issue, which makes the porting impossible.
Some test cases in crackerjack project use non-English in their comment text. Generally I'd like to port them as is. However, non-English comment text makes it hard. I'd like to explain this at Beijing, too. We can ignore the comment but I don't want.
Some test cases in crackerjack use LTP API compatible layer. These test cases may be easy to be ported. I'll work on these kind of test cases.

No.	System Call	Comment on its position w.r.t LTP	To-be-Ported-by	Status
1	ftruncate64	We have ftruncate at testcases/kernel/syscalls/ftruncate/. But this may not be doing the ftruncate64() test. You may need to include/port ftruncate64 on similar lines to sendfile64 and fadvise64	Masatake YAMATO <yamato@redhat.com>	Completed on 05/08/2008
2	setgid16	We have setgid at testcases/kernel/syscalls/setgid. You may need to include/port setgid16 on similar lines to sendfile64 and fadvise64.	Masatake YAMATO <yamato@redhat.com>	Completed on 04/08/2008
3	shmget	We have this at testcases/kernel/syscalls/ipc/shmget. Need not look into this right now. May be after the porting is over, we can see whether there is some additional functionality that Crackerjack shmget is covering over LTP-shmget. We may then need to add 1/2 test cases in this regard. But, not to be prioritized immediately.	Need Volunteer	N/A
4	mbind	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
5	removexattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
6	exit_group	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
7	io_cancel	We need this from Crackerjack as we do not have this in LTP anywhere	Masatake YAMATO<yamato@redhat.com>	Completed on 26/06/2008
8	migrate_pages	We have a different form of this at recently added Memory Hotplug testcases at testcases/kernel/hotplug/memory_hotplug/. But i would like to see this tested inside testcases/kernel/syscalls, as hotplug testing presently is optional. So, please include/port this as well	Need Volunteer	N/A
9	setgroups16	We have setgroups at testcases/kernel/syscalls/setgroups. You may need to include/port setgroups16 on similar lines to sendfile64 and fadvise64	Masatake YAMATO <yamato@redhat.com>	Completed on 27/08/2008
10	getdents64	We have getdents at testcases/kernel/syscalls/getdents. You may need to include/port getdents64 on similar lines to sendfile64 and fadvise64	Rishikesh K Rajak<risrajak@linux.vnet.ibm.com>	Completed on 07/11/2008
11	add_key	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
12	getegid16	We have getegid at testcases/kernel/syscalls/getegid. You may need to include/port getegid16 on similar lines to sendfile64 and fadvise64	Masatake YAMATO <yamato@redhat.com>	Completed on 04/09/2008
13	io_destroy	We need this from Crackerjack as we do not have this in LTP anywhere	Masatake YAMATO <yamato@redhat.com>	Completed on 12/07/2008
14	newfstat	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
15	set_mempolicy	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
16	timer_getoverrun	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
17	fadvise64_64	We have fadvise64 support added by Masatake Yamato. But i am not sure what this fadvise64_64 is. May need to include/port this as well	Already Taken care of by Masatake yamato	Completed
18	io_getevents	We need this from Crackerjack as we do not have this in LTP anywhere	Masatake YAMATO <yamato@redhat.com>	Completed on 12/07/2008
19	newlstat	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
20	rt_sigaction	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
21	sigreturn	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
22	timer_gettime	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
23	geteuid16	We have geteuid at testcases/kernel/syscalls/geteuid. You may need to include/port geteuid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
24	newuname	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
25	rt_sigprocmask	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
26	bdflush	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
27	getgid16	We have getgid at testcases/kernel/syscalls/getgid. You may need to include/port getgid16 on similar lines to sendfile64 and fadvise64	Masatake YAMATO <yamato@redhat.com>	Completed on 19/09/2008
28	rt_sigqueueinfo	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
29	ioprio_get	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
30	rt_sigsuspend	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
31	setregid16	We have setregid at testcases/kernel/syscalls/setregid. You may need to include/port setregid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Subrata Modak<subrata@linux.vnet.ibm.com>
32	tkill	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
33	getgroups16	We have getgroups at testcases/kernel/syscalls/getgroups. You may need to include/port getgroups16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Subrata Modak<subrata@linux.vnet.ibm.com>
34	ioprio_set	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
35	sched_getaffinity	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
36	ssetmask	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
37	fchown16	We have fchown at testcases/kernel/syscalls/fchown. You may need to include/port fchown16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
38	io_setup	We need this from Crackerjack as we do not have this in LTP anywhere	Masatake YAMATO <yamato@redhat.com>	Completed on 12/07/2008
39	mmap2	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
40	setresgid16	We have setresgid at testcases/kernel/syscalls/setresgid. You may need to include/port setresgid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
41	truncate64	We have truncate at testcases/kernel/syscalls/truncate. You may need to include/port truncate64 on similar lines to sendfile64 and fadvise64	Masatake YAMATO <yamato@redhat.com>	Completed on 05/08/2008
42	get_mempolicy	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
43	io_submit	We need this from Crackerjack as we do not have this in LTP anywhere	Masatake YAMATO <yamato@redhat.com>	Completed on 12/07/2008
44	stat64	We have stat at testcases/kernel/syscalls/stat. You may need to include/port stat64 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
45	keyctl	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
46	setresuid16	We have setresuid at testcases/kernel/syscalls/setresuid. You may need to include/port setresuid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
47	chown16	We have chown at testcases/kernel/syscalls/chown. You may need to include/port chown16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
48	fcntl64	We have fcntl at testcases/kernel/syscalls/fcntl. You may need to include/port fcntl64 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
49	move_pages	But work for this has been already initiated. Need not consider this	Vijay Kumar B. <vijaykumar@bravegnu.org>	Completed on 26/07/2008. This was not ported, but written by him
50	statfs64	We have statfs at testcases/kernel/syscalls/statfs. You may need to include/port statfs64 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
51	clock_getres	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
52	lchown16	We have lchown at testcases/kernel/syscalls/lchown. You may need to include/port lchown16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Subrata Modak<subrata@linux.vnet.ibm.com>
53	setreuid16	We have setreuid at testcases/kernel/syscalls/setreuid. You may need to include/port setreuid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
54	fgetxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
55	lgetxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
56	mq_getsetattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
57	unshare	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
58	clock_nanosleep	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
59	flistxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
60	mq_notify	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
61	ppoll	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
62	useclock_settime	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
63	mq_open	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
64	set_thread_area	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
65	listxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
66	mq_timedreceive	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
67	pread64	We have pread at testcases/kernel/syscalls/pread. You may need to include/port pread64 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
68	set_tid_address	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
69	fremovexattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
70	llistxatt	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
71	mq_timedsend	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
72	pselect64	We have pselect at testcases/kernel/syscalls/pselect. You may need to include/port pselect64 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
73	utimes	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
74	fsetxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
75	mq_unlink	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
76	gettid	We need this from Crackerjack as we do not have this in LTP anywhere	Masatake YAMATO <yamato@redhat.com>	Completed on 17/06/2008
77	lremovexattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
78	quotactl	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
79	setuid16	We have setuid at testcases/kernel/syscalls/setuid and testcases/kernel/syscalls/mount. You may need to include/port setuid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
80	fstat64	We have fstat at testcases/kernel/syscalls/fstat. You may need to include/port fstat64 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
81	setxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
82	fstatat64	We have fstatat at testcases/kernel/syscalls/fstatat. You may need to include/port fstatat64 on similar lines to ssendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
83	lsetxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
84	readahead	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
85	setfsgid16	We have setfsgid at testcases/kernel/syscalls/setfsgid. You may need to include/port setfsgid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
86	sgetmask	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
87	waitid	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
88	fstatfs64	We have fstatfs at testcases/kernel/syscalls/fstatfs. You may need to include/port fstatfs64 on similar lines to ssendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
89	getuid16	We have getuid at testcases/kernel/syscalls/getuid. You may need to include/port getuid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
90	getxattr	We need this from Crackerjack as we do not have this in LTP anywhere	Need Volunteer	N/A
91	lstat64	We have lstat at testcases/kernel/syscalls/lstat. You may need to include/port lstat64 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008
92	setfsuid16	We have setfsuid at testcases/kernel/syscalls/setfsuid. You may need to include/port setfsuid16 on similar lines to sendfile64 and fadvise64	Subrata Modak<subrata@linux.vnet.ibm.com>	Completed on 07/11/2008

Investigators may/need also to concentrate on the following points:

Investigate and fix all LTP False Positives
Investigate & understand what the testcase is supposed to do, and increase documentation content/comments on the same testcase,
Make the test case completely full proof to error/exception handling,
See whether more Log library calls(existing) need to be made to communicate more info to the Log Library for reporting errors/exceptions etc,
See whether functionality testing is properly adhered to, if not, how and where to close that GAP !!
Is there any need to change the code to improve the stress testing the testcase is doing ?? or do we need to add a new testcase for doing a separate stress, but, for the same functionality ??

Following Testcases are already part of LTP from '''1st Jan 2009''' till date

MESSAGE QUEUE NAMESPACE tests,
IPC NAMESPACE tests,
BLOCK Device Drivers tests,
Direct IO tests,
socket02() & socket03() syscall tests,
signalfd4_01() & signalfd4_02() syscall tests,
eventfd2_01() & eventfd2_02() syscall tests,
timerfd02() & timerfd03() syscall tests,
epoll_create2_01() syscall test,
dup3_01() syscall test,
pipe2_01() & pipe2_02() syscall tests,
inotify_init1_01() & inotify_init1_02() syscall tests,
socketpair02() syscall test,

QUOTA REMOUNT tests,

FREEZER CONTROLLER tests,

CPU CONSOLIDATION of POWER MANAGEMENT tests,

CONTROLLER AREA NETWORK tests,

LOAD ALL MODULES test,

SMACK SECURITY tests,

INTEGRETY MEASUREMENT ARCHITECTURE SECURITY tests tests,

Benchmark Test to LTP to compare Per_Cpu_Atomic_Operations vs Interrupt_Disabling tests,

Module to test Per CPU Allocator tests,

PERFORMANCE COUNTER in Linux tests,

Checking mmap() corruption tests,

splice02 test for splice() syscall

clock_nanosleep2_01 test for clock_nanosleep2() syscall

cacheflush01 test for cacheflush() syscall

sched_smt_power_saving interface for Power Management

CGROUP Regression Tests

CPUSET Tests

CAPBOUND Security Tests

Additional CPUSET Tests

Memory Control Group Tests

add_key01 test for add_key() syscall

add_key02 test for add_key() syscall

bdflush01 test for bdflush() syscall

exit_group01 test for exit_group() syscall

keyctl01 test for keyctl() syscall

newuname01 test for newuname() syscall

rt_sigaction01 test for rt_sigaction() syscall

rt_sigprocmask01 test for rt_sigprocmask() syscall

rt_sigprocmask02 test for rt_sigprocmask() syscall

rt_sigqueueinfo01 test for rt_sigqueueinfo() syscall

rt_sigsuspend01 test for rt_sigsuspend() syscall

set_thread_area01 test for set_thread_area() syscall

set_thread_area02 test for set_thread_area() syscall

set_tid_address01 test for set_tid_address() syscall

sgetmask01 test for sgetmask() syscall

sigreturn01 test for sigreturn() syscall

ssetmask01 test for ssetmask() syscall

timer_getoverrun01 test for timer_getoverrun() syscall

timer_gettime01 test for timer_gettime() syscall

tkill01 test for tkill() syscall

tkill02 test for tkill() syscall

get_mempolicy01 test for get_mempolicy() syscall

clock_getres01 test for clock_getres() syscall

clock_nanosleep01 test for clock_nanosleep() syscall

mq_notify01 test for mq_notify() syscall

ppoll01 test for ppoll() syscall

mq_open01 test for mq_open() syscall

mq_timedreceive01 test for mq_timedreceive() syscall

utimes01 test for utimes() syscall

mq_unlink01 test for mq_unlink() syscall

mq_timedsend01 test for mq_timedsend() syscall

unshare01 test for unshare() syscall

unshare02 test for unshare() syscall

eventfd2_03 test for eventfd2() syscall

mbind01 test for mbind() syscall

sched_getaffinity01 test for sched_getaffinity() syscall

waitid01 test for waitid() syscall

waitid02 test for waitid() syscall

quotactl01 test for quotactl() syscall

New Testcases for Coming Releases (and hopefully to be added in 2009)

Following Testcases are to be added in the coming Releases

Additional CPU & MEM Controllers Testcases
kdump on Xen hypervisor and guests, makedumpfile, mkdumprd, kexec, netdump, diskdump & lkcdm test scripts,
Union FS & Union Mounts tests,
RT Linux Graphics Stress tests,
GRU driver tests from SGI,

Following Testcases are already part of LTP from '''1st Jan 2008''' till '''31st Dec 2008'''

fallocate() syscall tests,
Filecaps tests,
CPU & Memory Controllers tests,
Message Queue (msgctl) test,
TIRPC (background Protocol for nfsv4) tests,
File System Bind Mount (Shared Subtree) tests,
timerfd() syscall tests,
CPU and Memory Hotplug tests,
utimensat() syscall tests,
gettid() syscall tests,
Process Event Connectors tests,
io_cancel() syscall tests,
hackbench test,
io_destroy() syscall test,
io_getevents() syscall test,
io_setup() syscall test,
io_submit() syscall test,
IO Controllers tests,
move_pages() syscall test,
setgid16() syscall test,
ftruncate64() syscall test,
truncate64() syscall test,
RO Bind Mount tests,
eventfd() syscall tests,
get_robust_list() syscall tests,
set_robust_list() syscall tests,
sync_file_range() syscall tests,
setfsuid04 test for syscall setfsuid(),
setresuid04 test for syscall setresuid(),
setreuid07 test for syscall setreuid(),
setuid04 test for syscall setuid(),
EVENTFD COUNTER OVERFLOW tests,
getcpu() syscall tests,
setgroups16() syscall tests,
getegid16() syscall tests,
switch() syscall tests,
getgid16() syscall tests,
Network Namespace tests,
signalfd() syscall tests,
SELinux Bounds tests,
Power Management tests,
GCOV-KDUMP tool,
getdents64() syscall tests,
geteuid16() syscall tests,
setregid16() syscall tests,
getgroups16() syscall tests,
fchown16() syscall tests,
setresgid16() syscall tests,
stat64() syscall tests,
setresuid16() syscall tests,
chown16() syscall tests,
fcntl64() syscall tests,
statfs64() syscall tests,
lchown16() syscall tests,
setreuid16() syscall tests,
pread64() syscall tests,
pselect64() syscall tests,
setuid16() syscall tests,
fstat64() syscall tests,
fstatat64() syscall tests,
setfsgid16() syscall tests,
fstatfs64() syscall tests,
getuid16() syscall tests,
lstat64() syscall tests,
setfsuid16() syscall tests,
accept4() syscall tests,
pidns04, pidns05, pidns06, pidns10, pidns12, pidns13, pidns14, pidns16, pidns17, pidns20, pidns21, pidns30, pidns31 PID Name Space tests,
ioctl03() syscall test,
inotify03() syscall test,
Video for Linux Two (V4L2) API Device Driver tests,
CPU Controller Latency tests,

If you feel you can add more points here, please feel free to do that.

New Features for which Testcases has to be written

Kindly update this wiki(or mail to shubham@linux.vnet.ibm.com) with any missing new features, and, also features from Kernel 2.6.18 onwards

Feature Name	Added in Kernel Version	Willing contributor
Execute-in-place support	2.6.13	(You can add your name here)
CPU hotplug for i386	do	We recently added this to LTP
suid_dumpable	do
PCI code rewritten to ignore the BIOS	do
Voluntary preemption	do
HZ can be changed at build time	do
XTensa arch	do
I/O priorities added to the CFQ scheduler	do
kexec	do
runtime selectable TCP congestion algorithm	do
Lots of TCP congestion modules added=>H-TCP,TCP Hybla,High Speed TCP,TCP Westwood,TCP BIC	do
DRM support for VIA Unichrome	do
DRM 32/64bit ioctl compatibility	do
ACLs for NFSv3	do
"Trusted computing" driver added	do
Numa aware scheduler	2.6.14	(You can add your name here)
smap file in proc to show true memory usage of processes	do
nmap_maps file in proc	do
lock-free file descriptor lookup, benchmarks are even faster now	do
SELinux memory improvements	do
spinlock cleanups	do
accounting subsystem can now handle files bigger than 2Gb	do
auto-detect kernel soft lockups	do
kzalloc() added	do
ppc64 adds 4 level page support, increasing the ammount of supported memory to 64 Terrabytes	do
32bit ppc gets cpu hotplug support	do
ISA DMA suspend support	do
PPTP protocol added for support with windows machines	do
DCCP protocol added	do
SKB fast clone added	do
netlink connector added	do
Lots of new IPV6 options added	do
FUSE added (userspace filesystems)	do
Plan9 filesystem added	do
NTFS fixes to allow Windows to work better	do
relayfs added	do	This is in our Plan. We will add them shortly
securityfs added	do
ext3 and ext2 get security labeling	do
SCSI SAS transport class added	do
removed ACPI S4 BIOS support	do
ipw2100 and ipw2200 (centrino wireless) drivers added	do
HostAP support added	do
radeon r300 3d support added	do
PCI PM CAP v3 support added	do
possessor permission added to key support	do
hdaps driver added	do
apple usb touchpad driver added	do
Shared subtrees support added	2.6.15	(This is successfully addedd)
page table scalability improvements (more benchmark fun)	do
SMP scalability optimizing for large systems	do
reordered files_struct	do
swaptoken tuning	do
demand faulting for huge pages	do
add generic memory add and remove support (memory hotplug!)	do
i/o scheduler improvements	do
process event connector added	do	Recently added to LTP
rcu torture testing kernel module	do
class device structures can now be nested	do
/proc/mounts is now pollable	do
LSM hooks for key management facilities added	do
i386 added cmpxchg function	do
i386 physical hotplug cpu added	do
NTFS write support	do
CIFS update for more interoperability	do
UFO networking feature added	do
randomize the bind port for security protection	do
netfilter support for ipv6	do
generic netlink	do
PPP MPPE protocol added for connections to windows machines	do
ABC network congestion control support added	do
IPV6 RFC 3484 support added	do
loads of wireless updates	do
SAK processing speedups	do
SATA drivers for SI, Marvel and Promise devices added	do
IDE support for AMD Geode, CS5535 and M3A-2170 devices	do
IDE support for BCM1250 dvices	do
IDE via82cxx driver	do
input wistron driver support added	do
e1000 multi-queue support added	do
sysfs support for ide tape, raid5 and md	do
frame console rotation can be done on the fly	do
Artemis and ATIK USB cameras support added	do
bluetty USB driver deleted	do
nvidia geforce 7800 series frame buffer support added	do
radeon pci express support added	do
matrox mystique AGP support added	do
Rapid I/O subsystem added	do
Omnikey cardman 4040 and 4000 smartcard reader support added	do
*13 new at syscalls**	2.6.16	(You can add your Name here)
pselect / ppoll syscalls added	do
unshare syscall added	do
mutex primitive added	do
hi resolution timers added	do	This will be part of LTP-RT tests
swap migration for NUMA systems added	do
allow all caches to be flushed by a userspace trigger	do
per mountpoint noatime/nodiratime flags	do
EDAC support added	do
Lots of performance fixes	do
dentry structure shrunk	do
struct page structure shrunk	do
tty layer buffering rework	do
gcc 2.9x support dropped	do
swsusp image encryption support removed	do
MD got sysfs support	do
SPI subsystem added	do
CPU topology support added to sysfs	do
AMD Geode GX and LX processor support added	do
sparsemem support for single processor	do
Intel ICH8 support added	do
SPU file system added for cell processors	do
CPU frequency support added for G5 processors	do
thermal control for G5s	do
new G5 processor support added	do
kexec for PPC added	do
OCFS2 cluster filesystem added	do
configfs filesystem added	do
FUSE limits removed	do
NFSv4 fixes	do
NFS large reads and writes added	do
readpage support for v9fs added	do
external journal device for ext3 added	do
direct I/O for FAT fs added	do
relayfs support for global relay buffers added	do
XFS support for SELinux	do
Kerberos and ACL support for CIFS added	do
TIPC network protocol added	do
IFB network device added	do
IPSec LSM hooks added	do
TCP BIC support updated	do
numa aware netfilter changes	do
DCCPv6 protocol added	do
add ability to corrupt network packets randomly	do
ata_piix suspend support added	do
modalias support added for ide devices	do
modalias support added for input devices	do
geyser2 touchpad driver added	do
USB ADSL modems for ADI eagle devices added	do
Lots of USB storage quirks added	do
ATI/Phillips USB RF driver support added	do
Posiflex PP-7000 USB printer driver added	do
ftdi_sio devices added	do
ATIK USB astronomical CCD support added	do
hostap firmware flashing support	do
Marvell Yukon2 Gigabit ethernet support added	do
vlan acceleration support added to sky2 driver	do
ipx2000 Intel network driver added	do
power management support for lots of sound devices added	do	This will be coming soon
Lots of video for linux driver updates	do
splice, tee, and vmsplice syscall added	2.6.17	(The test cases exist in LTP)
sync_file_range syscall added	do	(This is successfully addedd)
new schedular domain for multi-core processors added	do
lightweight robust futexes added	do
block queue io tracing support added	do
userspace interface to swsusp added	do
/proc/slab_allocators added	do
mountstats information added to procfs	do
mprotect now enabled for huge pages	do
EXPORT_SYMBOL_GPL_FUTURE added	do
CFQ performance tuning	do
FMODE_EXEC file flag support added	do
improved module version support for external modules added	do
SMP alternatices for i386 added to patch instructions in the kernel on the fly	do
CONFIG_REGPARM enabled by the default	do
1Gb process stack randomiziation added (used to be 8Mb)	do
make isoimage support added	do
memory hotadd without sparsemem added	do
lots of Cell processor updates	do
ext3 performance improvements	do
xfs tweaks	do
jfs mount options added	do
ext2 attributes added to jfs	do
jfs support for splice added	do
FUSE O_ASYNC and O_NONBLOCK support added	do
NFS I/O performance counters added	do
NFS client metrics added	do
RPC I/O stats added	do
relayfs support made generic	do
debugs blob support added	do
sysfs attributes are now pollable	do
syscall audit records added to SELinux	do
RFC 4191 IPv6 support added	do
DCCP sysctls added	do
softmac wireless driver layer added	do
lots of new wireless drivers added (broadcom included)	do
PCI legacy proc support removed	do
IPMI driver model support added	do
new device ids and drivers for video added	do
big libata update with new devices and fixes	do
SCSI cache settings added to sysfs	do
braille device support for all input devices added	do
SNES mouse support added	do
unified the USB touchscreen driver for all touchscreens	do
loads of new USB device support added	do
Huge network driver updates	do
large sound driver updates	do
acpi dock support added	do
i2c support for new controllers added	do
LED class support added, along with a lot of diferent LED drivers	do
Secure digital driver support added	do
Niagara multicore CPU processor support added	do

User Feedback

Feedback	User	Your Comment
(1) Use the same coding style for every tests because i saw many tests that is coded in different ways. We could use the linux kernel Coding Style to make the tests(with this itll be easier to maintain the LTP code). (2) Work more with the linux kernel regression bugs. The community general report all regression that was found at every kernel release. You can see the regression list at http://kernelnewbies.org/known_regressions . We have more detailed stuff at the kernel bugzilla. It'd be nice to have regression test cases, to be sure that these bugs are not in kernel anymore. (3) Work closely with the kernel community, making tests as soon we have new features (this is because the kernel community uses the ABAT + LTP to test every kernel release, at http://test.kernel.org/functional/index.html we have the results). For example, at the newest kernel release, 2.6.22-rc1, we have a new system call, named eventfd (http://lwn.net/Articles/234123/rss), and it'd be nice to have a test case that could test this system call. (4) Result graphics, using tools such as gnuplot. It'd be nice to have all results in an output that could be used with gnuplot to make some result graphics. With this we could see all pass/failed tests thought the time, watching the results with different kernels. (5) Improve LTP Website	rsalveti@br.ibm.com
The biggest pain of using LTP is the number of false positives it generates. There are so many false positives that we need to keep a list of the "known errors" in order to decide whether any real error messages have popped out. Fixing bugs is exactly where the focus should remain until it becomes the norm for LTP to declare "pass" for a run where there have been no true failures.Here's the list of LTP errors that we generally ignore as false positives. (1)fcntl17 attempt to signal child failed, (2)fsync took too long, (3)gettimeofday is going backwards, (4)gf18, (5)mlockall* (2944?), (6)msg* call failed - erro=28 (note: after you get error 28 or 'device out of space, you'll get tons and tons of msg* error msgs that are safe to ignore. I think this tends to happen only when swap is less than 2ram, and perhaps only on RHEL), (7)nanosleep* remaining time doesn't match foo (17215?) (8)pselect sleep time was incorrect, (9)syslog* failed to log msgs of all levels, (10)capset02 BROK Unexpected signal 15 received(matrix issue 1234 - if you see this, please mark as PASS, but attach the issue to help establish trends), (11)getsid02 (25542), (12)ioperm* (21070, and 21619), (13)setpgid* expect EACCES got 1 (21134 mention, ltp SF bug 1114033), (14)shmctl01 FAIL : # of attaches is incorrect, (15)shmget02 FAIL: call succeeded unexpectedly (21896), (16)socket* (21065), (17)syslogd no such command (21068 - diff failure than the historical), (18)vmsplice01 1 FAIL : vmsplice() Failed, errno=38 (bz28685)	jmtt@us.ibm.com
I understand, most of the tests generates two kinds of results file. Generally, we will look throu the .log file & if any failures, go through .out file to find out cause for failure & rerun individual test, if required to see it passes or not. I haven't come across such tests which says test failed in log file & end result showing all passed. During those initial days(2000-01), LTP used to yield many bugs. Now a days not much bugs through LTP could mean kernel is getting stronger day by day & hence very hard to find regression defects. Also it is understood that the kernel developers will test their patch for functionality to get ack or nack. LTP is meant for both functional & regression tests, it may not keep track of developers patch whether it has fixed particular issue or not, but if any regressions out of that patch can be found through introduction of new bug. There is a scope for improvement in this area, filter out logs, collect in precise format. As we own LTP maintainership, our job is to make sure each LTP release has latest good patches from the community. And definitely we can contribute on the improvement other areas of LTP (like improving log files, etc), but ultimately it is also the responsibility of kernel developers across the community to contribute to the addition new effective test cases to yield more no. of regression defects to get the glory back. A code coverage analysis on LTP could help us in finding out where the bugs are lacking.	iranna.ankad@in.ibm.com
(1)The networktests and ltpstress workloads use rsh, rcp, and rlogin. These out of date tools should be replaced with more current tools. A good replacement candidate would be ssh based tools. (2)I use ltprun to initiate the LTP base workload. This allows me to get a summary of the test status from the system that we use as a master by running ltp_check. There is no equivalent to ltprun or ltp_check available for the other LTP workloads. It would be nice to have some sort of tool to obtain a summary when running the other workloads. (3)It would be nice to have tests for CIFS/Samba in addition to ftp and nfs. (4)I would like to see long running stress tests for the various network file systems in addition to the test provided for ftp and nfs. Also a more diverse set of sample files for the test cases to operate on	billgo@us.ibm.com
I found that some of the system calls like bdflush, clock_getres, clock_gettime, epoll_create, getdents64, get_mempolicy, mmap2, pivot_root, remap_file_pages, rt_sigaction, endfile64, sched_getaffinity, set_thread_area, set_tid_address, stat64, statfs64, tgkill, waitid, add_key... are not tested.	sachi.linux@gmail.com
One of the nasty things about LTP, is I don't think we pick up any of the test failures and cascade them through to the final test run status. That makes it harder to tell if there's any regression or not, but I guess it's doable by hand (Picked up from his comment in autotest@test.kernel.org)	mbligh@mbligh.org