Test Plan for NFS in 2.6

Version: 1.25

Owner:
Robbie Williamson
robbiew@us.ibm.com
IBM Linux Technology Center
It is the responsibility of the user of this document to ensure that they are using the current version of this document. To validate that your copy of this document is at the latest level, view the latest version of this document at http://ltp.sf.net/nfs.

Table of Contents

Document Control

 

Reviewers

This document is distributed by LTC Test. The latest version can be obtained at  location .
Name
Email
Ray Young
rayy@us.ibm.com
Linda Scott
lindajs@us.ibm.com
Stephanie Glass sglass@us.ibm.cim 
Bruce Allan bruce.allan@us.ibm.com
Mark VanderWiele markv@us.ibm.com 
Trond Myklebust trond.myklebust@fys.uio.no
Yusuf Goolamabbas yusufg@outblaze.com

 

Change Summary

This section reflects the changes made to the document.
Date
Author
Description of Changes
06/09/03 Robbie Williamson Initial Draft, Version 1.0
06/10/03 Robbie Williamson Added Execution Plan, Version 1.1
06/11/03 Robbie Williamson Made changes based on suggestions received from Bruce Allan, Version 1.2
06/12/03 Robbie Williamson Updated Test Approach and Methodology section with 'sysctl' tuning information, Version 1.3
06/13/03 Robbie Williamson Updated tested kernel to 2.5.70-bk18 to avoid scheduler problems I was having with 2.5.70., Version 1.4
06/16/03 Robbie Williamson Applied patch to BugMe #805 to all kernels, Version 1.5
06/16/03 Robbie Williamson Updated tested kernel to 2.5.70-bk19 with BugMe #805 patch, Version 1.6
06/16/03 Robbie Williamson Changed server config to 4-way SMP. Ran into possible scheduler bug, updated to 2.5.71-bk2, Version 1.7
06/17/03 Robbie Williamson Added "Future Plans" section and Yusuf as a reviewer, Version 1.8
06/17/03 Robbie Williamson Updated all machines to 2.5.72 because this revision contained an NFS/RPC update, Version 1.9
06/17/03 Robbie Williamson Added ethernet driver information to "Environment" section, Version 1.10.
06/24/03 Robbie Williamson Up the kernel to 2.5.73 and still hit BugMe #813.  I've decided to switch the server's kernel back to UP in hopes to help avoid this bug.....and START TESTING!!!, Version 1.11.
07/02/03 Robbie Williamson Replaced JFS partition with XFS, due to instability issues with JFS, and updated kernels to 2.5.73-bk9, Version 1.12.
07/07/03 Robbie Williamson Replaced XFS partition with EXT2, due to BugMe #870, Version 1.13.
07/09/03 Robbie Williamson Changed the 'sysctl' network settings, number of NFSDs, and filesystem configuration based on readings from the NFS How-To, Version 1.14
07/11/03 Robbie Williamson Updated kernels to 2.5.75-bk1.
07/14/03 Robbie Williamson Updated server to use nfs-utils v1.0.4.  I also recompiled all kernels with Page Allocation Debugging turned off, to avoid possible problems with having it on, Version 1.16.
07/15/03 Robbie Williamson Updated kernels to 2.6.0-test1 and applied a patch for nfsd/nfs3xdr.c that I received from Neil Brown, version 1.17
07/18/03 Robbie Williamson Updated server to nfs-utils-1.0.5., version 1.18
07/29/03 Robbie Williamson Updated kernels to 2.6.0-test2., version 1.19
08/13/03 Robbie Williamson Changed the title to reflect 2.6 and updated kernels to 2.6.0-test3, version 1.20
08/28/03 Robbie Williamson Changed the client kernels to 2.6.0-test4-bk2 and this seems to avoid BugMe #1097, version 1.21
09/01/03 Robbie Williamson Hit BugMe #1097 after 3 days of running.  Changed all kernels to 2.6.0-test4-bk4 and restarting. Version 1.22
09/09/03 Robbie Williamson Added Results section and updated kernel versions to 2.6.0-test5. Version 1.23
09/17/03 Robbie Williamson Updated nfs-utils to 1.0.6 on server. Version 1.24
09/22/03 Robbie Williamson Updated all kernels to 2.6.0-test5-bk8.  Version 1.25

Overview

Purpose

This document lays out a plan to test the robustness and reliability of the NFS network component of the 2.6 kernel for the Linux operating system.

Scope

The testing will cover the NFS kernel code available for the latest 2.6 Linux kernel release.

Environment

 
Hardware
Operating Systems
Kernel  and Software Versions
Server
Pentium III Xeon 700Mhz
4way-SMP
4Gb of RAM
4 X 36.4GB disks
10/100 ethernet (netsemi driver code)
SuSE 8.0 Personal
Kernel: 2.6.0-test5-bk8
nfs-utils: 1.0.6
Client 1
Pentium III 866Mhz 
UP 
256Mb of RAM
30GB disk
10/100 ethernet (e100 driver code)
SuSE 8.0 Personal
Kernel: 2.6.0-test5-bk8
Client 2
Pentium III 866Mhz 
UP 
256Mb of RAM
30GB disk
10/100 ethernet (e100 driver code)		 SuSE 8.0 Personal Kernel: 2.6.0-test5-bk8

Assumptions and Dependencies

Assumptions

Dependencies

Test Goals and Objectives

Test Approach and Methodology

The test will consist of 1 server hosting 24 client mounts with 50 connections per client mount.  The 24 mounts will consist of 3 filesystems (EXT2, EXT3, & ReiserFS) mounted 4 ways (v2/udp, v3/udp, v2/tcp, v3/tcp) to 2 client machines.  The server will be configured for 32 NFS server daemons (nfsd) to support each server cpu ( 4 CPUs X 8 ).  All NFSv3 connections will be configured for 8Kb (8192 bytes) read and write buffer sizes. The test program, fsstress, will run 1000 random file operations per connection in a continuous loop for durations of: 1 hour, 24 hours, and 120 hours. System utilization will be monitored and recorded using the programs: 'sar' and 'nfsstat'.
Server Export Options
/nfsserver/ext2        *(rw,no_root_squash,sync,no_subtree_check)
/nfsserver/ext3*       *(rw,no_root_squash,sync,no_subtree_check)
/nfsserver/reiserfs*   *(rw,no_root_squash,sync,no_subtree_check)
*These filesystems have there logs stored on external drives to allow better NFS performance.
 

Client Mount Options
#EXT2
<server>:/nfsserver/ext2     /mnt/nfs/jfs/udp/2      nfs     soft,intr,rw,proto=udp,vers=2
<server>:/nfsserver/ext2     /mnt/nfs/jfs/udp/3      nfs     soft,intr,rw,proto=udp,vers=3,rsize=8192,wsize=8192
<server>:/nfsserver/ext2     /mnt/nfs/jfs/tcp/2      nfs     soft,intr,rw,proto=tcp,vers=2
<server>:/nfsserver/ext2     /mnt/nfs/jfs/tcp/3      nfs     soft,intr,rw,proto=tcp,vers=3,rsize=8192,wsize=8192

#EXT3
<server>:/nfsserver/ext3    /mnt/nfs/ext3/udp/2     nfs     soft,intr,rw,proto=udp,vers=2
<server>:/nfsserver/ext3    /mnt/nfs/ext3/udp/3     nfs     soft,intr,rw,proto=udp,vers=3,rsize=8192,wsize=8192
<server>:/nfsserver/ext3    /mnt/nfs/ext3/tcp/2     nfs     soft,intr,rw,proto=tcp,vers=2
<server>:/nfsserver/ext3    /mnt/nfs/ext3/tcp/3     nfs     soft,intr,rw,proto=tcp,vers=3,rsize=8192,wsize=8192

#REISERFS
<server>:/nfsserver/reiserfs        /mnt/nfs/reiserfs/udp/2 nfs     soft,intr,rw,proto=udp,vers=2
<server>:/nfsserver/reiserfs        /mnt/nfs/reiserfs/udp/3 nfs     soft,intr,rw,proto=udp,vers=3,rsize=8192,wsize=8192
<server>:/nfsserver/reiserfs        /mnt/nfs/reiserfs/tcp/2 nfs     soft,intr,rw,proto=tcp,vers=2
<server>:/nfsserver/reiserfs        /mnt/nfs/reiserfs/tcp/3 nfs     soft,intr,rw,proto=tcp,vers=3,rsize=8192,wsize=8192

Sysctl Tuning
Clients:

fs.file-max=471859
Server:
net.core.rmem_max=1048576*
net.core.rmem_default=1048576*
fs.file-max=471859
* These are set prior to starting nfsd, then set back to default afterwards, per the NFS How-To "Performance Tuning" instructions.

Problem Reporting and Tracking

Entry and Exit Criteria

Entry Criteria

Exit Criteria

Testcase Information

fsstress
This test was developed by SGI and is distributed as a part of the Linux Test Project (LTP) test suite.  The test was originally developed for testing SGI's XFS, however it has proved useful in testing filesystems of all types.  The test is able to create enough file system activity to cover and extensively stress all the NFS functions reported in the NFS statistics monitor, 'nfsstat'.  It does this by randomly executing the following operations:
chown
creat
dread
dwrite
fdatasync
fsync
getdents
link
mkdir
mknod
read
rename
readlink
rmdir
stat
symlink
sync
truncate
unlink
write
and using random name padding and file sizes.
The test package and execution script can be downloaded at http://ltp.sf.net/nfs/fsstress.tgz.

Execution Plan

This portion of the plan will be updated with actual dates as the product is under test. This document will be the best source to determine in what state the product test is in. The execution plan does not take into account time required for equipment and/or power failures, this will be counted towards the setup time.
 
Test Duration
Planned Start
Actual Start
Planned Finish
Actual Finish
1 Hour
06/13/03
 09/04/03
06/13/03
09/04/03*
24 Hours
06/16/03
 09/04/03
06/17/03
09/05/03*
120 Hours
06/18/03
 09/04/03
06/23/03
09/09/03*
*Only over NFSv2.

Results

NFSv2 - nfsstat statistics
 
null getattr setattr root lookup readlink
0 58197509 7610976 0 76955279 801737
read wrcache write create remove rename
13162219 0 53428800 8333973 10371479 3211979
link symlink mkdir rmdir readdir fsstat
1369436 701055 2890169 4144025 8821289 3

 
 

Future Plans

End of Document

Owner: Robbie Williamson