I have started working with ZFS and its various ways of protecting disks from failure. It’s a low end setup at best, where a JBOD is used as an extension to a M-series server. Many JBODs come with SAS connectivity only, and no on-board intelligence so a host based solution has to be chosen to protect the lot from disk failures.
For Solaris, you can use ZFS amongst other solutions. Alternatively, ASM is a possibility. I couldn’t reproduce the exact setup, so I had to improvise. Still I wanted to find out how ZFS performs compared to ASM. For this purpose I used an EMC VMX and a Sun 5410 server which had 10 multipathed 10G LUNs presented to it via EMC Power Path 5.3.
To test the file system performance I decided to use both IOZone and Bonnie++. Bonnie++ is no problem, you can get it from Sun Freeware. Note that Oracle no longer produce the Companion CD, which leaves SunFreeware the only source for alternative packages.
Before you can compile anything on Solaris, you need a compiler (why doesn’t Solaris come with one?). Installing the compiler was simple. I got the required files from these URLS:
They can be installed by unzipping and pkgadd:
bash-3.00# pkgadd -d gcc-3.4.6-sol10-sparc-local The following packages are available: 1 SMCgcc gcc (sparc) 3.4.6 Select package(s) you wish to process (or 'all' to process all packages). (default: all) [?,??,q]: Processing package instance <SMCgcc> from </root/martin/gcc-3.4.6-sol10-sparc-local> gcc(sparc) 3.4.6 FSF The selected base directory </usr/local> must exist before installation is attempted. Do you want this directory created now [y,n,?,q] y Using </usr/local> as the package base directory. ## Processing package information. ## Processing system information. ## Verifying disk space requirements. ## Checking for conflicts with packages already installed. ## Checking for setuid/setgid programs. Installing gcc as <SMCgcc> ## Installing part 1 of 1. /usr/local/bin/c++ /usr/local/bin/cpp ... /usr/local/share/locale/rw/LC_MESSAGES/gcc.mo /usr/local/share/locale/sv/LC_MESSAGES/gcc.mo /usr/local/share/locale/tr/LC_MESSAGES/gcc.mo [ verifying class <none> ]
Repeat the procedure for libiconv as well.
Once that is done, it’s time to compile iozone. Get the latest tar ball from http://www.iozone.org/ (it was iozone3_397.tar in my case) and unzip it somewhere on your system. Change directory to stageDIR/src/current. For the makefile to work, set your path to include /usr/local/bin and /usr/ccs/bin. The makefile comes with different targets depending on your architecture and compiler. In my case I thought that Solaris10gcc-64 would be most appropriate. Trying this option however didn’t succeed:
bash-3.00# make Solaris10gcc-64 Building iozone for Solaris10gcc-64 gcc -O -c -Dunix -DHAVE_ANSIC_C -DASYNC_IO -D__LP64__ \ -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Dsolaris \ -m64 libbif.c -o libbif10-64.o gcc -O -c -Dunix -DHAVE_ANSIC_C -DASYNC_IO -D__LP64__ \ -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Dsolaris \ -DNAME='"Solaris10gcc-64"' -m64 libasync.c -o libasync10-64.o gcc -c -O -Dunix -DHAVE_ANSIC_C -DASYNC_IO \ -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Dsolaris \ -DNAME='"Solaris10gcc-64"' -m64 iozone.c -o iozone_solaris10gcc-64.o Building fileop for Solaris10gcc-64 gcc -c -O -m64 fileop.c -o fileop_Solaris10gcc-64.o Building the pit_server cc -c pit_server.c -o pit_server.o gcc -O -m64 iozone_solaris10gcc-64.o libasync10-64.o libbif10-64.o \ -lthread -lpthread -lposix4 -lnsl -laio \ -lsocket -o iozone gcc -O -m64 fileop_Solaris10gcc-64.o -o fileop gcc -O -m64 pit_server.o -lthread -lpthread -lposix4 \ -lnsl -laio -lsocket -o pit_server ld: fatal: file pit_server.o: wrong ELF class: ELFCLASS32 ld: fatal: File processing errors. No output written to pit_server collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `Solaris10gcc-64'
ELFCLASS32? I specified that I wanted 64bit! Looking at the output a bit closer it turned out that cc (which I sym-linked to gcc in /usr/local/bin) created a 32bit object file. This was easy to fix. Instead of
# cc -c pit_server.c -o pit_server.o
# gcc -m64 -c pit_server.c -o pit_server.o
That created a 64bit object file:
bash-3.00# file pit_server.o
pit_server.o: ELF 64-bit MSB relocatable SPARCV9 Version 1
Now the linking worked as well:
bash-3.00# gcc -O -m64 pit_server.o -lthread -lpthread -lposix4 \ > -lnsl -laio -lsocket -o pit_server bash-3.00# echo $? 0
And finally pit_server was 64 bit:
bash-3.00# file pit_server pit_server: ELF 64-bit MSB executable SPARCV9 Version 1, dynamically linked, not stripped, no debugging information available