I have been lucky enough to do some work with Fusion IO cards in a blade server, soon to be followed by another set of tests on a full rack mounted server. I didn’t know exactly where model I was given, but powered my server down in eager anticipation of the events to come.
After the engineer plugged the card in, and powered the server up I logged in as root to find out what about the pre-christmas present. I knew it was a PCI card, so surely lspci would tell me more. Here’s the output:
lspci -vvvv 41:00.0 Mass storage controller: Fusion-io ioDimm3 (v1.2) (rev 01) Subsystem: Hewlett-Packard Company Unknown device 324d Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 10 Region 5: Memory at dfef0000 (32-bit, non-prefetchable) [size=64K] [virtual] Expansion ROM at 98400000 [disabled] [size=1M] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot-,D3cold-) Status: D0 PME-Enable- DSel=0 DScale=0 PME- Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- Address: 0000000000000000 Data: 0000 Capabilities: [60] Express Endpoint IRQ 0 Device: Supported: MaxPayload 512 bytes, PhantFunc 1, ExtTag+ Device: Latency L0s <64ns, L1 <1us Device: AtnBtn- AtnInd- PwrInd- Device: Errors: Correctable- Non-Fatal+ Fatal+ Unsupported- Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes Link: Supported Speed 2.5Gb/s, Width x4, ASPM L0s, Port 0 Link: Latency L0s unlimited, L1 unlimited Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch- Link: Speed 2.5Gb/s, Width x4
So it’s indeed a FusionIO card (an ioDrive to be precise), and it’s plugged into a x4 slot, the mimimum required.
What to do next? Always good to read the docs. The fusionio.com website allows you to download documentation and drivers after a free registration. Luckily the website didn’t have kernel modules for Oracle Linux I’m using (RHEL 5 only), which gives me the opportunity to build the software from source. I don’t like surprises, therefore I created my ~/.rpmmacros with the following content:
[root@computer1 rpm]# cat ~/.rpmmacros %_topdir /home/martin/rpm %_tmppath /home/martin/rpm/tmp
This obviously requires the full tree underneath the topdir, namely
- BUILD
- BUILDROOT
- RPMS
- SOURCES
- SPECS
- SRPMS
- tmp
With these directories in place it’s as simple as running rpmbuild –rebuild iomemory-vsl-2.3.1.123-1.0.src.rpm as a non-root user (martin in my case) and wait for the RPMs to be created in the RPMs/x86-64 directory. Following the documentation again I installed the needed software:
[root@computer1 x86_64]# rpm -ihv iomemory-vsl-2.6.18-194.26.1.0.1.el5-2.3.1.123-1.0.x86_64.rpm Preparing... ########################################### [100%] 1:iomemory-vsl-2.6.18-194########################################### [100%] [root@computer1 fusionio]# rpm -Uvh lib*.rpm Preparing... ########################################### [100%] 1:libfio ########################################### [ 50%] 2:libfusionjni ########################################### [100%] [root@computer1 fusionio]# rpm -Uvh fio*.rpm Preparing... ########################################### [100%] 1:fio-common ########################################### [ 14%] 2:fio-util ########################################### [ 29%] 3:fio-remote-util ########################################### [ 43%] 4:fio-smis ########################################### [ 57%] 5:fio-snmp-agentx ########################################### [ 71%] 6:fio-snmp-mib ########################################### [ 86%] 7:fio-sysvinit ########################################### [100%] [root@computer1 fusionio]#
Note that you don’t actually need the fio-sysvinit package if your distribution is reasonably modern. UDEV will load any drivers automatically.
With that completed as well it’s time to load the kernel module and watch Linux do the rest. The fio-status tool queries the card’s helth:
[root@computer1 x86_64]# fio-status Found 1 ioDrive in this system Fusion-io driver version: 2.3.1 build 123 fct0 Attached as 'fioa' (block device) HP StorageWorks 320GB IO Accelerator, Product Number:AJ878A SN:07902 Alt PN:507152-001 PCI:41:00.0 Firmware v5.0.5, rev 43674 322.55 GBytes block device size, 396 GBytes physical device size Sufficient power available: Unknown Internal temperature: 40.4 degC, max 40.9 degC Media status: Healthy; Reserves: 100.00%, warn at 10.00%
That’s it! so simple-another blog post will detail how I ran a first orion benchmark on it and show some impressive numbers.
