Getting started with FusionIO

I have been lucky enough to do some work with Fusion IO cards in a blade server, soon to be followed by another set of tests on a full rack mounted server. I didn’t know exactly where model I was given, but powered my server down in eager anticipation of the events to come.

After the engineer plugged the card in, and powered the server up I logged in as root to find out what about the pre-christmas present. I knew it was a PCI card, so surely lspci would tell me more. Here’s the output:

lspci -vvvv

41:00.0 Mass storage controller: Fusion-io ioDimm3 (v1.2) (rev 01)
Subsystem: Hewlett-Packard Company Unknown device 324d
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 10
Region 5: Memory at dfef0000 (32-bit, non-prefetchable) [size=64K]
[virtual] Expansion ROM at 98400000 [disabled] [size=1M]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
Address: 0000000000000000  Data: 0000
Capabilities: [60] Express Endpoint IRQ 0
Device: Supported: MaxPayload 512 bytes, PhantFunc 1, ExtTag+
Device: Latency L0s <64ns, L1 <1us
Device: AtnBtn- AtnInd- PwrInd-
Device: Errors: Correctable- Non-Fatal+ Fatal+ Unsupported-
Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes
Link: Supported Speed 2.5Gb/s, Width x4, ASPM L0s, Port 0
Link: Latency L0s unlimited, L1 unlimited
Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
Link: Speed 2.5Gb/s, Width x4

So it’s indeed a FusionIO card (an ioDrive to be precise), and it’s plugged into a x4 slot, the mimimum required.

What to do next? Always good to read the docs. The fusionio.com website allows you to download documentation and drivers after a free registration. Luckily the website didn’t have kernel modules for Oracle Linux I’m using (RHEL 5 only), which gives me the opportunity to build the software from source. I don’t like surprises, therefore I created my ~/.rpmmacros with the following content:

[root@computer1 rpm]# cat ~/.rpmmacros
%_topdir /home/martin/rpm
%_tmppath /home/martin/rpm/tmp

This obviously requires the full tree underneath the topdir, namely

• BUILD
• BUILDROOT
• RPMS
• SOURCES
• SPECS
• SRPMS
• tmp

With these directories in place it’s as simple as running rpmbuild –rebuild iomemory-vsl-2.3.1.123-1.0.src.rpm as a non-root user (martin in my case) and wait for the RPMs to be created in the RPMs/x86-64 directory. Following the documentation again I installed the needed software:

[root@computer1 x86_64]# rpm -ihv iomemory-vsl-2.6.18-194.26.1.0.1.el5-2.3.1.123-1.0.x86_64.rpm
Preparing...                ########################################### [100%]
1:iomemory-vsl-2.6.18-194########################################### [100%]
[root@computer1 fusionio]# rpm -Uvh lib*.rpm
Preparing...                ########################################### [100%]
1:libfio                 ########################################### [ 50%]
2:libfusionjni           ########################################### [100%]
[root@computer1 fusionio]# rpm -Uvh fio*.rpm
Preparing...                ########################################### [100%]
1:fio-common             ########################################### [ 14%]
2:fio-util               ########################################### [ 29%]
3:fio-remote-util        ########################################### [ 43%]
4:fio-smis               ########################################### [ 57%]
5:fio-snmp-agentx        ########################################### [ 71%]
6:fio-snmp-mib           ########################################### [ 86%]
7:fio-sysvinit           ########################################### [100%]
[root@computer1 fusionio]#

Note that you don’t actually need the fio-sysvinit package if your distribution is reasonably modern. UDEV will load any drivers automatically.

With that completed as well it’s time to load the kernel module and watch Linux do the rest. The fio-status tool queries the card’s helth:

[root@computer1 x86_64]# fio-status

Found 1 ioDrive in this system
Fusion-io driver version: 2.3.1 build 123

fct0    Attached as 'fioa' (block device)
HP StorageWorks 320GB IO Accelerator, Product Number:AJ878A SN:07902
Alt PN:507152-001
PCI:41:00.0
Firmware v5.0.5, rev 43674
322.55 GBytes block device size, 396 GBytes physical device size
Sufficient power available: Unknown
Internal temperature: 40.4 degC, max 40.9 degC
Media status: Healthy; Reserves: 100.00%, warn at 10.00%

That’s it! so simple-another blog post will detail how I ran a first orion benchmark on it and show some impressive numbers.