OpenSolaris 2009.06 domU on opensuse 11.2 dom0

I was curious to get started with opensolaris and quite eager to install it as a domU on a Linux dom0. There would have been little problem to do it the other way around. Actually, I could have installed opensolaris on my Toshiba R600 too!

Then I tried out a number of current linux distributions, but except for openSuSE none had a dom0 kernel out of the box which really is a shame. Seems I need to look more closely into KVM with virtio support.

Preparations

Create a network bridge br1 as a host only interface without bridging to any ethx interface. I assigned IP 192.168.99.10 to it, statically:

dom0:~ # cat /etc/sysconfig/network/ifcfg-br1
BOOTPROTO='static'
BRIDGE='yes'
BRIDGE_FORWARDDELAY='0'
BRIDGE_PORTS=''
BRIDGE_STP='off'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR='192.168.99.10/24'
MTU=''
NAME=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'

dom0:~ # ifconfig br1
br1       Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
 inet addr:192.168.99.10  Bcast:192.168.99.255  Mask:255.255.255.0
 inet6 addr: fe80::ac66:2ff:fe8e:c63/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
 RX packets:1668 errors:0 dropped:0 overruns:0 frame:0
 TX packets:1186 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:1352469 (1.2 Mb)  TX bytes:89157 (87.0 Kb)

Now prepare the open Solaris install media (which you can download for free from Sun’s website). I chose the following directory structure:

• /m/osol – mount the open solaris image via the loopback driver
• /m/xen/kernels – to store the kernels for the initial installation
• /m/xen/vmname – to store files pertaining to a vm

Now let’s mount mount the install image and copy the 64bit kernel and ramdisk to /m/xen/kernels

dom0:~ # mount -o loop osol-0906-x86.iso /m/osol
dom0:~ # mkdir /m/xen/kernels && cd /m/xen/kernels
dom0:~ # cp /m/osol/boot/amd64/x86.microroot
dom0:~ # cp /m/osol/platform/i86xpv/kernel/amd64/unix .

dom0:~ # mkdir /m/xen/osol && /m/xen/osol

The last step is to create a sparse file of 8 GB size for the solaris root file system

dom0:/m/xen/osol # dd if=/dev/zero of=system bs=1 count=0 seek=8G
dom0:/m/xen/osol # ls -lh
total 1.1G
-rw-r--r-- 1 root root 8.0G Feb  4 14:34 system

Since non of the configuration tools known to me can boot the live image, we need to create a custom configuration file for the installation of opensolaris. Remember that opensolaris boots into a live mode and then you install it to your disk.

dom0:~ # vi /etc/xen/vm/osol
name = "osol"
maxmem = 1536
memory = 1024
vcpus = 2
kernel = "/m/xen/kernels/unix"
ramdisk = "/m/xen/kernels/x86.microroot"
extra = "/platform/i86xpv/kernel/amd64/unix - nowin -B install_media=cdrom"
on_poweroff = "destroy"
on_reboot = "destroy"
on_crash = "destroy"
disk = [
 'phy:/dev/loop1,xvdc:cdrom,r',
 "file:/m/xen/osol/system,xvda,w",
]
vif = [ "mac=00:16:3e:1b:e8:18,bridge=br1,script=vif-bridge" ]

replace loop1 with the loop device where your CD is mounted – use the following if you are unsure:

dom0:/etc/xen/vm # mount | grep loop
/dev/loop0 on /srv/www/htdocs/rhel type iso9660 (rw)
/dev/loop1 on /m/osol type iso9660 (rw)

As you can see it’s loop1 in my case.

Installation

Begin by starting the new domU and jump straight to it’s console. Text in red indicates user input:

dom0:/etc/xen/vm # xm create -c osol
Using config file "./osol".
Started domain osol (id=10)
 v3.4.1_19718_04-2.1 chgset '19718'
SunOS Release 5.11 Version snv_111b 64-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: opensolaris
Remounting root read/write
Probing for device nodes ...
Preparing live image for use
Done mounting Live image
USB keyboard
 1. Albanian                      23. Lithuanian                    
 2. Belarusian                    24. Latvian                       
 3. Belgian                       25. Macedonian                    
 4. Brazilian                     26. Malta_UK                      
 5. Bulgarian                     27. Malta_US                      
 6. Canadian-Bilingual            28. Norwegian                     
 7. Croatian                      29. Polish                        
 8. Czech                         30. Portuguese                    
 9. Danish                        31. Russian                       
10. Dutch                         32. Serbia-And-Montenegro         
11. Finnish                       33. Slovenian                     
12. French                        34. Slovakian                     
13. French-Canadian               35. Spanish                       
14. Hungarian                     36. Swedish                       
15. German                        37. Swiss-French                  
16. Greek                         38. Swiss-German                  
17. Icelandic                     39. Traditional-Chinese           
18. Italian                       40. TurkishQ                      
19. Japanese-type6                41. TurkishF                      
20. Japanese                      42. UK-English                    
21. Korean                        43. US-English                    
22. Latin-American                
To select the keyboard layout, enter a number [default 43]: 43

 1. Arabic                        
 2. Chinese - Simplified          
 3. Chinese - Traditional         
 4. Czech                         
 5. Dutch                         
 6. English                       
 7. French                        
 8. German                        
 9. Greek                         
10. Hebrew                        
11. Hungarian                     
12. Indonesian                    
13. Italian                       
14. Japanese                      
15. Korean                        
16. Polish                        
17. Portuguese - Brazil           
18. Russian                       
19. Slovak                        
20. Spanish                       
21. Swedish                       
To select desktop language, enter a number [default is 6]: 6

User selected: English
Configuring devices.
Mounting cdroms
Reading ZFS config: done.

opensolaris console login: jack
Password:
Sun Microsystems Inc.   SunOS 5.11      snv_111b        November 2008
jack@opensolaris:~$

Congratulations, you just booted open solaris! Log in using user jack, password jack. You can su to root, password is opensolaris.

BTW, the configuration file translates into the following libvirt XML file:

dom0:~ # virsh dumpxml 10
<domain type='xen' id='10'>
 <name>osol</name>
 <uuid>e6d15d8d-0356-f9c7-9baf-35ac2e51bd38</uuid>
 <memory>1572864</memory>
 <currentMemory>1048576</currentMemory>
 <vcpu>2</vcpu>
 <bootloader></bootloader>
 <os>
   <type>linux</type>
   <kernel>/m/xen/kernels/unix</kernel>
   <initrd>/m/xen/kernels/x86.microroot</initrd>
   <cmdline>/platform/i86xpv/kernel/amd64/unix - nowin -B install_media=cdrom</cmdline>
 </os>
 <clock offset='utc'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>destroy</on_reboot>
 <on_crash>destroy</on_crash>
 <devices>
   <disk type='block' device='cdrom'>
     <driver name='phy'/>
     <source dev='/dev/loop1'/>
     <target dev='xvdc' bus='xen'/>
     <readonly/>
   </disk>
   <disk type='file' device='disk'>
     <driver name='file'/>
     <source file='/m/xen/osol/system'/>
     <target dev='xvda' bus='xen'/>
   </disk>
   <interface type='bridge'>
     <mac address='00:16:3e:1b:e8:18'/>
     <source bridge='br1'/>
     <script path='/etc/xen/scripts/vif-bridge'/>
     <target dev='vif10.0'/>
   </interface>
 <console type='pty' tty='/dev/pts/5'>
 <source path='/dev/pts/5'/>
 <target port='0'/>
 </console>
 </devices>
</domain>

OpenSolaris broadcasts for a DHCP address which isn’t working yet when para virtualised. We have to give it a static IP address:

root@opensolaris:~# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
 inet 127.0.0.1 netmask ff000000
xnf0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 2
 inet 0.0.0.0 netmask ff000000
 ether 0:16:3e:1b:e8:18
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
 inet6 ::1/128
xnf0: flags=2004841<UP,RUNNING,MULTICAST,DHCP,IPv6> mtu 1500 index 2
 inet6 fe80::216:3eff:fe1b:e818/10
 ether 0:16:3e:1b:e8:18
root@opensolaris:~# ifconfig xnf0 192.168.99.11 netmask 255.255.255.0
root@opensolaris:~# ifconfig xnf0
xnf0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
 inet 192.168.99.11 netmask ffffff00 broadcast 192.168.99.255
 ether 0:16:3e:1b:e8:18

Start a vncserver as root in order to kick off the installation.

root@opensolaris:~# vncserver

You will require a password to access your desktops.

Password:
Verify:
xauth:  creating new authority file /root/.Xauthority

New 'opensolaris:1 ()' desktop is opensolaris:1

Creating default startup script /root/.vnc/xstartup
Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/opensolaris:1.log

On your dom0, start a vnc session and connect to 192.168.99.11:1 (note the “:1” !)

root# /usr/bin/gui-install &

The installer is very user friendly so I won’t go into detail. Don’t reboot at the end – change your domU configuration file. In the past (prior to xen 3.4.x) you had to play around with the zpool and extra= line in the configuration file which could be a real pain and I wasted quite a bit of time finding out what arguments to put into the line.

Luckily we’re on xen 3.4.1 which has zfs support in pygrub. So the complex looking configuration files can be reduced to:

name = "osol"
maxmem = 1536
memory = 1024
vcpus = 2

on_poweroff = "destroy"
on_reboot = "destroy"
on_crash = "destroy"

disk = [
 "file:/m/xen/osol/system,xvda,w",
]
vif = [ "mac=00:16:3e:1b:e8:18,bridge=br1,script=vif-bridge" ]

bootloader = '/usr/bin/pygrub'

You can then restart the domU and it boots straight into the installed os:

dom0:/etc/xen/vm # xm create -c osol
Using config file "./osol".
Started domain osol (id=24)
 v3.4.1_19718_04-2.1 chgset '19718'
SunOS Release 5.11 Version snv_111b 64-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: opensolaris
Configuring devices.
Loading smf(5) service descriptions: 127/150
Feb  4 17:12:23 svc.startd[7]: Lost repository event due to disconnection.
150/150
svccfg import warnings. See /var/svc/log/system-manifest-import:default.log .
Reading ZFS config: done.
Mounting ZFS filesystems: (6/6)
Creating new rsa public/private host key pair
Creating new dsa public/private host key pair

opensolaris console login:

Remember that you have to use the username and password you defined during the installation-jack won’t do it anymore.

Reference

http://bderzhavets.blogspot.com/2009/06/setup-opensolaris-2009.html