Category Archives: Cloud

Ansible tips’n’tricks: using the OCI Dynamic Inventory Plugin in playbooks

After having covered how to configure the Ansible Dynamic Inventory Plugin for Oracle Cloud Infrastructure (OCI) in the previous posts now it’s time to get it to work with my simple-app cloud application. Before I go into more detail, I’d like to add the usual caveat first.

Caveat

As I said in the previous post, the Dynamic Inventory Plugin is a great time saver, especially when used as part of build pipelines. However, as with anything that’s misconfigured and doesn’t adhere to best practices, there is a risk associated. You might run playbooks against hosts you didn’t intend to. Ensure you have proper Identity and Access (IAM) policies in place, and use the principle of last privilege throughout. And use tags to filter hosts, as you see in this and the previous example. And finally, use the ansible toolset (ansible-inventory fwiw) to validate the hosts you target.

Using the Dynamic Inventory Plugin

The Dynamic Inventory Plugin nicely prints all the VMs in my configuration, broken down by tag. If the plugin returns fewer hosts than expected, your configuration might need tweaking as I explained in this post.

[opc@supersecureVM ansible]$ ansible-inventory -vi ougdemo-compartment.oci.yml --graph
@all:
  |--@IHsr_EU-FRANKFURT-1-AD-2:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@IHsr_EU-FRANKFURT-1-AD-3:
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@all_hosts:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@ougdemo-department:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@project#name=simple-app:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@region_eu-frankfurt-1:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@tag_role=appserver:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@tag_role=bastionhost:
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@ungrouped: 

As you can see there are plenty of tags assigned. Tagging is one of the most important tasks in the cloud, as it is the most convenient way to identify resources.

The logical next step is to make use of that detail! I used free form tags to identify my bastion host and the app servers. So how can I target them with my playbook?

Saying hello to both app servers

I wrote a little playbook as a quick example on how I can use the free form tags to address the app servers. I guess that’s the bare minimum example I can get away with.

[opc@supersecureVM ansible]$ cat hello-appservers.yml 
- hosts: tag_role=appserver
  name: say hello to app servers
  tasks:
  - name: say hello
    debug:
      msg: Hello from {{ ansible_hostname }}
[opc@supersecureVM ansible]$  

You call it just as you would call any other Ansible playbook, substituting the static inventory with the Dynamic Inventory. As you can see the app servers are referenced by their role, indicated by the tags assigned.

[opc@supersecureVM ansible]$ ansible-playbook -i ougdemo-compartment.oci.yml hello-appservers.yml 

[... output showing the inventory is build skipped ...]

PLAY [say hello to app servers] *************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************
ok: [appserver1.app.simpleapp.oraclevcn.com]
ok: [appserver2.app.simpleapp.oraclevcn.com]

TASK [say hello] ****************************************************************************************************************
ok: [appserver1.app.simpleapp.oraclevcn.com] => {}

MSG:

Hello from appserver1
ok: [appserver2.app.simpleapp.oraclevcn.com] => {}

MSG:

Hello from appserver2

PLAY RECAP **********************************************************************************************************************
appserver1.app.simpleapp.oraclevcn.com : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
appserver2.app.simpleapp.oraclevcn.com : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

[opc@supersecureVM ansible]$  

Happy automating!

Ansible Dynamic Inventory Plugin for OCI: where are all my hosts?

I wrote about the configuration of the Ansible Dynamic Inventory Plugin for Oracle Cloud Infrastructure (OCI) in a previous post. As it seems to happen all the time, the length of the post escalated quickly. When I finished the previous post there was still a lot to say! For example, I wanted to share the answer to the question where all my hosts were hiding :)

Caveat

As I said in the previous post, the Dynamic Inventory Plugin is a great time saver, especially when used as part of build pipelines. However, as with anything that’s misconfigured and doesn’t adhere to best practices, there is a risk associated. You might run playbooks against hosts you didn’t intend to. Ensure you have proper Identity and Access (IAM) policies in place, and use the principle of last privilege throughout. And use tags to filter hosts, as you see in this and the previous example. And finally, use the ansible toolset to validate the hosts you target.

I can’t see all my cloud VMs!

After I have been able to resolve the initial problems with the Dynamic Inventory Plugin I was a bit surprised to see it report fewer hosts than expected. At the time I had used the following configuration (YAML) file after piecing the various examples together:

vagrant@ocidev:~/ansible$ cat ougdemo-compartment.oci.yml 
plugin: oracle.oci.oci

regions:
- eu-frankfurt-1

filters:
- defined_tags: { "project": { "name": "simple-app" } }

compartments:
- compartment_ocid: "ocid1.compartment...."
  fetch_hosts_from_subcompartments: true 

This should show me all the hosts of my simple-app project in the compartment I indicated. However the output did not match the number of VMs I had deployed:

vagrant@ocidev:~/ansible$ ansible-inventory -i ougdemo-compartment.oci.yml --graph

[...]

@all:
  |--@IHsr_EU-FRANKFURT-1-AD-2:
  |  |--130.61.233.113
  |--@Oracle-Tags#CreatedBy=ougdemouser:
  |  |--130.61.233.113
  |--@Oracle-Tags#CreatedOn=2020-11-11T18_01_04.484Z:
  |  |--130.61.233.113
  |--@all_hosts:
  |  |--130.61.233.113
  |--@ougdemo-department:
  |  |--130.61.233.113
  |--@project#name=simple-app:
  |  |--130.61.233.113
  |--@region_eu-frankfurt-1:
  |  |--130.61.233.113
  |--@tag_role=bastionhost:
  |  |--130.61.233.113
  |--@ungrouped:
vagrant@ocidev:~/ansible$ 

Although there is a lot of output, actually there is just 1 IP address (VM) discovered. Despite the fact I had a few more hosts created as shown by the OCI CLI:

$ oci compute instance list \
-c ${COMPARTMENT} \
--query "data[].{name:\"display-name\",state:\"lifecycle-state\",tags:\"defined-tags\"}" \
--output table
+------------+---------+---------------------------------------------------------------------------------------------------------------------------+
| name       | state   | tags                                                                                                                      |
+------------+---------+---------------------------------------------------------------------------------------------------------------------------+
| appserver1 | RUNNING | {'project': {'name': 'simple-app'}, 'Oracle-Tags': {'CreatedBy': 'ougdemouser', 'CreatedOn': '2020-11-11T18:01:03.639Z'}} |
| bastion1   | RUNNING | {'project': {'name': 'simple-app'}, 'Oracle-Tags': {'CreatedBy': 'ougdemouser', 'CreatedOn': '2020-11-11T18:01:04.484Z'}} |
| appserver2 | RUNNING | {'project': {'name': 'simple-app'}, 'Oracle-Tags': {'CreatedBy': 'ougdemouser', 'CreatedOn': '2020-11-11T18:01:03.916Z'}} |
+------------+---------+---------------------------------------------------------------------------------------------------------------------------+ 

So where are the missing hosts? I knew they were part of my private subnet so after a little bit of digging around the docs, I found this:

> ORACLE.OCI.OCI    (/home/vagrant/.ansible/collections/ansible_collections/oracle/oci/plugins/inventory/oci.py)

        Get inventory hosts from oci. Uses a .oci.yaml (or .oci.yml)
        YAML configuration file.

OPTIONS (= is mandatory):

[ more documentation ]

- hostname_format
        Host naming format to use. Use 'fqdn' to list hosts using the instance's
        Fully Qualified Domain Name (FQDN). These FQDNs are resolvable within the
        VCN using the VCN resolver specified through the subnet's DHCP options.
        Please see https://docs.us-
        phoenix-1.oraclecloud.com/Content/Network/Concepts/dns.htm for more details.
        Use 'public_ip' to list hosts using public IP address. Use 'private_ip' to
        list hosts using private IP address. By default, hosts are listed using
        public IP address.
        [Default: (null)]
        set_via:
          env:
          - name: OCI_HOSTNAME_FORMAT

[ more documentation ]

That explains it – by default only public IPs are returned. Since my app servers are in a private subnet, they don’t have public IPs, and won’t show up.

Fetching all hosts

With this knowledge at hand the solution merely implied changing the configuration file. I chose for it to display the fully qualified domain name (FQDN). Now all my hosts are fetched by the Dynamic Inventory Plugin:

vagrant@ocidev:~/ansible$ ansible-inventory -i ougdemo-compartment.oci.yml --graph

[...]

@all:
  |--@IHsr_EU-FRANKFURT-1-AD-2:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@IHsr_EU-FRANKFURT-1-AD-3:
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedBy=ougdemouser:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedOn=2020-11-11T18_01_03.639Z:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedOn=2020-11-11T18_01_03.916Z:
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedOn=2020-11-11T18_01_04.484Z:
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@all_hosts:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@ougdemo-department:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@project#name=simple-app:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@region_eu-frankfurt-1:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@tag_role=appserver:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@tag_role=bastionhost:
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@ungrouped: 

Happy scripting!

Ansible tips’n’tricks: configuring the dynamic inventory plugin for Oracle Cloud Infrastructure

As yet another preparation for a talk I’m about to give later this year I ran into a documentation bug concerning the use of the Ansible Dynamic Inventory plugin for Oracle Cloud Infrastructure. It has since been fixed, but it prompted me to write the entire procedure down anyway as I couldn’t really make sense of it at first. That’s probably more of a reflection on me than on the docs, which I found generally to be quite good.

My environment

In case you find this post via a search engine, this is the kit I was using when I put this post together:

  • A Debian 10 (Buster) VM where all the fun takes place
  • Ubuntu 20.04 LTS hosting the VM
  • VirtualBox 6.1.16

If you use an Oracle Linux VM in OCI, or an Oracle Linux VM the steps to configure the inventory plugin are different as most software can be installed using yum. This post shows the manual steps. If I find the time I’ll put another post together describing how to install the Ansible inventory plugin on Oracle Linux.

Ansible

Debian 10 ships with Ansible 2.7 so I decided to try installing Ansible 2.10 via pip3. I seem to have read somewhere that a modern Ansible distribution is a prerequisite for the Dynamic Inventory Plugin, although I can’t find the source now. Nevertheless, there are interesting new features in 2.10 worth trying, so I went with it in the end.

Installing Ansible 2.10 is via pip is quite simple, and it resulted in the desired version. The only downside was that I am now responsible for keeping Ansible current. I went with the distribution’s python3 interpreter and pip3. The same goes for virtualenv you will see referenced later.

I followed the documented steps and installed Ansible into my user account:

vagrant@ocidev:~$ pip3 install ansible --user
Collecting ansible
Downloading 

...

Installing collected packages: PyYAML, MarkupSafe, jinja2, pyparsing, 
packaging, ansible-base, ansible
 Successfully installed MarkupSafe-1.1.1 PyYAML-5.3.1 ansible-2.10.3 
ansible-base-2.10.3 jinja2-2.11.2 packaging-20.4 pyparsing-2.4.7

Note the --user flag as part of the command, it is very important! After having added /home/vagrant/.local/bin/ to the PATH, I had access to my favourite automation toolkit.

vagrant@ocidev:~$ ansible --version
ansible 2.10.3
   config file = None
   configured module search path = ['/home/vagrant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
   ansible python module location = /home/vagrant/.local/lib/python3.7/site-packages/ansible
   executable location = /home/vagrant/.local/bin/ansible
   python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]

On to the next step, the installation of the OCI Python SDK.

OCI Python SDK

The installation is straight forward as well. To stay in line with the vast majority of bloggers describing the installation I decided to create a virtualenv environment.

vagrant@ocidev:~$ virtualenv python_oci_sdk
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/vagrant/python_oci_sdk/bin/python2
Also creating executable in /home/vagrant/python_oci_sdk/bin/python
Installing setuptools, pkg_resources, pip, wheel…done.
vagrant@ocidev:~$

Once the virtualenv exists it’s time to enable it and install the OCI SDK:

vagrant@ocidev:~$ . python_oci_sdk/bin/activate

(python_oci_sdk) vagrant@ocidev:~$ pip3 install oci
Collecting oci 
...
Successfully installed cffi-1.14.3 configparser-4.0.2 cryptography-2.8 
oci-2.23.4 pyOpenSSL-19.1.0 pycparser-2.20 python-dateutil-2.8.1 
pytz-2020.4

After the installation completed, you need to ensure you have a working configuration as explained in the official documentation set. There are many blog posts out there describing the process, and I won’t repeat it here. I found a post of a colleague of mine, Christoph, to be really good. And there are many more other good posts out there, too!

Ansible Galaxy

Before I can make use of the Dynamic Inventory Plugin I need to install the oracle.oci collection from Ansible galaxy:

(python_oci_sdk) vagrant@ocidev:~$ ansible-galaxy collection install oracle.oci
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Installing 'oracle.oci:2.9.0' to '/home/vagrant/.ansible/collections/ansible_collections/oracle/oci'
Downloading https://galaxy.ansible.com/download/oracle-oci-2.9.0.tar.gz to /home/vagrant/.ansible/tmp/ansible-local-1723ne90h04l/tmprgpltva8
oracle.oci (2.9.0) was installed successfully
(python_oci_sdk) vagrant@ocidev:~$

This concludes the preparation.

Configuring the Ansible Dynamic Environment Plugin

With all the software in place it’s time to configure the Dynamic Inventory plugin. As I said before I was a little confused after having read the documentation.

Configuration files

You will need to create 2 configuration files:

  • The Dynamic Inventory Plugin Configuration File (for lack of a better description)
  • The well-known ansible configuration file (ansible.cfg)

Dynamic Inventory Plugin Configuration File

First of all, you need to create a YAML file as inventory source. According to the docs, in its simplest form, all it requires is this line (spoiler: we will put a a lot more into the file):

plugin: oracle.oci.oci

However, it’s not quite as simple. The file name you save this as has to follow a specific form, or else you get an error:

(python_oci_sdk) vagrant@ocidev:~/ansible$ ansible-inventory --graph -vvv -i oci.yml
ansible-inventory 2.10.3
config file = /home/vagrant/ansible/ansible.cfg

[...]

Using /home/vagrant/ansible/ansible.cfg as config file
ansible_collections.oracle.oci.plugins.inventory.oci 
declined parsing /home/vagrant/ansible/oci.yml as it did not pass its verify_file() method

Name your file in the form someIdentifier.oci.yml, only changing someIdentifier, leave the rest as it is.

There are many more things to put into the config file as you will see in a minute, we aren’t done yet :)

Ansible.cfg

For the above YAML file to have any effect you need a local ansible.cfg file white-listing the use of the plugin. My local ansible.cfg has the following contents:

(python_oci_sdk) vagrant@ocidev:~/ansible$ cat ansible.cfg 
[defaults]
stdout_callback = debug

[inventory]
enable_plugins = oracle.oci.oci

Actually only the part under the inventory tag is important. I do like to have some more human readable output which is why I change the stdout_callback.

Caveats

The use of the Dynamic Inventory plugin is a great time saver, but comes with a certain risk. If you aren’t careful, you can end up running playbooks against far too many hosts. Clever Identity and Access Management (IAM) is a must to prevent accidents. And don’t ever use hosts: all in your playbooks! Principle of least privilege is key.

I always follow a principle of least privilege, and created a new IAM user for this blog post. The user merely has the privileges to manage resources in his own compartment, everything else is off limits. In addition I’m filtering on the compartment and the project’s tags.

Limiting scope to a specific compartment

Now I need access to the docs: the Oracle inventory plugin is documented in the web, but thankfully there is also a version for consumption at the console window:

(python_oci_sdk) vagrant@ocidev:~/ansible$ ansible-doc -t inventory oracle.oci.oci | head -n 10
> ORACLE.OCI.OCI    (/home/vagrant/.ansible/collections/ansible_collections/oracle/oci/plugins/inventory/oci.py)

        Get inventory hosts from oci. Uses a .oci.yaml (or
        .oci.yml) YAML configuration file.

OPTIONS (= is mandatory):

- cache
        Toggle to enable/disable the caching of the inventory's source
        data, requires a cache plugin setup to work.

...

Using the docs I came up with the following inventory source:

(python_oci_sdk) vagrant@ocidev:~/ansible$ cat ougdemo-compartment.oci.yml 
plugin: oracle.oci.oci

hostname_format: "fqdn"

filters:
 defined_tags: { "project": { "name": "simple-app" } } 

regions:
- eu-frankfurt-1

compartments:
- compartment_ocid: "ocid1.compartment.oc1..aa...a"
  fetch_hosts_from_subcompartments: true
(python_oci_sdk) vagrant@ocidev:~/ansible$ 

With the inventory in place as shown, it’s possible to graph the resources:

(python_oci_sdk) vagrant@ocidev:~/ansible$ ansible-inventory -i ougdemo-compartment.oci.yml --graph
...
@all:
  |--@IHsr_EU-FRANKFURT-1-AD-2:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@IHsr_EU-FRANKFURT-1-AD-3:
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedBy=ougdemouser:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedOn=2020-11-08T10_35_08.490Z:
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedOn=2020-11-08T10_35_08.765Z:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |--@Oracle-Tags#CreatedOn=2020-11-08T10_35_08.896Z:
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@all_hosts:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@ougdemo-department:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@project#name=simple-app:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@region_eu-frankfurt-1:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@tag_role=appserver:
  |  |--appserver1.app.simpleapp.oraclevcn.com
  |  |--appserver2.app.simpleapp.oraclevcn.com
  |--@tag_role=bastionhost:
  |  |--bastion1.bastion.simpleapp.oraclevcn.com
  |--@ungrouped:
(python_oci_sdk) vagrant@ocidev:~/ansible$  

Note how the use of tags (both free form as well as defined) allows me to group resources easily. Tagging is really important in the cloud ;)

Summary

The Ansible Dynamic Inventory Plugin is a great time saver, allowing me to create and configure Infrastructure as Code without having to manually update an Ansible inventory file each time a Terraform script completed. Using tags and appropriate Identity and Access Management (IAM)/security/access control policies in conjunction with a suitable filter configuration in the config file it is possible to manage cloud resources with very little manual effort.

A failure to adhering to industry best known methods comes at the risk of running Ansible playbooks against the wrong hosts, something that has to be ruled out by means of a secure configuration.

Oracle Cloud Infrastructure: using Network Security Groups and the caveat with the subnet’s default security list

This is going to be one of these posts I’m mainly writing to myself, in the hope that a) I don’t forget about that topic too soon and b) someone might have the same question and doesn’t want to spin up an environment to find out.

Broadly speaking Oracle Cloud Infrastructure (and some other cloud providers) give you 2 different means of securing the Virtual Cloud Network at the VCN level:

After I got to grips with the latter when Oracle released them, I have used them a lot, for reasons outlined in the documentation. So why this post? It’s to raise awareness that you might feel secure thanks to a Network Security Group (NSG) when in fact you aren’t.

Obligatory Cloud Post Warning: if you create resources described in this post you might incur cost.

It’s easier to show with an example

Assume you created a VCN with 1 public, regional subnet for your admin host, running Oracle Linux 7. You didn’t create a VPN to connect to the VCN yet. However, you created a NSG for the admin host only allowing your IP to access the admin host. The admin host’s VNIC is assigned to the NSG. Like in this Terraform example, shortened to include only the necessary resources:

#
# VCN
#
resource "oci_core_vcn" "simplevm_vcn" {

  cidr_block     = var.simplevm_vcn_cidr_block
  compartment_id = var.compartment_ocid
  display_name   = "simplevm VCN"
  dns_label      = "simplevm"

  defined_tags   = { "project.name" = "simple-vm" }
}

#
# NSG + SSH rule
#
resource "oci_core_network_security_group" "simplevm_bastion_nsg" {
  compartment_id = var.compartment_ocid
  vcn_id         = oci_core_vcn.simplevm_vcn.id
  display_name   = "Simple VM bastion NSG"
  defined_tags   = { "project.name" = "simple-vm" }
}

resource "oci_core_network_security_group_security_rule" "simplevm_bastion_nsg_ssh_rule" {
  network_security_group_id = oci_core_network_security_group.simplevm_bastion_nsg.id
  description               = "ssh"
  direction                 = "INGRESS"
  protocol                  = 6
  source_type               = "CIDR_BLOCK"
  source                    = var.controlhost

  tcp_options {
    destination_port_range {
      min = 22
      max = 22
    }
  }
}

#
# Subnet
#
resource "oci_core_subnet" "simplevm_vcn_bastion_sn" {
  cidr_block     = var.simplevm_vcn_bastion_sn_cidr_block
  compartment_id = var.compartment_ocid
  vcn_id         = oci_core_vcn.simplevm_vcn.id

  display_name   = "Simple VM bastion SN"
  dns_label      = "simplevm"

  defined_tags   = { "project.name" = "simple-vm" }

  prohibit_public_ip_on_vnic = false
  route_table_id             = oci_core_route_table.simplevm_igw_rt.id

} 

The big question must be: is the admin host accessible via SSH from everywhere, or merely from whatever is the value stored in var.controlhost? You’ll have to take my word for it, it’s a /32 address ;)

#
# simple VM
#
resource "oci_core_instance" "simplevm" {
  availability_domain = var.availability_domain
  compartment_id      = var.compartment_ocid
  shape               = "VM.Standard.E2.1.Micro"

  defined_tags        = { "project.name" = "simple-vm" }

  create_vnic_details {

    assign_public_ip = true
    display_name     = "simplevm"
    hostname_label   = "simplevm"
    nsg_ids = [
      oci_core_network_security_group.simplevm_bastion_nsg.id
    ]
    subnet_id = oci_core_subnet.simplevm_vcn_bastion_sn.id
  }

...

} 

Unfortunately “simplevm” is accessible from everywhere, thanks to the Default Security List implicitly created with the public subnet. It opens port 22 to the world for incoming traffic. The way Security Lists and Network Security work, is described in the Security Rules section of the documentation:

A packet in question is allowed if any rule in any of the relevant lists and groups allows the traffic…

— aforementioned Oracle Cloud Infrastructure documentation

Clearly the default security list matches with its ingress rule of 0.0.0.0/0 to port 22, and traffic is allowed to reach the VM.

Summary

So with that said, in scenarios like the one shown above, please ensure you either change the default security list, or create a separate one and assign it to the subnet.

Better still, ensure you use a more secure approach to accessing your cloud network. There are many ways to do so, in the case of Oracle you find a dedicated module in the free OCI foundation training. Under “Connectivity” you will find both a video as well as slides.

The use of bastion hosts is frowned upon in some circles, and that sounds about right to me.

Upgrading oraclelinux-release-el7 might trigger an upgrade to UEK Release 6

While building a demo environment for an upcoming presentation I noticed an upgrade from UEK 5 to UEK 6 on my Oracle Linux 7 VM. As it turned out, the kernel change has been triggered by an upgrade of oraclelinux-release-el7 RPM. I am a great fan of Oracle’s UEK and the team behind it, so this is a welcome change for me. It might however meet you unprepared, which is why I put this little article together.

Whether or not the contents of my article applies to you depends on your yum configuration, and the source of your packages.

Background

Oracle split the monolithic public-yum.repo file into more manageable parts a little while ago (see for example my older post here). The bulk of the files in /etc/yum.repos.d is managed by the aforementioned oraclelinux-release-el7 package:

[opc@simplevm ~]$ rpm -ql oraclelinux-release-el7
/etc/yum.repos.d/oracle-linux-ol7.repo
/etc/yum.repos.d/uek-ol7.repo
/etc/yum.repos.d/virt-ol7.repo
/usr/bin/ol_yum_configure.sh
/usr/share/doc/oraclelinux-release-el7
/usr/share/doc/oraclelinux-release-el7/LICENSE
[opc@simplevm ~]$  

To prove the point I spun up a VM in Oracle Cloud Infrastructure based on the latest Oracle Linux image at the time of writing: Oracle-Linux-7.8-2020.09.23-0. The image is based on Oracle Linux 7.8 with fixes up to September 23rd. It ships with UEK5 4.14.35-1902.306.2.el7uek.x86_64.

The following yum repositories are configured when the machine comes up:

[opc@simplevm ~]$ sudo yum repolist 
Loaded plugins: langpacks, ulninfo
repo id                         repo name                                                                         status
ol7_UEKR5/x86_64                Latest Unbreakable Enterprise Kernel Release 5 for Oracle Linux 7Server (x86_64)     217
ol7_addons/x86_64               Oracle Linux 7Server Add ons (x86_64)                                                467
ol7_developer/x86_64            Oracle Linux 7Server Development Packages (x86_64)                                 1,611
ol7_developer_EPEL/x86_64       Oracle Linux 7Server EPEL Packages for Development (x86_64)                       33,126
ol7_ksplice                     Ksplice for Oracle Linux 7Server (x86_64)                                          8,957
ol7_latest/x86_64               Oracle Linux 7Server Latest (x86_64)                                              20,917
ol7_oci_included/x86_64         Oracle Software for OCI users on Oracle Linux 7Server (x86_64)                       627
ol7_optional_latest/x86_64      Oracle Linux 7Server Optional Latest (x86_64)                                     15,257
ol7_software_collections/x86_64 Software Collection Library release 3.0 packages for Oracle Linux 7 (x86_64)      15,366
repolist: 96,545
[opc@simplevm ~]$  

The effect of upgrading oraclelinux-release-el7

To cut a long story short let’s update oraclelinux-release-el7:

[opc@simplevm ~]$ rpm -q oraclelinux-release-el7
oraclelinux-release-el7-1.0-12.1.el7.x86_64
[opc@simplevm ~]$ sudo yum upgrade oraclelinux-release-el7
Loaded plugins: langpacks, ulninfo
Resolving Dependencies
--> Running transaction check
---> Package oraclelinux-release-el7.x86_64 0:1.0-12.1.el7 will be updated
---> Package oraclelinux-release-el7.x86_64 0:1.0-13.1.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================
 Package                               Arch                 Version                      Repository                Size
========================================================================================================================
Updating:
 oraclelinux-release-el7               x86_64               1.0-13.1.el7                 ol7_latest                19 k

Transaction Summary
========================================================================================================================
Upgrade  1 Package

Total download size: 19 k
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
oraclelinux-release-el7-1.0-13.1.el7.x86_64.rpm                                                  |  19 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : oraclelinux-release-el7-1.0-13.1.el7.x86_64                                                          1/2 
warning: /etc/yum.repos.d/oracle-linux-ol7.repo created as /etc/yum.repos.d/oracle-linux-ol7.repo.rpmnew
  Cleanup    : oraclelinux-release-el7-1.0-12.1.el7.x86_64                                                          2/2 
  Verifying  : oraclelinux-release-el7-1.0-13.1.el7.x86_64                                                          1/2 
  Verifying  : oraclelinux-release-el7-1.0-12.1.el7.x86_64                                                          2/2 

Updated:
  oraclelinux-release-el7.x86_64 0:1.0-13.1.el7                                                                         

Complete!
[opc@simplevm ~]$  

Once that’s done, let’s compare the list of enabled repositories:

[opc@simplevm ~]$ sudo yum repolist
Loaded plugins: langpacks, ulninfo
ol7_UEKR6                                                                                        | 2.8 kB  00:00:00     
ol7_addons                                                                                       | 2.8 kB  00:00:00     
ol7_latest                                                                                       | 3.4 kB  00:00:00     
ol7_optional_latest                                                                              | 2.8 kB  00:00:00     
(1/2): ol7_UEKR6/x86_64/updateinfo                                                               |  52 kB  00:00:00     
(2/2): ol7_UEKR6/x86_64/primary_db                                                               | 7.6 MB  00:00:00     
repo id                         repo name                                                                         status
ol7_UEKR6/x86_64                Latest Unbreakable Enterprise Kernel Release 6 for Oracle Linux 7Server (x86_64)     174
ol7_addons/x86_64               Oracle Linux 7Server Add ons (x86_64)                                                467
ol7_developer/x86_64            Oracle Linux 7Server Development Packages (x86_64)                                 1,611
ol7_developer_EPEL/x86_64       Oracle Linux 7Server EPEL Packages for Development (x86_64)                       33,126
ol7_ksplice                     Ksplice for Oracle Linux 7Server (x86_64)                                          8,957
ol7_latest/x86_64               Oracle Linux 7Server Latest (x86_64)                                              20,917
ol7_oci_included/x86_64         Oracle Software for OCI users on Oracle Linux 7Server (x86_64)                       627
ol7_optional_latest/x86_64      Oracle Linux 7Server Optional Latest (x86_64)                                     15,257
ol7_software_collections/x86_64 Software Collection Library release 3.0 packages for Oracle Linux 7 (x86_64)      15,366
repolist: 96,502 

If you look carefully you notice the absence of the previously present ol7_UEKR5 repository, which has been replaced by ol7_UEKR6. If you were to upgrade kernel-uek – after all there are more recent 4.14.x kernels out there – you might be surprised:

[opc@simplevm ~]$ sudo yum upgrade kernel-uek
Loaded plugins: langpacks, ulninfo
Resolving Dependencies
--> Running transaction check
---> Package kernel-uek.x86_64 0:5.4.17-2011.7.4.el7uek will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================
 Package                   Arch                  Version                                 Repository                Size
========================================================================================================================
Installing:
 kernel-uek                x86_64                5.4.17-2011.7.4.el7uek                  ol7_UEKR6                 58 M

Transaction Summary
========================================================================================================================
Install  1 Package

Total download size: 58 M
Installed size: 65 M 

Had I hit return at this stage, my system would have upgraded UEK5 to UEK6.

This is exactly the upgrade I would have done, but you might not. UEK Release 6 is still relatively fresh out there, and you should dobule-check if your Oracle software is certified with it. This applies specifically to ASM Filter Driver, ASM Cluster File System (ACFS) and ASMLib. To stick with UEK Release 5 for now you can either change the yum configuration in /etc/yum.repos.d/uek-ol7.repo or alternatively call yum-config-manager.

Since I tend to perform all these tasks with Ansible, I simply go with the proven enablerepo/disablerepo attributes:

  - name: update all packages (remain on the UEK5 branch)
    yum:
      name: '*'
      state: latest
      enablerepo: ol7_UEKR5
      disablerepo: ol7_UEKR6
      update_cache: yes 

A similar approach is possible on the command line:

[opc@simplevm ~]$ sudo yum upgrade kernel-uek --enablerepo ol7_UEKR5 --disablerepo ol7_UEKR6
Loaded plugins: langpacks, ulninfo
Resolving Dependencies
--> Running transaction check
---> Package kernel-uek.x86_64 0:4.14.35-2025.401.4.el7uek will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================
 Package                  Arch                 Version                                    Repository               Size
========================================================================================================================
Installing:
 kernel-uek               x86_64               4.14.35-2025.401.4.el7uek                  ol7_UEKR5                53 M

Transaction Summary
========================================================================================================================
Install  1 Package

Total download size: 53 M
Installed size: 58 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
kernel-uek-4.14.35-2025.401.4.el7uek.x86_64.rpm                                                  |  53 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : kernel-uek-4.14.35-2025.401.4.el7uek.x86_64                                                          1/1 

...

  Verifying  : kernel-uek-4.14.35-2025.401.4.el7uek.x86_64                                                          1/1 

Installed:
  kernel-uek.x86_64 0:4.14.35-2025.401.4.el7uek                                                                         

Complete!
[opc@simplevm ~]$  

This way I remained on UEK5.

Summary

By enabling the UEK5 repository while at the same time disabling the UEK6 repository I can remain on the Release 5 branch for now, until my Oracle Linux 7.x systems are generally ready for UEK Release 6.

Happy patching!

Introducing packer: building immutable infrastructure in the cloud

After having spent a bit of time with Packer to create Vagrant base boxes it was time to focus on the cloud. I have referenced Packer multiple times in my cloud talks as a popular way to create immutable infrastructure and/or custom images, and I wanted to share how you could potentially make use of the technology

Be aware that this is a post about cloud technology. If you are following along, be aware that you will incur cost.

The case for using Packer

Saying it with my words, Packer is a tool allowing me to build immutable infrastructure. It does so by using a base image a cloud provider offers, spinning it up temporarily and installing the software you want to use. The result is then converted to a custom image for later use.

Think for example you would like to deploy Oracle 19.8.0 binaries across your cloud infrastructure. One way to get there would be to create a VM, followed by running Ansible playbooks to install Oracle 19.8.0. That’s a perfectly fine approach! However, bear in mind that every execution of an Ansible playbooks might fail, and it certainly takes a bit of time for the playbook to finish.

The alternative is to create a custom image with Oracle 19.8.0 already installed. f you want to be absolutely sure all of your deployments are identical, Packer might be the tool for you. Deploying a custom image should take a lot less time than the VM + Ansible approach, too. And once you validated the custom image by means of rigorous regression testing, it should provide a lot of confidence. A long time ago many of us tried to achieve the same goal by using custom RPMs for example, a much less reliable approach compared to what we can do these days.

Oh, and by the way: regardless which way you choose to deploy Oracle software, you have to always ensure you are license compliant.

Let’s give it a try!

In this post I won’t install Oracle 19.8.0, I’ll stick with XE to keep it reasonably simple. The overall process is identical though, which should allow you to plug your Ansible playbooks into the Packer template with ease.

My build environment is based on Packer 1.6 running on top of Ubuntu 20.04 LTS. I am using Oracle Cloud Infrastructure.

Building Oracle XE in Packer

The process is similar to what I described in my first post about Packer, so please head over there for additional background. Unlike building a Virtualbox VM based on a local ISO image, Packer has to know which base OS image to use. It also has to be able to spin it up, so a Virtual Cloud Network as well as a subnet with appropriate security lists are needed. I have written how to create these via Terraform in a separate post.

Most examples about Oracle OCI and Packer don’t appear overly secure to me: you find username/password combinations used pretty much everywhere. I prefer to stick to SSH keys instead.

When configuring the JSON template for Packer you can make use of an existing Oracle Cloud CLI configuration file, or use variables very similar to those the Terraform provider for OCI requires as well. I don’t need a local installation of the OCI CLI since the cloud shell became GA so I went with the latter approach.

To save me some work I’m using the same shell variables in Packer as I do in Terraform. Packer 1.6 can read environment variables when invoked, but as far as I understand it only does so in the “variables” section. This leads to funny looking code, but it works. It might of course be so that I’m wrong, and I’ll happily stand corrected. Please let me know!

Packer Template

Let’s get started with the Packer JSON template. It is essentially a faithful implementation of the documented options of the oci-oracle builder.

 
{
    "variables": {
        "oci_tenancy_ocid": "{{env `TF_VAR_tenancy_ocid`}}",
        "oci_user_oci": "{{env `TF_VAR_user_ocid`}}",
        "oci_compartment_ocid": "{{env `TF_VAR_compartment_ocid`}}",
        "oci_api_key_fingerprint": "{{ env `TF_VAR_fingerprint`}}",
        "oci_api_key_file": "{{ env `TF_VAR_private_key_path`}}",
        "oci_region": "eu-frankfurt-1"
    },
    "builders": [
        {
            "type": "oracle-oci",
            "tenancy_ocid": "{{user `oci_tenancy_ocid`}}",
            "user_ocid": "{{user `oci_user_oci`}}",
            "compartment_ocid": "{{user `oci_compartment_ocid`}}",
            "fingerprint": "{{ user `oci_api_key_fingerprint`}}",
            "key_file": "{{user `oci_api_key_file`}}",

            "region": "{{user `oci_region`}}",
            "availability_domain": "IHsr:EU-FRANKFURT-1-AD-1",
            "subnet_ocid": "ocid1...",

            "base_image_ocid": "ocid1.image.oc1.eu-frankfurt-1.aaaaaaaahxue6crkdeevk75bzw63cmhh3c4uyqddcwov7mwlv7na4lkz7zla",
            "image_name": "oracle-xe-ol7.8",
            "shape": "VM.Standard.E2.1.Micro",
            "ssh_username": "opc",
            "ssh_agent_auth": true,
            "ssh_timeout": "10m",
            "metadata" : {
                "ssh_authorized_keys" : "ssh-rsa ... key"
            }
        }
    ],
    "provisioners": [
        {
            "type": "ansible",
            "user": "opc",
            "playbook_file": "ansible/xe.yml",
            "extra_arguments": [ "-v" ]
        }
    ]
} 

Let’s have a look at the various sections in the Template.

Variables

The first section, “variables”, allows me to read environment variables. I’m using the same environment variables for Packer and Terraform, hence the somewhat strange looking names. The variables section populates Packer variables I prefixed “oci”.

Builders

The builders section is where the actual action starts. I need to provide the usual suspects (tenancy OCID, user OCID, API key location and fingerprint etc) as per the documentation. The first part of the “builders” section should be self-explanatory. Well maybe apart from the fact I couldn’t use environment variables in this part of the template, hence the reference to my user variables declared earlier. Again, if there is a better way of doing this, please let me know.

The middle part of the code is concerned with resource location. As I live in Germany the easiest option is to use EU-Frankfurt-1. One of the pre-requisites I need to complete before Packer can go and do its magic is to define a public “build” subnet. I have an automatically, pre-created separate build network (VCN) for this purpose. All I need is to grab the OCID for my subnet and provide it to Packer.

The third part of the builders section is where all the fun starts:

  • I chose the base image to be the latest Oracle Linux 7.8 image at the time of writing. More information about Oracle Cloud IaaS images and their corresponding OCIDs can be found in the official documentation reference
  • The new custom image to be created is to be named oracle-xe-ol7.8
  • I would like the VM shape for the build to be VM.Standard.E2.1.Micro

As with my Vagrant base box I’ll use SSH authentication rather than a more insecure username/password combination. To do so I need to use the local ssh agent to store and forward my key to the VM. I’m providing that exact key via the metadata directive. Once the VM has been created, opc’s authorized_keys file contains my SSH key allowing me to connect.

Provisioners

In comparison to the earlier section this is rather uneventful. The only action it performs is to call the Ansible playbook responsible for the installation of Oracle XE. The playbook is too boring to show it here, all it does is update all RPMs before it installs the Oracle XE RPM.

Thanks to the SSH settings provided earlier no passwords have to go over the wire. If the provisioner fails, you most likely didn’t add your SSH key to the agent. Like I did ;)

==> oracle-oci: Waiting for SSH to become available...
==> oracle-oci: Error waiting for SSH: Packer experienced an authentication error when trying to connect via SSH. 
 This can happen if your username/password are wrong. You may want to double-check your credentials as 
 part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods 
 [publickey none], no supported methods remain
==> oracle-oci: Terminating instance 

This is super easy to fix: once the prompt returns you add the key to the agent using ssh-add and restart the build.

End result

After a little while, Packer build will finish and create a new custom image with its own OCID for use in later deployments.

$ ANSIBLE_STDOUT_CALLBACK=debug packer build oracle-xe-oci.json 
oracle-oci: output will be in this color.

==> oracle-oci: Creating temporary ssh key for instance...
==> oracle-oci: Creating instance...
==> oracle-oci: Created instance (ocid1.instance.oc1.eu-frankfurt-1.ant...).
==> oracle-oci: Waiting for instance to enter 'RUNNING' state...
==> oracle-oci: Instance 'RUNNING'.
==> oracle-oci: Instance has IP: 11.22.33.44.
==> oracle-oci: Using ssh communicator to connect: 11.22.33.44
==> oracle-oci: Waiting for SSH to become available...
==> oracle-oci: Connected to SSH!
==> oracle-oci: Provisioning with Ansible...
    oracle-oci: Setting up proxy adapter for Ansible....
==> oracle-oci: Executing Ansible: ansible-playbook -e packer_build_name="oracle-oci"...
    oracle-oci: Using /etc/ansible/ansible.cfg as config file
    oracle-oci:
    oracle-oci: PLAY [all] *********************************************************************

[...]

    oracle-oci:
    oracle-oci: PLAY RECAP *********************************************************************
    oracle-oci: default : ok=11   changed=9    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
    oracle-oci:
==> oracle-oci: Creating image from instance...
==> oracle-oci: Image created.
==> oracle-oci: Terminating instance (ocid1.instance.oc1.eu-frankfurt-1.aaa)...
==> oracle-oci: Terminated instance.
Build 'oracle-oci' finished.

==> Builds finished. The artifacts of successful builds are:
--> oracle-oci: An image was created: 'oracle-xe-ol7.8' (OCID: ocid1.image.oc1.eu-frankfurt-1.aaa) in region 'eu-frankfurt-1' 

As you can see, a new custom image has been created. You can read more about using custom images in a later post.

Summary

Compared to my first attempt at using Packer to build a Vagrant base box, using Packer with Oracle Cloud Infrastructure turned out to be a lot easier. The only tricky bit is the use of SSH, however that’s quickly overcome if you ever used the OCI Terraform provider and know about the metadata directive.

Bootstrapping a VM image in Oracle Cloud Infrastructure using cloud-init

At the time of writing Oracle’s Cloud Infrastructure as a Service (IaaS) offers 2 ways to connect block storage to virtual machines: paravirtualised and via iSCSI. There are important differences between the two so please read the documentation to understand all the implications. I need all the performance I can get with my systems so I’m going with iSCSI.

It’s the little differences

Using the paravirtualised driver couldn’t be easier: you boot the VM, and all block devices are automatically attached and available. When using iSCSI you need to run a few iscsiadm commands (once) to discover and mount the remote storage. These commands are available on the click of a button in the GUI. It’s been ages that I used the GUI and I prefer a scripted approach to cloud infrastructure. My tool of choice when it comes to “infrastructure as code” is terraform

Until fairly recently I have made use of the null provider combined with a remote-exec provisioner in my terraform scripts. The combination allows me to execute the iscsiadm commands necessary to attach the iSCSI devices to the VM. A number of enhancements in this space allowed me to ditch the rather cumbersome remote-exec step and use cloud-init combined with OCI utilities instead. As I hope to show you, using these two combined make the management of iSCSI device just as simple as the paravirtualised ones.

Cloud Init

When creating VMs I often need to perform a few extra steps that don’t quite justify the creation of a custom image. The cloud-init toolkit in OCI allows me to pass a shell script as “user_data” to the instance’s metadata, provided it’s encoded in base64. Have a look at the documentation I just referenced for more details about restrictions etc. In my terraform script, I use something like this:

resource "oci_core_instance" "docker_tf_instance" {
[...]
   
    metadata {
        ssh_authorized_keys = "${var.ssh_public_key}"
        user_data = "${base64encode(file("bootstrap.sh"))}"
    }

[...]
}

Most examples I found specify the input to the file() function as a variable, I didn’t do this in this post for the sake of simplicity. The script I’m passing as user_data makes use of the OCI utilities.

OCI Utilities

I wasn’t aware of these until my colleague Syed asked me why I didn’t use them. It couldn’t be easier: just install a RPM package and start a service. This will take care of the iSCSI part for you. The only caveat is that currently they can only be used for Oracle provided images based on Oracle Linux. Here is a really basic example of a shell script calling the OCI utilities:

$ cat bootstrap.sh 
#!/bin/bash

cp /etc/motd /etc/motd.bkp
cat << EOF > /etc/motd

I have been modified by cloud-init at $(date)

EOF

yum install -y python-oci-cli
systemctl enable ocid.service
systemctl start ocid.service
systemctl status ocid.service

The first line has to start with #!/bin/bash to indicate to cloud-init that you want to run a shell script. Following the instructions for using OCI utilities, I am installing the python-oci-cli and start the ocid.service. This in turn will perform the iSCSI volume attachment for me – super nice! After my terraform script completed, I can log in to see if this worked:

[root@docker-tf-instance ~]# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdb      8:16   0   50G  0 disk 
sda      8:0    0 46.6G  0 disk 
├─sda2   8:2    0    8G  0 part [SWAP]
├─sda3   8:3    0 38.4G  0 part /
└─sda1   8:1    0  200M  0 part /boot/efi
[root@docker-tf-instance ~]# 

[root@docker-tf-instance ~]# systemctl status ocid.service
● ocid.service - Oracle Cloud Infrastructure utilities daemon
   Loaded: loaded (/etc/systemd/system/ocid.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2018-11-27 19:52:45 GMT; 19min ago
 Main PID: 15138 (python2.7)
   CGroup: /system.slice/ocid.service
           └─15138 python2.7 /usr/libexec/ocid

Nov 27 19:52:04 docker-tf-instance python2.7[15138]: ocid - INFO - Starting ocid thread 'iscsi'
Nov 27 19:52:04 docker-tf-instance python2.7[15138]: ocid - INFO - Starting ocid thread 'vnic'
...
Nov 27 19:52:09 docker-tf-instance python2.7[15138]: ocid - INFO - secondary VNIC script reports: Info: no changes, IP configuration is up-to-date
Nov 27 19:52:44 docker-tf-instance python2.7[15138]: ocid - INFO - Attaching iscsi device: 169.254.a.b:3260 (iqn.2015-12.com.oracleiaas:e1af1...)
Nov 27 19:52:45 docker-tf-instance systemd[1]: Started Oracle Cloud Infrastructure utilities daemon.

You can see cloud-init in action by checking /var/log/messages for occurrences of “cloud-init”. The file /var/log/cloud-init.log doesn’t contain information relevant to the “user-data” processed by the way. If you want to see how your script arrived on the VM, check /var/lib/cloud/instance/user-data.txt.

Summary

It would seem you can have the cake and eat it. Using cloud-init for bootstrapping my VM and OCI utilities to attach my block devices I don’t need to write any remote-exec hacks using the null provider and use the iSCSI volumes with the same ease of use as the paravirtualised ones. Without having to make compromises. I like it!

Log in to Ubuntu VMs in Oracle Cloud Infrastructure

When I learned that Oracle was providing Ubuntu images in Oracle Cloud Infrastructure (OCI) I was a bit surprised at first. After all, Oracle provides a great Enterprise Linux distribution in the form of Oracle Linux. As a Ubuntu fan I do of course appreciate the addition of Ubuntu to the list of supported distributions. In fact it doesn’t end there, have a look at the complete list of Oracle provided images to see what’s available.

Trying Ubuntu LTS

I wanted to give Ubuntu a spin on OCI and decided to start a small VM using the 16.04 LTS image. I have been using this release quite heavily in the past and have yet to make the transition to 18.04. Starting the 16.04 VM up was easily done using my terraform script. Immediately after the terraform prompt returned I faced a slight issue: I couldn’t log in:

$ ssh opc@w.x.y.z
The authenticity of host ... can't be established.
...
opc@w.x.y.z: Permission denied (publickey)

This is entirely my fault, for some reason I didn’t scroll down within the page to read more about users. Assuming the account created during the VM provisioning would be the same as for the Oracle Linux image, I tried logging in as user “opc”. The result is what I showed you earlier in the listing.

The clue about users is found in Linux Image Details, section “users” and aforementioned documentation page. I am quoting verbally because I couldn’t possibly say it any better:

For instances created using the Ubuntu image, the user name ubuntu is created automatically. The ubuntu user has sudo privileges and is configured for remote access over the SSH v2 protocol using RSA keys. The SSH public keys that you specify while creating instances are added to the /home/ubuntu/.ssh/authorized_keys file.

There it is.

It seems I wasn’t the only one, and beginning with the Canonical-Ubuntu-16.04-2018.11.15-0 image, a message is displayed when you try to log in as opc:

$ ssh opc@w.x.y.z
...
Warning: Permanently added ... to the list of known hosts
Please login as the user "ubuntu" rather than the user "opc".

Connection to w.x.y.z closed
$ 

So no more missing this important piece of information :)

Terraforming the Oracle Cloud: choosing and using an image family

For a few times now I have presented about “cloud deployments done the cloud way”, sharing lessons learned in the changing world I find myself in. It’s a lot of fun and so far I have been far too busy to blog about things I learned by trial and error. Working with Terraform turned out to be a very good source for blog posts, I’ll put a few of these up in the hope of saving you a few minutes.

This blog post is all about creating Ubuntu images in Oracle Cloud Infrastructure (OCI) using terraform. The technique is equally applicable for other Linux image types though. In case you find this post later using a search engine, here is some version information that might put everything into context:

$ ./terraform version
Terraform v0.11.10
+ provider.null v1.0.0
+ provider.oci v3.7.0

I used the “null” provider to run a few post-installation commands as shown in various terraform examples for OCI. Right now I’m trying to work out if I can’t do the same in a different way. If I am successful you can expect a blog post to follow…

Creating a Ubuntu 18.04 LTS image in OCI

To create the Ubuntu image (or any other image for that matter), I need information about the image family. Documentation about image families in OCI can be found at https://docs.cloud.oracle.com/iaas/images/.

Scrolling down/selecting the entry from the left hand side I found the link to the Ubuntu 18.04 LTS image family. Each supported image has its own documentation link, containing crucial data: an OCID per location. At the time of writing, the latest Ubuntu image was named Canonical-Ubuntu-18.04-2018.11.17-0 and had an image OCID of ocid1.image.oc1.eu-frankfurt-1.aaaa...i57q7bfsa. An OCID is short for Oracle Cloud Identifier and it’s used in many places in OCI. There are different OCIDs for the image depending on location; the (shortened) OCID I just showed you was for Frankfurt.

With the OCID at hand, I can open my favourite code editor and start putting the terraform script together. I create instances in OCI using the oci_core_instance type, documented at the terraform website.

Be careful, many of the references and code examples I found about oci_core_image are written for older versions of the terraform provider. I noticed some attributes used in the examples are deprecated. It might be useful to compare the source code examples against the current documentation

Part of the definition of an oci_core_instance requires the specification of the operating system in the source_details {} section. To create the Ubuntu VM in the Frankfurt region, I have to specify – amongst other things of course – this:

resource "oci_core_instance" "docker_tf_instance" {
...
    source_details {
        source_type = "image"
        source_id   = "ocid1.image.oc1.eu-frankfurt-1.aaaa..."
...
    }
...

The actual OCID is far longer, the example above is shortened for the sake of readability. I didn’t like it wrapping around the text box and thus destroying my layout. Make sure you use the correct OCID ;)

With the information at hand I can create the Ubuntu VM and connect to it using the specified SSH key. Have fun!