16 steps to a fully high available Red Hat OpenStack Platform environment with ceph build by OSP Director 7.1

This post is going to walk you through a full HA Network isolated Red Hat OpenStack installation with 3 Controllers, 3 Compute and 3 Ceph OSD nodes in only 16 simple steps.

To be successful with this deployment you will have to follow this network diagram (create the trunks and vlans on the switch) and work with OSP Director 7.1. Working NTP and IPMI access (from director and controllers) is critical for this environment:

OSP7.1 Network IPMI

Ok lets start the installation. All scripts you are going to run are a collection of the steps described in the Red Hat documentation. The scripts can be found here: ospdirectorinstall

Step 1 I assume that you already have a RHEL 7.1 installation. Download the following script.

Step 2 Once downloaded fill out the variables RHNUSER to UNDERCLOUD_DEBUG_BOOL. Note I set DISCOVERY_RUNBENCH_BOOL=false. If you are in a PROD environment this is not recommended as it disables the gathering of host metrics during introspection.

Step 3 Run the script to prepare for the actual undercloud/director software and follow the commands which the script tells you to execute.

Step 4 Download the Red Hat images Deployment Ramdisk for RHEL-OSP director 7.1 , Discovery Ramdisk for RHEL-OSP director 7.1, Overcloud Image for RHEL-OSP director 7.1 and place them in the image folder. You can find those images here:
OSP Images

Step 5 Download all the scripts you’ll need for this install

Step 6 Create the flavor compute, controller, ceph-storage and baremetal

Step 7 Add DNS for the overcloud nodes (You will have to edit your own DNS in neutrondns.sh)

Step 8 Create your instackenv.json file and start introspection. Remember to choose the correct ipmi_type

Please edit the 5_registernodes.sh and add your macaddress,ipmi url,ipmi user, ipmi password, ipmi tool. I am using Joe talerico python script https://github.com/jtaleric/csv-to-instack to create the instackenv.json files.

Step 9 Assign nodes to their profiles. If you used benchmark equals true you can also run the ahc-match instead of assigning the hosts manually.

Step 10 Copy the Director template files from /usr/share/openstack-tripleo-heat-templates/ into your /home/stack/templates/ folder

Step 11 Create the network-environment.yaml file so it maps to your environment. If you do not use lacp remove the bonding option as it might cause problem.

Step 12 Now we download the files fix_rabbit.yaml, limits.yaml, ceph-environment.yaml firstboot-environment.yaml , ceph_wipe.yaml (edit this files so it reflects your disk setup) and place them in the templates and firstboot folder (we are doing some maintenance on first boot. Set the root password of the instances in the firstboot-environment.yaml if you like. The default will be redhat.

Step 13 Make the necessary changes to /home/stack/templates/puppet/hieradata/ceph.yaml. In my case the journal is on the osd. Thats why the journal is blank.

If you want to have the journals on a separate partition your can exchange the ceph::profile::params::osds: with

Step 14 Go ahead and launch the install.

Step 15 As a postinstall task you could enable fencing for the PCS cluster nodes. Repeat this for each cluster/controller node:

Step 16 Once done enable fencing

Enjoy your Full HA 9 node environment.

6 thoughts on “16 steps to a fully high available Red Hat OpenStack Platform environment with ceph build by OSP Director 7.1”

  1. Hello,

    i have 2 bond in control server.nic1,2 =bond 1 and nic3,4 =bond2..could you please prvide me control.yaml file for the same..also share me compute.yaml with one bond using nic1 and nic2..i am stuck with my installation and any help on this is much appreciated…i have inernal network with vlan id 356 and external vlan with 450..can we use the same vlan id of internal network (356) for all remaining network like storage,tenent etc..

  2. Here is an example on how you would add 2 bonds. Also for the vlans 356 add them to network-environment.yaml.


    type: ovs_bond
    name: bond0
    ovs_options: {get_param: BondInterfaceOvsOptions}

    type: interface
    name: nic1
    primary: true

    type: interface
    name: nic2

    type: ovs_bond
    name: bond1
    ovs_options: {get_param: BondInterfaceOvsOptions}

    type: interface
    name: nic3
    primary: true

    type: interface
    name: nic4

  3. Really nice tutorial! 🙂

    One possible error, I think “deploy –templates” in step 14 should be “openstack overcloud deploy –templates” ?

  4. In Step 10 and Step 13 lots of yaml’s are copied over to /home/stack/templates yet when you run openstack overcloud deploy only a very few gets called and non references back to /home/stack/templates/puppet/hieradata/ceph.yaml so is that really necessary or should that be included in the openstack overcloud deploy?

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.