Ceph for Cinder and Glance in OpenStack Juno Part 3 – Building the Ceph Cluster

Part 3 of the post series is going to walk you through how to integrate cinder and glance with ceph on RHEL OSP 6. Most of the instructions on what you need to do can be found here Block Devices and OpenStack

1. Delete the default pools as you won’t need them

2. Create new pools matching the upstream documentation.

3. As I am using a packstack installed OpenStack my controller runs glance-api, cinder-volume and cinder backup. This menas I need to install the following packages on my controller

4. On the compute nodes we need to install the ceph package

5. As we have cephx enabled we need to create the 3 new users for Nova and Cinder

6. The controller needs the following keys (ceph.client.glance.keyring, ceph.client.cinder.keyring, ceph.client.cinder-backup.keyring)

7. For nova compute you will only need the client.cinder.key

8. Ceph was complaining on my installation that the ceph.client.admin.keyring could not be found on the controller. So I had to add it to get it working

9. Create the following /etc/ceph/ceph.conf for the controller. the line rbd default format = 2 is essential if you want to create layered images. Layered images are needed if you need to convert qcow2 to raw in rbd.

10. Copy the ceph config from the controller to the compute nodes and remove the lines client.glance and client.cinder-backup.

11. Add the following line in the ceph.conf client section for the compute nodes. You might need to create the /var/run ceph directory.

12. Next we need to add a secret key to libvirt. The uuid only needs to be created on the first compute host and then copied to the others. the rest of the commands need to run on all hosts.

cat > secret.xml <<EOF
<secret ephemeral='no' private='no'>
  <usage type='ceph'>
    <name>client.cinder secret</name>

13. On the controller edit /etc/glance/glance-api.conf and add the following lines. A complete glance-api.conf can be found here glance-api.conf

14. Now we can configure cinder for ceph/rbd. Add the following lines under the ceph section. A complete cinder.conf can be found here cinder.conf

15. Also make sure that you enable ceph as a backend in cinder.conf

16. Add the following config to nova.conf on the compute nodes

17. And ensure that the live migration options are set correctly

18. Restart the openstack services

19. Create a new cinder type ceph

20. Test if you can create a cinder volume in rbd

If you see your volume here your all good and ready to go to part 4 uploading images to glance via rbd.

1 thought on “Ceph for Cinder and Glance in OpenStack Juno Part 3 – Building the Ceph Cluster”

  1. Pingback: ceph | nnooss

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.