The goal of the ceph \/ OpenStack integration post series is to give you a step by step guide on how you can build a ceph cluster and integrate it with your openstack environment for a POC as well as demonstrate how you can upload qcow2 images into rbd, transform them into raw and import the raw image into glance. This setup is by no means a production setup as I do not have SSD for the ceph journals. Meaning that the journals will reside on the osd’s them self. <\/p>\n
Part 1<\/strong> of these post series is going to walk you through how to get your RHEL 7.1 hosts ready for ceph 1.2.3 and install the calamari web interface on the admin \/ ceph-mgmt node. <\/p>\n For this post series I have the following hardware\/ software available: <\/p>\n For the OpenStack environment<\/strong> For the Ceph environment<\/strong> 1. My POC environment does not need a host based firewall. Therefore I will turn it off. <\/p>\n 2. Create the correct network config for your 10GBE interface in \/etc\/sysconfig\/network-script\/ifcfg-Your_interface_name. It is very important that your ceph nodes are connected with at least 1 10GBE<\/strong> Nic which is available across the cluster and MTU 9000<\/strong> is set. <\/p>\n 3. Disable NetworkManager and set the network service as a default. <\/p>\n 4. Disable selinux as ceph doesn’t like it yet. <\/p>\n 5. If you do not have a DNS server make sure your ceph server has the correct host entries<\/p>\n 6. Register your host to rhn with the correct subscription and attach the following repos.<\/p>\n 7. Update and restart the system<\/p>\n 8. Create the root ssh key on the ceph-mgmt host (will also act as admin node) and distribute the keys to the other monitors and osd's in the cluster<\/p>\n 9. Install chrony on all hosts and enable it<\/p>\n 10. Adjust the pid_max count on all the osd servers. As I have more than 20 disks I have to do that. <\/p>\n 11. Now its time to download the Ceph components from RHN (rhceph-1.2.3-rhel-7-x86_64.iso). Once downloaded mount the iso on your ceph-mgmt host and copy the following files:<\/p>\n 12. Install the ice_setup-*.rpm's<\/p>\n 13. Before we start configuring or installing any further ceph components we need to create a ceph-config folder which will store future ceph configs<\/p>\n 14. We now launch the ice server setup which creates a cephdeploy.conf for us which is needed for the ceph deployment later on. <\/p>\n 15. Update the ice_setup packages on the calamari admin node.<\/p>\n 16. Initialize the calamari server. During the setup it will ask you for an admin email address and password for user root (weblogin)<\/p>\n 17. After a successful calamari install you should be able to login to the calamari webserver with root\/your_password (You won't see much yet) <\/p>\n","protected":false},"excerpt":{"rendered":" The goal of the ceph \/ OpenStack integration post series is to give you a step by step guide on how you can build a ceph cluster and integrate it with your openstack environment for a POC as well as demonstrate how you can upload qcow2 images into rbd, transform them into raw and import…<\/p>\n","protected":false},"author":2,"featured_media":1013,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_themeisle_gutenberg_block_has_review":false,"footnotes":""},"categories":[3,36],"tags":[38],"class_list":["post-986","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-linux","category-openstack","tag-ceph-openstack-juno-cinder-glance"],"_links":{"self":[{"href":"https:\/\/blog.domb.net\/index.php?rest_route=\/wp\/v2\/posts\/986","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.domb.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.domb.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.domb.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.domb.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=986"}],"version-history":[{"count":25,"href":"https:\/\/blog.domb.net\/index.php?rest_route=\/wp\/v2\/posts\/986\/revisions"}],"predecessor-version":[{"id":1038,"href":"https:\/\/blog.domb.net\/index.php?rest_route=\/wp\/v2\/posts\/986\/revisions\/1038"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.domb.net\/index.php?rest_route=\/wp\/v2\/media\/1013"}],"wp:attachment":[{"href":"https:\/\/blog.domb.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=986"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.domb.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=986"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.domb.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=986"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n1x Dell r720 (Controller)
\n6x Dell r720xd (Compute)
\nI am using RHEL OSP 6 on RHEL 7.1<\/p>\n
\n1x Dell r720 (ceph-mgmt\/admin)
\n3x Dell r720xd (ceph-mon \/ This could also be a VM in the real world)
\n6x Dell r720xd (ceph-osd) a 24 disks
\nI am using RHEL 7.1 with CEPH 1.2.3<\/p>\n\r\n# systemctl disable firewalld\r\n# systemctl stop firewalld\r\n<\/pre>\n
\r\nTYPE=\"Ethernet\"\r\nBOOTPROTO=none\r\nDEFROUTE=\"yes\"\r\nIPV4_FAILURE_FATAL=\"no\"\r\nIPV6INIT=\"yes\"\r\nIPV6_AUTOCONF=\"yes\"\r\nIPV6_DEFROUTE=\"yes\"\r\nIPV6_FAILURE_FATAL=\"no\"\r\nDEVICE=p2p1\r\nONBOOT=yes\r\nUUID=25f57a8c-5cc9-41d2-9ae4-9e9d70161442\r\nNAME=\"System p2p1\"\r\nHWADDR=dc:0e:a1:8c:a0:2f\r\nIPADDR=192.168.1.25\r\nPREFIX=16\r\nGATEWAY=192.168.1.1\r\nDOMAIN=\"local.domb.com\"\r\nDNS1=192.168.1.23\r\nMTU=9000 #####> Essential for ceph cluster\r\n<\/pre>\n
\r\n# systemctl disable NetworkManager.service\r\n# systemctl stop NetworkManager.service\r\n# systemctl start network.service\r\n# systemctl enable network.service\r\n<\/pre>\n
\r\n# sed -i 's\/SELINUX=.*\/SELINUX=disabled\/' \/etc\/selinux\/config\r\n<\/pre>\n
\r\ncat > \/etc\/hosts << EOF\r\n192.168.1.180 compute1.local.domb.com compute1\r\n192.168.1.195 compute2.local.domb.com compute2\r\n192.168.1.198 compute3.local.domb.com compute3\r\n192.168.1.201 compute4.local.domb.com compute4\r\n192.168.1.204 compute5.local.domb.com compute5\r\n192.168.1.207 compute6.local.domb.com compute6\r\n192.168.1.210 compute7.local.domb.com compute7\r\n192.168.1.213 compute8.local.domb.com compute8\r\n192.168.1.216 ceph-mgmt.local.domb.com ceph-mgmt\r\n192.168.1.235 ceph-osd01.local.domb.com ceph-osd01\r\n192.168.1.219 ceph-osd02.local.domb.com ceph-osd02\r\n192.168.1.222 ceph-osd03.local.domb.com ceph-osd03\r\n192.168.1.225 ceph-osd04.local.domb.com ceph-osd04\r\n192.168.1.228 ceph-osd05.local.domb.com ceph-osd05\r\n192.168.1.231 ceph-mon01.local.domb.com ceph-mon01\r\n192.168.1.242 ceph-mon02.local.domb.com ceph-mon02\r\n192.168.1.238 ceph-mon03.local.domb.com ceph-mon03\r\nEOF\r\n<\/pre>\n
\r\n# subscription-manager register --username your_rhn_username --password your_rhn_password\r\n# subscription-manager attach --pool=your_pool_id\r\n# subscription-manager repos --enable=rhel-7-server-rpms --enable=rhel-7-server-rhceph-1.2-calamari-rpms --enable=rhel-7-server-rhceph-1.2-installer-rpms --enable=rhel-7-server-rhceph-1.2-mon-rpms --enable=rhel-7-server-rhceph-1.2-osd-rpms\r\n<\/pre>\n
\r\n# yum -y update\r\n# init 6\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# ssh-keygen\r\n[root@ceph-mgmt ~]# for i in ceph-mgmt ceph-mon01 ceph-mon02 ceph-mon03 ceph-osd01 ceph-osd02 ceph-osd03 ceph-osd04 ceph-osd05;do \r\n ssh-copy-id $i;\r\n done\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# for i in ceph-mgmt ceph-mon01 ceph-mon02 ceph-mon03 ceph-osd01 ceph-osd02 ceph-osd03 ceph-osd04 ceph-osd05; do \r\n ssh $i \"yum -y install chrony ; systemctl enable chronyd ; systemctl start chronyd\"; \r\n done\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# for i in ceph-osd01 ceph-osd02 ceph-osd03 ceph-osd04 ceph-osd05; do \r\n ssh $i \"cat 'kernel.pid_max = 4194303' > \/etc\/sysctl.d\/99-pid_max.conf ; sysctl -p \/etc\/sysctl.d\/99-pid_max.conf\";\r\ndone\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# mount -o loop rhceph-1.2.3-rhel-7-x86_64.iso \/mnt\/\r\n[root@ceph-mgmt ~]# cp \/mnt\/RHCeph-Calamari-1.2-x86_64-c1e8ca3b6c57-285.pem \/etc\/pki\/product\/285.pem\r\n[root@ceph-mgmt ~]# cp \/mnt\/RHCeph-Installer-1.2-x86_64-8ad6befe003d-281.pem \/etc\/pki\/product\/281.pem\r\n[root@ceph-mgmt ~]# cp \/mnt\/RHCeph-MON-1.2-x86_64-d8afd76a547b-286.pem \/etc\/pki\/product\/286.pem\r\n[root@ceph-mgmt ~]# cp \/mnt\/RHCeph-OSD-1.2-x86_64-25019bf09fe9-288.pem \/etc\/pki\/product\/288.pem\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# yum install \/mnt\/ice_setup-*.rpm\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# mkdir ~\/ceph-config\r\n[root@ceph-mgmt ~]# cd ~\/ceph-config\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# ice_setup -d \/mnt\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# yum -y update && ice_setup update all\r\n<\/pre>\n
\r\n[root@ceph-mgmt ~]# calamari-ctl initialize\r\n<\/pre>\n