Redhat Virtualization / rhev 3.5 on rhel7 with hosted-engine (rhel7 rhev 3.5)

This howto is going to walk you trough an installation of hosted engine on rhel7 for redhat virtualization 3.5.

Important, before you start make sure you have the following already setup (3 Hosts with these components):
Network: Your hostname (hypervisor and hosted engine) must have a dns name which resolve!
Storage: Have an nfs or iscsi storage ready ( i will use an nfs share on my asustor box)
Software: A rhel 6.5 iso as 6.6 did not work for me.

1. Subscribe to the redhat repos:

# subscription-manager repos --disable=*
# subscription-manager repos --enable=rhel-7-server-rpms
# subscription-manager repos --enable=rhel-7-server-supplementary-rpms
# subscription-manager repos --enable=jb-eap-6-for-rhel-7-server-rpms
# subscription-manager repos --enable=rhel-7-server-rhevh-rpms
# subscription-manager repos --enable=rhel-7-server-rhevh-rpms
# subscription-manager repos --enable=rhel-7-server-rhev-mgmt-agent-rpms
# subscription-manager repos --enable=rhel-7-server-optional-rpms

2. Disable NetworkManager, firewalld and enable iptables

# systemctl disable NetworkManager && systemctl stop NetworkManager
# systemctl stop firewalld && systemctl mask firewalld
# yum install iptables-services
# systemctl enable iptables

3. Add the following firewall rules in /etc/sysconfig/iptables and restart it.

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -i lo -j ACCEPT
# ssh
-A INPUT -p tcp --dport 22 -j ACCEPT
# snmp
-A INPUT -p udp --dport 161 -j ACCEPT
# consoles
-A INPUT -p tcp -m multiport --dports 5900:6923 -j ACCEPT
# vdsm
-A INPUT -p tcp --dport 54321 -j ACCEPT
# libvirt tls
-A INPUT -p tcp --dport 16514 -j ACCEPT
# vm migration
-A INPUT -p tcp -m multiport --dports 49152:49216 -j ACCEPT
# portmapper
-A INPUT -p udp -m udp --dport 111 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 38465 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 38466 -j ACCEPT
# nfs
-A INPUT -p tcp -m tcp --dport 111 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 38467 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2049 -j ACCEPT
# status
-A INPUT -p tcp -m tcp --dport 39543 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 55863 -j ACCEPT
# nlockmgr
-A INPUT -p tcp -m tcp --dport 38468 -j ACCEPT
-A INPUT -p udp -m udp --dport 963 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 965 -j ACCEPT
# Reject any other input traffic
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -m physdev ! --physdev-is-bridged -j REJECT --reject-with icmp-host-prohibited
COMMIT

Then run

# service iptables restart

4. Lets go ahead and install the needed packages for hosted engine

# yum install ovirt-hosted-engine-setup screen  nfs-utils system-storage-manager

5. Launch screen

# screen

6. Copy the rhel-server-6.5-x86_64-dvd.iso to your hosted engine server. I copied it into tmp. Do NOT mount the 6.6 image. The setup script will do that for you! Make sure that the image is world readable as vdsm needs to read it.

7. Now we can start with the hosted engine setup

# hosted-engine --deploy

8. Once you answered all the questions you will see something similar to this

--== CONFIGURATION PREVIEW ==--
          Bridge interface                   : enp2s3
          Engine FQDN                        : hrhevm.local.domb.com
          Bridge name                        : rhevm
          SSH daemon port                    : 22
          Gateway address                    : 192.168.1.1
          Host name for web application      : hrhevm1
          Host ID                            : 1
          Image alias                        : hrhevm_engine
          Image size GB                      : 25
          Storage connection                 : 192.168.1.3:/volume1/rhevdata
          Console type                       : vnc
          Memory size MB                     : 4096
          MAC address                        : 00:16:3e:2b:4d:14
          Boot type                          : cdrom
          Number of CPUs                     : 2
          ISO image (for cdrom boot)         : /tmp/rhel-server-6.5-x86_64-dvd.iso
          CPU Type                           : model_Opteron_G3

          Please confirm installation settings (Yes, No)[Yes]:

9. If everything went well you you will see the following screen.

          You can now connect to the VM with the following command:
                /bin/remote-viewer vnc://localhost:5900
          Use temporary password "2618VYiP" to connect to vnc console.
          Please note that in order to use remote-viewer you need to be able to run graphical applications.
          This means that if you are using ssh you have to supply the -Y flag (enables trusted X11 forwarding).
          Otherwise you can run the command from a terminal in your preferred desktop environment.
          If you cannot run graphical applications you can connect to the graphic console from another host or connect to the console using the following command:
          virsh -c qemu+tls://Test/system console HostedEngine
          If you need to reboot the VM you will need to start it manually using the command:
          hosted-engine --vm-start
          You can then set a temporary password using the command:
          hosted-engine --add-console-password
          The VM has been started.  Install the OS and shut down or reboot it.  To continue please make a selection:
         
          (1) Continue setup - VM installation is complete
          (2) Reboot the VM and restart installation
          (3) Abort setup
          (4) Destroy VM and abort setup

Do Not press 1 yet. Launch your favorite vnc client and connect to
/bin/remote-viewer vnc://yourrhel7server:5900. Follow the installer to complete the rhel 6.5 installation. Once your VM shuts down press 1.

(1, 2, 3, 4)[1]:
          Waiting for VM to shut down...
[ INFO  ] Creating VM

10.Ssh to your vm after it rebooted and register your system to rhn with the following channels.

# subscription-manager repos --enable=rhel-6-server-rpms
# subscription-manager repos --enable=rhel-6-server-supplementary-rpms
# subscription-manager repos --enable=rhel-6-server-rhevm-3.5-rpms
# subscription-manager repos --enable=jb-eap-6-for-rhel-6-server-rpms
# subscription-manager repos --enable=rhel-6-server-rhevh-rpms

11. Update your rhel6.5 host

# yum -y update

12. Now its time to install rhevm

# yum -y install rhevm

13. After rhevm is installed you can run engine setup. Make sure that the password you used for your hypervisor hosted-engine –deploy is the same as engine-setup. If you do not have the same passwords your hypervisor will not get added to the rhevm hosted engine vm and you’ll end up with

[ ERROR ] Cannot connect to engine APIs on hrhevm.local.domb.com:  JBWEB000065: HTTP Status 401
[ ERROR ] Failed to execute stage 'Closing up': Cannot connect to engine APIs on hrhevm.local.domb.com
# engine-setup

14. After you entered all the information you’ll see the following screen

          --== CONFIGURATION PREVIEW ==--
         
          Application mode                        : both
          Firewall manager                        : iptables
          Update Firewall                         : True
          Host FQDN                               : hrhevm.local.domb.com
          Engine database name                    : engine
          Engine database secured connection      : False
          Engine database host                    : localhost
          Engine database user name               : engine
          Engine database host name validation    : False
          Engine database port                    : 5432
          Engine installation                     : True
          NFS setup                               : True
          PKI organization                        : local.domb.com
          NFS mount point                         : /var/lib/exports/iso
          NFS export ACL                          : hrhevm.local.domb.com(rw)
          Configure local Engine database         : True
          Set application as default page         : True
          Configure Apache SSL                    : True
          Configure WebSocket Proxy               : True
          Engine Host FQDN                        : hrhevm.local.domb.com
          Please confirm installation settings (OK, Cancel) [OK]:

15. Once done with the setup run it again to get the latest 3.5

[root@hrhevm ~]# engine-setup
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-wsp.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf']
          Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20150217210109-kn2sm9.log
          Version: otopi-1.3.0 (otopi-1.3.0-2.el6ev)
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Stage: Environment customization
         
          Welcome to the RHEV 3.5 setup/upgrade.
          Please read the RHEV 3.5 install guide
          https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.5/html/Installation_Guide/index.html.
          Please refer to the RHEV Upgrade Helper application
          https://access.redhat.com/labs/rhevupgradehelper/ which will guide you in the upgrading process.
          Would you like to proceed? (Yes, No) [Yes]:
         
          --== PRODUCT OPTIONS ==--

16. Finally go back to the window on your actual hypervisor and press 1

          (1, 2, 3, 4)[1]:
[ INFO  ] Engine replied: DB Up!Welcome to Health Status!
          Enter the name of the cluster to which you want to add the host (Default) [Default]:
[ INFO  ] Waiting for the host to become operational in the engine. This may take several minutes...
[ INFO  ] Still waiting for VDSM host to become operational...
[ INFO  ] The VDSM Host is now operational
          Please shutdown the VM allowing the system to launch it as a monitored service.
          The system will wait until the VM is down.

17. Login to your rhevm instance via http://yourhostrhevm
rhevm1

18. Click on Administrative portal and login.
rhevm2

19. You should now see your hypervisor in the hosts tab and the hosted engine in the virtual machines tab.
hosted

20. Shutdown the rhevm vm so it will get added as a monitored service

[ INFO  ] Enabling and starting HA services
          Hosted Engine successfully set up
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150218010918.conf'
[ INFO  ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination

20. Repeat these steps on the next 2 hosts 🙂

Hint: If you are running hosted engine in a non clustered environment it might be possible that your hosted engine does not come up after a reboot. You can start it manually by running the following commands on the hypervisor:

# hosted-engine --set-maintenance --mode=global
# hosted-engine --vm-start