Installing Eucalyptus (1.4)

Before you proceed with the installation, be sure to take a look at the list of Eucalyptus' prerequisites.

A Eucalyptus cloud setup consists of three components -- the cloud controller, the cluster controller(s) and node controller(s). The cloud controller is a Java program that, in addition to high-level resource scheduling and system accounting, offers a Web services interface and a Web interface to the outside world. Cluster controller and node controller are written in C and deployed as Web services inside Apache.

Communication among these three types of components goes over SOAP with WS-security. There is one cluster controller per cluster, running on the head node; there is one node controller per each compute node, running as root in dom0. So, if you are installing Eucalyptus on one cluster, then one cloud and one cluster controller should be deployed on the head node and one node controller should be deployed on each compute node.

If you are upgrading from a previous version of Eucalyptus, please follow the instructions in the Upgrade Document.

Eucalyptus can be installed from source or using a set of RPM packages. The former method is more general and should work on practically any Linux system, the latter should work on most RPM-based systems. (In case you're curious, converting RPMs into DEBs with alien will not work.) Furthermore, we have special advice for those using Rocks-based clusters.

If run into any problems, be sure to check the troubleshooting guide for solutions to commonly encountered problems.

Installing Eucalyptus from source (1.4)

NOTE - If you are upgrading from a Eucalyptus 1.3 or older installation, please consult the Upgrade Documentation for instructions that will explain how to preserve user account information and re-import the images. If you are re-installing version 1.4, you may want to delete cached images on all nodes (in INSTANCE_PATH/eucalyptus/cache so they are not wasting disk space).

1. Download Eucalyptus

Download

  • eucalyptus-1.4-src.tar.gz (Eucalyptus source) and
  • eucalyptus-1.4-src-deps.tar.gz

From the Eucalyptus Web site:

Unpack the Eucalyptus source:

tar zvxf eucalyptus-1.4-src.tar.gz

Now you should have a directory eucalyptus-1.4. To simplify the remainder of the installation, define EUCALYPTUS_SRC environment variable to be the top of the source tree of eucalyptus and the variable EUCALYPTUS to be the directory where eucalyptus will be installed:

cd eucalyptus-1.4
export EUCALYPTUS_SRC=`pwd`
export EUCALYPTUS=/opt/eucalyptus

2. Dependencies

To install Eucalyptus, you need to build packages that Eucalyptus depends on, which we provide in the above-mentioned package eucalyptus-1.4-src-deps.tar.gz. For the sake of this discussion, we are going to assume that all packages have been untarred inside "$EUCALYPTUS_SRC/eucalyptus-src-deps/" as above and will be installed in "$EUCALYPTUS/packages".

Unpack the dependencies and create the directory you'll use to install them:

cd $EUCALYPTUS_SRC
tar zvxf ../eucalyptus-1.4-src-deps.tar.gz 
mkdir -p $EUCALYPTUS/packages/

Build and install the dependencies as follows:

a. Axis2

cd $EUCALYPTUS/packages
tar zxvf $EUCALYPTUS_SRC/eucalyptus-src-deps/axis2-1.4.tgz

b. Apache (Httpd)

The Eucalyptus node and cloud controller depend on Apache 2. We rely on running apache as root, therefore we recommend that apache 2 be installed separately from the webserver that might be included as part of your distribution.

cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
tar zvxf httpd-2.2.8.tar.gz
cd httpd-2.2.8
CFLAGS="-DBIG_SECURITY_HOLE" ./configure --prefix=$EUCALYPTUS/packages/httpd-2.2.8 --with-included-apr
make ; make install

c. Axis2/C

export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.5.0
cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
tar zvxf axis2c-src-1.5.0.tar.gz
cd axis2c-src-1.5.0
./configure --with-apache2=$EUCALYPTUS/packages/httpd-2.2.8/include --prefix=${AXIS2C_HOME}
make ; make install

d. Rampart/C

export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.5.0
export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib:$LD_LIBRARY_PATH
cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
tar zvxf rampartc-src-1.2.0.tar.gz
cd rampartc-src-1.2.0
./configure --prefix=${AXIS2C_HOME} --enable-static=no --with-axis2=${AXIS2C_HOME}/include/axis2-1.5.0
make ; make install

Now edit the file $AXIS2C_HOME/axis2.xml: search for "Security" and change

  <!--phase name="Security"/-->

to

  <phase name="Security"/>

and save the file.

e. Libvirt

cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
tar zvxf libvirt-0.4.6.tar.gz
cd libvirt-0.4.6
./configure --prefix=$EUCALYPTUS/packages/libvirt-0.4.6 --without-storage-disk --without-storage-iscsi --without-storage-fs --without-storage-lvm --without-libvirtd --without-remote --without-lxc --without-kvm --without-qemu --without-sasl --without-openvz
make ; make install

f. Other software

Additionally, the following dependencies need to be met:

  • Java Development Kit (JDK), version 1.6 or above
  • Apache ant, version 1.6 or above
  • GNU C compiler
  • Make

NOTE: The Eucalyptus dependencies may, in turn, depend on libraries or packages not present in your installation: Please, refer to the documentation of the dependent package for help (e.g., see apache.org Web site for help with compiling the httpd server).

Finally, we recommend that you install functioning EC2 command-line tools from Amazon to interact with Eucalyptus. The latest version of these tools that we support are ec2-api-tools-1.3-30349 and ec2-ami-tools-1.3-26357.

3. Building Eucalyptus

cd $EUCALYPTUS_SRC
./configure --with-axis2=$EUCALYPTUS/packages/axis2-1.4 --with-axis2c=$EUCALYPTUS/packages/axis2c-1.5.0 --with-libvirt=$EUCALYPTUS/packages/libvirt-0.4.6/ --prefix=$EUCALYPTUS
make ; make install

4. Eucalyptus on multiple machines

You need now to instruct Eucalyptus on where it is installed. We provide the 'euca_conf' script as a convenience for setting up the configuration file located in '$EUCALYPTUS/etc/eucalyptus/eucalyptus.conf'. (Expert users may edit this file manually instead of using 'euca_conf'.)

$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS $EUCALYPTUS/etc/eucalyptus/eucalyptus.conf

At this point you're ready to push the software out to the nodes. If you installed Eucalyptus in its own directory, you can just sync the entire package to all hosts using whatever mechanism you typically use to push changes to nodes (rsync, for instance):

rsync -a $EUCALYPTUS/ hostname1:$EUCALYPTUS/
rsync -a $EUCALYPTUS/ hostname2:$EUCALYPTUS/
...

Eucalyptus on a Rocks cluster

If you want to install Eucalyptus on a Rocks cluster, you can now follow the regular RPM instructions. The Cloud Controller and the Cluster Controller will need to be installed on your Rocks front-end, and the Node Controller will need to be installed on any of your Rocks nodes that have been configured as 'VM Containers'.

Eucalyptus will not run on a Rocks virtual cluster!

If you have previously installed Eucalyptus on your Rocks cluster, you should disable the old Eucalyptus rolls:

rocks disable roll eucalyptus

rebuild the rocks distribution:

cd /home/install
rocks-dist dist

and finally reinstall the nodes as 'VM Containers'. You can then follow the RPM instructions to install and configure Eucalyptus. Keep in mind that the java roll in Rocks V (and V.I) includes JDK version 1.5 which is not enough to run Eucalyptus. You have to install the 1.6 JDK. For our testing we used Sun's JDK, which can be found at http://java.sun.com/javase/downloads/index.jsp.

Installing Eucalyptus using RPMs (1.4)

1. Download RPMs

Eucalyptus RPM installation is broken up into several packages. Besides the general files required on all nodes, we separately package Eucalyptus running components: the cloud controller (-cloud RPM), the cluster controller (-cc RPM), and the node controller (-nc RPM). Furthermore, to simplify the installation, we offer copies of third-party packages that Eucalyptus depends on (Apache -compiled to run as root-, libvirt and axis2c) bundled into a single "deps" tarball.

E.g., for an x86_64 installation, download from http://eucalyptus.cs.ucsb.edu/downloads

  • eucalyptus-1.4-2.x86_64.rpm
  • eucalyptus-nc-1.4-2.x86_64.rpm
  • eucalyptus-cc-1.4-2.x86_64.rpm
  • eucalyptus-gl-1.4-2.x86_64.rpm
  • eucalyptus-cloud-1.4-2.x86_64.rpm
  • eucalyptus-1.4-rpm-deps-x86_64.tar.gz

Change x86_64 into i386 to download 32 bit RPMs instead.

2. Install RPMs on the front end

On front end, where cloud controller and cluster controller will run, install the -cloud and -cc RPMs, along with the third-party dependencies:

rpm -Uvh euca-axis2c-1.4-1.x86_64.rpm \
         euca-httpd-1.4-1.x86_64.rpm \
         euca-libvirt-1.4-1.x86_64.rpm \
         eucalyptus-1.4-2.x86_64.rpm \
         eucalyptus-cloud-1.4-2.x86_64.rpm \
         eucalyptus-gl-1.4-2.x86_64.rpm \
         eucalyptus-cc-1.4-2.x86_64.rpm

2.3 Install RPMs on the nodes

On the compute nodes, install the node controller RPM, along with the third-party dependencies:

rpm -Uvh euca-axis2c-1.4-1.x86_64.rpm \
         euca-libvirt-1.4-1.x86_64.rpm \
         euca-httpd-1.4-1.x86_64.rpm \
         eucalyptus-1.4-2.x86_64.rpm \
         eucalyptus-gl-1.4-2.x86_64.rpm \
         eucalyptus-nc-1.4-2.x86_64.rpm

Upgrading to Eucalyptus (1.4)

These are instructions for those who would like to upgrade to Eucalyptus 1.4 from earlier source-based or RPM-based installations of Eucalyptus. Both upgrade paths involve making a backup of the key state from the old installation. The last section of this document explains how to roll back to the previous installation using the backup.

1. Clean up Eucalyptus running state

  • Terminate all Eucalyptus instances
    ec2-terminate-instances ...      # (as admin)
    
  • Shut down Eucalyptus on all nodes.
    /etc/init.d/eucalyptus stop
    
  • Check for errant Eucalyptus processes on all nodes and kill them
    ps aux | grep euca
    kill -9 ...
    
    

2. Install Eucalyptus 1.4

Both source- and RPM-based installations can be upgraded:

  • Option A: Source-based installation upgrade:
    • Move away the old installation on the head-node and all compute nodes. E.g.:
      export EUCALYPTUS_OLD=/opt/eucalyptus-1.3
      mv /opt/eucalyptus $EUCALYPTUS_OLD
      
    • Follow the steps in the Source Code Installation section of the Administrator's Guide and, afterwards, return here.
    • Copy back the old database and keys (we assume that $EUCALYPTUS points to the new location, such as /opt/eucalyptus):
      cp $EUCALYPTUS_OLD/var/eucalyptus/db/eucalyptus.* $EUCALYPTUS/var/eucalyptus/db/
      rm $EUCALYPTUS/var/eucalyptus/db/*.lck
      cp $EUCALYPTUS_OLD/etc/eucalyptus/cloud.d/*.bks $EUCALYPTUS/etc/eucalyptus/cloud.d/
      
  • Option B: RPM-based installation upgrade:
    • Follow the steps in the RPM-based Installation section of the Administrator's Guide and, afterwards, return here.

3. Update the configuration

  • Edit eucalyptus.conf file on head node. Do not start with the old one, rather, copy over matching parts to new one. (If you updated an RPM-based install, the new configuration is in eucalyptus.conf.rpmnew while eucalyptus.conf contains your old one.) Specifically, copy over the values of the following variables:
    • *_PORT
    • NODES
    • INSTANCE_PATH
  • Note that the following parameters were renamed:
    • VNET_PUB_INTERFACE renamed to VNET_INTERFACE
    • VNET_DHCPDEAMON renamed to VNET_DHCPDAEMON
  • Due to dramatic changes in the virtual network implementation, there is, unfortunately, no simple way to convert the old network configuration to 1.4. We suggest that to set the remaining VNET_* variables appropriately you read the Network Configuration section of the Administrator's Guide and, afterwards, return here.
  • On each compute node, go over the config file, comparing it with the old one, same as with the head-node file. (Naturally, you are free to create a new config file for compute-nodes and rsync it with all the nodes.)
  • Start NCs, the CC, and the CLC. On all node do
    $EUCALYPTUS/etc/init.d/eucalyptus start
    
  • In a Web browser, load https://headnode:8443/ and log in as admin with your old password. User accounts should still be there, but not the images. In the new 'Configuration' tab:
    • Click 'Add Cluster' and fill out the fields (You can use host/port/name values from <cluster host="" port="" name="...."> in $EUCALYPTUS/etc/eucalyptus/cloud.d/eucalyptus.xml.)
    • Verify that the Walrus path is set to a reasonable path (this is where all images, kernels, ramdisks, and generic buckets will be stored).
  • Synchronize node keys
    $EUCALYPTUS/usr/sbin/euca_sync_key -c $EUCALYPTUS/etc/eucalyptus/eucalyptus.conf
    
  • Verify that the nodes are back up (if not, see the Troubleshooting section.)
    ec2-describe-availability-zones verbose
    

4. Re-import old images

Storage of disk images, kernels, and ramdisks has changed in 1.4 due to the introduction of Walrus, an S3-compatible bucket-based storage service. Instead of using euca add_image, these files are now added using standard EC2/S3 tools. To upgrade, the files will have to be re-added (and, hence, their EMIs will change!).

Note that if you made any changes to config.xml files of your images, you will have to re-integrate these changes into the Perl script named $EUCALYPTUS/usr/share/eucalyptus/gen_libvirt_xml which now generates the equivalent XML document for all images on-the-fly. Be sure to sync that script to all your nodes.)

cd (the registered images directory)
  • For each image in the registered images directory, add the kernel, the ramdisk (if any), and the root file system image. See Image Management section for more information.
cd (an image dir)
gunzip *
ec2-bundle-image -i (kernel image name) --kernel true -d .
Please peicfy a value for the arch [x86_64]: ENTER
...
ec2-upload-bundle -b (unique bucket) -m (kernel image name).manifest.xml
ec2-register (unique bucket)/(kernel image name).manifest.xml
IMAGE   eki-.......
  • If the image has a ramdisk, then:
    ec2-bundle-image -i (ramdisk image name) --ramdisk true -d .
    Please peicfy a value for the arch [x86_64]: ENTER
    ...
    ec2-upload-bundle -b (unique bucket 2) -m (ramdisk image name).manifest.xml
    ec2-register (unique bucket 2)/(ramdisk image name).manifest.xml
    IMAGE   eri-.......
    
ec2-bundle-image -i (disk image name) --kernel (eki-......) [--ramdisk (eri-......)] -d .
Please peicfy a value for the arch [x86_64]: ENTER
...
ec2-upload-bundle -b (unique bucket 3) -m (disk image name).manifest.xml
ec2-register (unique bucket 3)/(disk image name).manifest.xml
IMAGE   emi-......
  • Check that it is there:
    ec2-describe-images
    
  • repeat for other images

5. Clean up old disk state

Once you are confident that the new installation is working, delete the old state on disk.

rm -rf $EUCALYPTUS_OLD 
rm -rf (the template path)
rm -rf (the registration path)
rm /etc/defaults/eucalyptus    # (this is no longer used)

If you upgraded using RPM packages, delete on all nodes the backup of old state that was created during the upgrade:

rm /root/eucalyptus-pre1.4-rollback.tar

6. Rolling back to an earlier installation

  • Stop Eucalyptus 1.4 processes, if any, on all nodes
  • Option A: Rolling back source-based 1.4 upgrade:
    • Remove any files added during the failed upgrade. For example:
      rm -rf $EUCALYPTUS
      
    • Move back the old installation on all nodes:
      mv $EUCALYPTUS_OLD $EUCALYPTUS
      
  • Option B: Rolling back RPM-based 1.4 upgrade:
    • Remove the failed RPMs on all affected nodes (depending on the failure, you might have to use the --nopreun option)
      rpm -e eucalyptus-cloud eucalyptus-cc euca-httpd euca-axis2c euca-libvirt eucalyptus
      
    • Download and install the 1.3 RPMs on all nodes as discussed in the Administrator's Guide (it may be unnecessary to install euca-vde as it does not get removed during the upgrade). For example, on the front-end you can use:
      rpm -ivh eucalyptus-1.3-1.i386.rpm euca-httpd-1.0-1.i386.rpm eucalyptus-cloud-1.3-1.i386.rpm eucalyptus-cc-1.3-1.i386.rpm
      
    • Copy the old state saved during the upgrade process on all nodes:
      cd $EUCALYPTUS
      tar xvf /root/eucalyptus-pre1.4-rollback.tar
      
  • Start Eucalyptus, as before