Private Cloud Computing: Comprehensive Beginner's Guide

The purpose of this document is to help you achieve a running cloud with just 2 machines: a Host and a Node - and maintain it.

Following several other guides on the web (and better ones at that), you'll reach a point where you'll have at least one machine acting as the cloud/storage/etc controller (i.e. the Host) and a node controller (i.e. a cloud node).

http://fnords.wordpress.com/2009/10/04/run-your-own-uec-part-1/
http://fnords.wordpress.com/2009/10/07/run-your-own-uec-part-2/
http://fnords.wordpress.com/2009/10/13/run-your-own-uec-part-3/
http://aakash19.wordpress.com/2009/09/30/private-cloud-setup/

This document is also based on the Ubuntu Enterprise Cloud (UEC) which uses Eucalyptus as part of the whole solution.

Some of the topics discussed below are open to improvements and basically, it's the notion of things that matters most.

What will a newcomer want to see happen in his cloud, how to prevent the virtual machines to lose all the configs, how to manage the clouds... etc.

Having said that, and getting this far, you've most likely managed to download the UEC images from the Store via Eucalyptus and also run instances from them.

What's next?

1) Cloud Management (I went with RightScale):

Registering the cloud is simple enough, if using a LAN router, make sure ports 8443 to 8773 are begin forwarded to the Cloud Controller and then just follow Eucalytus steps under Services in your Eucalyptus frontend, and later on, at the RightScale website.

Registering an account is also needed and free.

Problems registered here:

once the cloud is registered, you'll be able to start/stop instances but due to a bug (to be corrected in the next version of the RightScale Site), all instances are started as "m1.small".

2) Data persistence

As far as I could tell from all that I've read, cloud computing expects instances to be volatile, meaning that after shutting down an instance and restarting it, all the changes made in config files, or new files brought in by installing new apps - will be gone.

If one wishes to have data persistence, one must resort to volumes and snapshots. But volumes and snapshots usually resemble virtual harddrives which are created in the cloud and attached to the running instances.

Once logged into the instance, the user can partition, format and mount these volumes - and store any data he/she wants.

The instance shuts down, dies, a new instance is brought up, the volume is reattached and all the stored files are back.

But what about root system persistence?

NOTE:

There is a mis-configuration in the present UEC images available in the Ubuntu Store via Eucalyptus - the "acpiphp" module is not loaded at boot with these images when running an instance, and that means that any attached volume won't be properly identified inside the instance.

So, if using these, you'll need to:

start the instance
ssh into it
modprobe acpiphp
exit ssh
attach volume
ssh back in and mess with the volume

However, the daily builds from the ubuntu EUC images site use Lucid Lynx, not Karmic Koala, and have the module being loaded properly.

It's pretty easy to bundle one of those:

https://help.ubuntu.com/community/UEC/BundlingImages

This guide worked for me.

2.1) Root System Persistence

The purpose is to make the root system of that instance fully persistent. Any config files, installed apps - everything is kept and if the instance dies and comes back, everything comes back with it. But how to do it?

2.1.1) The chroot method

This method is as plain as it comes and has little to do with cloud computing per se. Anyone who has installed Gentoo Linux or any other "hard-on" distro in the past will be familiar with the concept:

- Start up the instance
- attach a volume
- ssh into the instance
- format the volume
- create a mount point
- mount the volume
- mount /proc and /dev
- chroot into the volume

Basically, run an instance, attach a volume, install another root system to it and then chroot: and you'll be operating on an OS which lives in a volume, which is persistent. If you exit the chroot environment, unmount the volume, kill the instance and bring up a new one, you can re-attach the volume, chroot again and start any services you had - everything will be there.

Good Guide here:

https://help.ubuntu.com/community/Installation/FromLinux <-- at the "Without CD" section.

Pros: works
Cons: doesn't feel like a "clean" way to have persistence (to me at least)

2.1.2) the rsync method

With this method, more actions to bring it up, cleaner outcome:

- Start up the instance
- attach a volume
- ssh into the instance
- format the volume
- create a mount point
- mount the volume
- rsync all the instance system files to the backup (keeping permissions, symlinks, etc)

Basically, take advantage of the power of Rsync. Rsync will only copy to the backup volume any files with altered hash. Permissions will be kept, hidden files will be managed, etc. You can exclude specific dirs too (/dev, /proc, /sys...) and it's even recursive.

First time it runs, it backs up all the instance root files, next runs will be faster (as not all files are altered)

So, now the volume is a backup copy of the instance, minus a couple directories.

- Shut down the instance, bring up a new one
- re-attach the volume
- run a custom restore script (also rsync)
- the instance that died is back!

Wiki pages, database tables, created users... you name it.

Pros: seems cleaner that chroot
Cons: still to see

scripts used for this:
==============================================================================================================
#/bin/bash

#perform backup

cd /mnt/backup/
rsync_result=`rsync -pural --stats --exclude /dev --exclude /proc --exclude /sys --exclude /mnt/backup --exclude /mnt/media / . | grep "files transferred"`
rsync_date=`date +"%Y-%m-%d %H:%M:%S"`

echo $rsync_date" Rsync Backup Completed: "$rsync_result

==============================================================================================================
#!/bin/bash

#perform restore
cd /
rsync_result2=`rsync -pural --stats --exclude /dev/ --exclude /proc/ --exclude /sys/ --exclude /mnt/backup/ --exclude /mnt/media/ /mnt/backup/ . | grep "files transferred"`
rsync_date=`date +"%Y-%m-%d %H:%M:%S"`

echo $rsync_date" Rsync Restore Completed: "$rsync_result2

==============================================================================================================

2.1.3) the inotify method

Rsync for backups is run every 2 minutes in a cron inside the instance. It's not 100% accurate because of that.

So, inotify comes up as an alternative: inotify waits for events and then acts upon them. Let's say, a file being altered in the filesystem.

Haven't tried with it yet, but read that rsync + inotify can do wonders!

3) Image Editing/Creating/Bundling

Another way to go is to create a custom image. If it has the necessary apps built-in, more than half the way there. If, later on, you need to add more - create a new image!

Some images, to some needs, won't require config settings to be changed after first boot. So, setting your needs from the beginning and going for the ultimate image can also be valuable.

I've tried several solutions to bundle my own custom image. Most guides failed, either during the build process or during instance runs, so I've managed with the previous solutions:

- one solid, "vanilla" i.e. base image
- run an instance
- evolve from that running system onwards

More advanced guides on image building:

http://kiranmurari.wordpress.com/2010/03/23/bundling-linux-image-for-uec/
http://kiranmurari.wordpress.com/2010/03/29/uec-bundling-windows-image/

That's it.

Hopefully this has been useful to you in some way, and I haven't broken any Site rules.