Eucalyptus provides the network infrastructure that enables access to and from VM instances. Typically, these VMs reside on virtual subnets that must be configured to be separate and distinct from the physical network on which Eucalyptus components interact. Eucalyptus creates these virtual subnets during instance creation and disposes of them when the last instance on a particular virtual subnet is terminated. Note that the administrator must ensure that the virtual subnet IP address space does not contain, is not contained by, and does not conflict with any part of the physical network IP address space.

Eucalyptus offers four different networking modes (MANAGED, MANAGED-NOVLAN, SYSTEM, and STATIC) that accommodate Eucalyptus deployment on a variety of physical network infrastructures. Before starting Eucalyptus, the administrator should select one of these four modes by modifying the eucalyptus.conf file on each machine running a Eucalyptus component.

WARNING: If you edit any networking related value in eucalyptus.conf, for the changes to take effect you must perform a "clean restart" of the CC using the following command (make sure to terminate all instances before performing the clean restart):

$EUCALYPTUS/etc/init.d/eucalyptus-cc cleanrestart

This document presents the networking features available in Eucalyptus, along with information and requirements pertaining to the underlying physical network, to help you select the appropriate networking mode for your needs and your infrastructure. We follow up with detailed instructions for configuring Eucalyptus using each Eucalyptus networking mode.

1. Networking Features and Requirements

Ultimately your selection of a Eucalyptus networking mode will depend on the networking features you want for your cloud and the degree of control you have over the configuration of the underlying physical network. Here we present a list of Eucalyptus networking features and physical network requirements for your reference when choosing and configuring a Eucalyptus networking mode.

Eucalyptus Networking Features:

Connectivity: All network modes provide IP-level connectivity for VMs started by Eucalyptus. Whether a VM is reachable via IP on a specific port from a host outside of a Eucalyptus installation depends on whether the VM received a public IP address and whether the VM's security group rules (described below) allow traffic on the port.

IP Control: In all modes except SYSTEM, Eucalyptus assigns IP addresses to VMs. In SYSTEM mode, one must allow a DHCP server outside of Eucalyptus to assign IPs to VMs that Eucalyptus starts.

Security Groups: Security groups are sets of networking rules (in effect a firewall) applied to all VM instances associated with a group. In practice a security group defines the access rules for all VM instances associated with a group. For example, a user can specify ingress rules, such as allowing ping (ICMP) or SSH (TCP, port 22) traffic to reach VMs in a specific security group. Note that when you create a VM instance, unless otherwise specified at instance run-time, it is assigned to a "default" security group that denies incoming network traffic from all sources. Thus, to allow login and usage of a new VM instance you must authorize network access to the default security group with the euca-authorize command. Security groups are available in MANAGED and MANAGED-NOVLAN Mode.

Elastic IPs: With elastic IPs the user gains control over a set of Public IP addresses. Once allocated to the user, those same IPs can be dynamically associated to running instances, thus overriding pre-assigned public IPs. This allows users to run well-known services (e.g., Web sites, etc.) within the Eucalyptus cloud. Elastic IPs are available in MANAGED and MANAGED-NOVLAN mode.

Metadata service: Eucalyptus provides an AWS-compatible metadata service that lets you access instance-specific information from inside a running VM. Data provided includes the instance's public and private IP address, reservation ID, user-data (specified with euca-run-instance command), and a launch index (useful for differentiating multiple instances launched with -n option of the euca-run-instance command).

The metadata service is available in all Eucalyptus networking modes. However, the way you access the service from within virtual machines differs depending on the networking mode used to configure your Eucalyptus installation: If your system is configured using MANAGED or MANAGED-NOVLAN modes, then the metadata service is available to VMs just as in Amazon EC2 as shown:

http://169.254.169.254/<specific meta-data information request>

If your system is configured using SYSTEM or STATIC networking modes, then retrieving data requires the IP (or hostname) and port number of the Eucalyptus Cloud Controller (CLC):

http://<IP or hostname of CLC>:8773/<specific meta-data information request>

(We recommend that you set up a DNS entry for each cloud controller. This way you can configure your images to access the metadata service using a DNS name, thus avoiding the need to recreate pre-configured images in the event a specific IP address changes.)

VM Isolation: While network traffic between VM instances belonging to a security group is always open, Eucalyptus can enforce isolation of network traffic between different security groups. This is enforced using a VLAN tag per security group, thus, protecting VMs from unwanted eavesdropping by VM instances belonging to other security groups. VM isolation is available only in MANAGED mode. In MANAGED-NOVLAN mode, VM isolation is guaranteed by having different security groups on different subnets—this translates into Layer-3 only VM isolation.

Network Requirements:

VLAN clean: MANAGED mode uses a Virtual LAN (VLAN) to enforce network isolation between instances. If the underlying physical network is also using a VLAN, there can be conflicts that prevent instances from being network accessible. Thus, it is important to know if your network is 'VLAN clean' (that is, VLANs are usable by Eucalyptus).

Testing VLAN clean:

The admin can verify that the network is VLAN clean (allows/forwards VLAN tagged packets) between machines running Eucalyptus components by performing the following test:

First, choose two IP addresses from the subnet you plan to use with Eucalyptus. (In the following example, those IP addresses are 192.168.1.1 and 192.168.1.2). Next, on the front end machine, choose the interface that is on the local Ethernet (and is set in eucalyptus.conf as VNET_PRIVINTERFACE), and run:

[root@clc]# vconfig add <interface> 10
[root@clc]# ifconfig <interface>.10 192.168.1.1 up

Next, on the node, choose the interface on the local network (and is set in eucalyptus.conf as VNET_PRIVINTERFACE), and run:

[root@node1]# vconfig add <interface> 10
[root@node1]# ifconfig <interface>.10 192.168.1.2 up

Then, perform a ping between the two hosts to validate the interface settings.

On the front end:

[root@clc]# ping 192.168.1.2

On the node:

[root@node1]# ping 192.168.1.1

If this VLAN clean test fails, then your switch needs to be configured to forward VLAN tagged packets (if it is a managed switch, see your switch's documentation to determine how to do this).

Available Public IPs: Instances typically have two IPs associated with them: a private one (belonging to the virtual subnet allocated to their group) and a public one (an IP belonging to the subnet that allows the CC to communicate externally). Most Eucalyptus networking modes are responsible for the allocation of both sets of IPs: In this case the administrator must specify in advance a range of both private IP addresses (used for the virtual subnet) and public IP addresses (used for accessing VM instances).

DHCP server: In all Eucalyptus networking modes (except SYSTEM), a DHCP server binary must be installed on the CC machine. Usually this DHCP server is not configured and thus not active. In these modes an active DHCP server on the same subnet may prevent VM instances from receiving the correct IP address. In SYSTEM mode, however, a pre-configured DHCP server already active on the physical subnet is required.

For the modes requiring an installed DHCP server, that binary must be command-line compatible with ISC DHCP Daemon version 3.0 (this daemon is provided with most Linux distributions, for example with RHEL/CentOS it is provided in a package named dhcp). To configure Eucalyptus to use the DHCP server, you must edit eucalyptus.conf to instruct Eucalyptus as to the location of the DHCP binary, as shown:

VNET_DHCPDAEMON="/usr/sbin/dhcpd"

If your DHCP daemon binary is configured to run as user “root,” (as is the case with RHEL/CentOS and openSUSE) then you do not need to specify a VNET_DHCPUSER.

If your DHCP daemon binary is configured to run as 'non-root' (say, as the user 'dhcpd' as is the case in Ubuntu 8.10 or later), then you must configure Eucalyptus to be aware of that user:

VNET_DHCPUSER="<dhcpUsername>"

2. Choosing a Networking Mode

Before choosing a networking mode for your Eucalyptus network configuration, it is important to understand the interrelationship between feature availability and the requirements of the underlying physical network for each Eucalyptus networking mode—some features can be implemented only if certain requirements are met by the underlying physical network.

The following chart shows the relationship between the networking features available in each Eucalyptus networking mode and the corresponding requirements for the underlying physical network.

NETWORKING MODE	PHYSICAL NETWORK REQUIREMENTS			EUCALYPTUS NETWORKING FEATURES
	VLAN clean	Public IPs	Non-Eucalyptus DHCPD	Connectivity	IP control	Security Groups	Elastic IPs	Metadata service	VM Isolation
MANAGED	✔	✔		✔	✔	✔	✔	✔	✔
MANAGED-NOVLAN		✔	✘	✔	✔	✔	✔	✔	✔✱
STATIC		✔	✘	✔	✔			✔
SYSTEM			✔	✔				✔

✘ - A non-Eucalyptus DHCP server present on the network will interfere with Eucalyptus IP address allocation.
✱ - Provides Layer-3 only VM isolation.

Eucalyptus offers four different networking modes: (MANAGED, MANAGED-NOVLAN, SYSTEM, and STATIC. MANAGED mode is the most full-featured networking mode, providing security groups, VM isolation, elastic IPs and Metadata service. Note that in MANAGED mode the underlying physical network must be VLAN clean, as Eucalyptus provides and manages its own VLAN tagging. If your network is not VLAN clean, you can use MANAGED-NOVLAN mode, which provides the full set of networking features, with the exception of VM isolation between instances.

In the remaining networking modes, SYSTEM and STATIC, there are no virtual subnets—VM instances appear on the physical network as if they were physical machines; and VM instances are directly bridged with the NC machine's physical ethernet device. SYSTEM mode is designed to be the least demanding on the physical network. In this mode, VM instances obtain their configuration from the DHCP server serving the whole physical network. STATIC mode offers the Eucalyptus administrator more control over VM IP address assignment. Here, the administrator configures Eucalyptus with a 'map' of MAC address/IP address pairs. When a VM is instantiated, an unused MAC/IP pair is chosen for the instance. A Eucalyptus-controlled DHCP server then serves the VM instances.

3. Configuring Eucalyptus Networking Modes

In this section, we provide detailed configuration instructions—including configuration examples and additional information—for each of the four Eucalyptus networking modes.

3.1 About Network Configuration Examples

For the configuration examples in the following sections, we use two different network configurations loosely based on our Eucalyptus Community Cloud (shown in Figure 1 and Figure 2 below). In both configurations, the public network used is 173.205.188.0/24 with the router at 173.205.188.1 and the local DNS server at 173.205.188.129.

Figure 1:

Figure 1 has a very simple configuration: all the machines have one Ethernet device (eth0) and they are all connected directly to the public network.

Figure 2:

Figure 2 shows node machines on a private subnet using the front-end machine as a gateway to the public network. Note that the front-end machine has two Ethernet devices (eth0 is on the public network; eth1 is on the private network) and uses NAT to allow the nodes access to the public network. The network between the front end and the node machines is a private network of the range 10.0.1.0/24.

3.2 About Bridge Names

Most Eucalyptus networking modes require a bridge. Bridge names are both hypervisor and Linux distribution dependent. To properly configure a network mode you must know the bridge name for your system. Typically bridge names are as follows:

• For Xen 3.0 or earlier: xenbr0
• For Xen 3.2 and above: eth0
• Most distributions using KVM: br0

To ensure that you are using the correct bridge within your Eucalyptus configuration, enter the brctl show command as shown:

[root@clc]# brctl show
bridge name     bridge id       STP enabled     interfaces
virbr0          8000.000000000000       yes
xenbr0          8000.feffffffffff       no      peth0

Note that the bridge name virbr0 is created by libvirt. This name should not be used. Ensure the bridge is associated with the correct Ethernet device. In the above example, peth0 is attached to the bridge.

For the remainder of this document, we assume that you have correctly identified the bridge and that such bridge is named xenbr0, as shown above.

3.3 About VNET_ Options

All network-related options specified in eucalyptus.conf use the prefix VNET_. The following options are the most commonly used:

VNET_DNS

This option is used to specify a nameserver available on your network. DNS must be specified as an IP address.

VNET_SUBNET, VNET_BROADCAST, VNET_NETMASK

These three options—network address, the broadcast address on the network, and the subnet mask, respectively—work together to define the configuration of a specific network. It is necessary to specify all three options when Eucalyptus requires a virtual subnet.

VNET_ADDRESSPERNET

This option is used to control how many VM instances may simultaneously be part of an individual user's named network (called a 'security group' in Amazon EC2). This option is used only when security groups are available. Typically these numbers are 16, 24, 32, 64, etc, but should never be less than 8. The value specifying this option, alongside VNET_NETMASK, will determine the number of available security groups in the system (VNET_NETMASK determines the size of the address space, while VNET_ADDRESSPERNET determines how the address space is partitioned, so with a bigger limit on security group size, fewer security groups can be created). WARNING: If VNET_ADDRESSPERNET is too large relative to VNET_NETMASK you may have very few security groups or the CLC may refuse to start altogether.

VNET_PUBLICIPS

This is the list or range of public IP addresses available for VMs. You can specify the IP addresses as a list, for example: “10.0.0.1 10.0.0.2 10.0.0.3” or as a range, for example: “10.0.0.1-10.0.0.3.”

3.4 Networking Modes

Here we show you the proper configuration for each Eucalyptus networking mode. Included are requirements, limitations, VNET_ options (that must be set in eucalyptus.conf), and a configuration example for each mode. We also discuss important caveats where applicable.

3.4.1 MANAGED mode

In this mode, Eucalyptus manages the local network of VM instances and provides all networking features Eucalyptus currently supports, including VM network isolation, security groups, elastic IPs, and metadata service. Note that in this mode each security group requires a separate VLAN, which Eucalyptus controls and maintains, thus the underlying physical network must be VLAN clean.

Requirements

• There is an available range of IP addresses to be used for the virtual subnets that do not interfere with the physical network. Typically these IP addresses are selected from the private IP ranges: 192.168.x.x, 10.x.x.x, etc.
• Network must be VLAN clean, meaning that all switch ports that Eucalyptus components are connected to will allow and forward VLAN tagged packets.
• You are not running a firewall on the front end (CC) or your firewall is compatible with the dynamic changes performed by Eucalyptus when working with security groups. (Note that Eucalyptus will flush the 'filter' and 'nat' tables upon boot).
• A range of IP addresses must be available for use by Eucalyptus.
• Front end must have installed DHCP server daemon compatible with ISC DHCP Daemon version 3.0.X.

Limitations

None.

Configuration

The options in eucalyptus.conf that must be configured correctly in MANAGED mode are as follows:

On the front end (options annotated with a '*# may not be required depending on your installation, as follows):

VNET_MODE="MANAGED"
VNET_PUBINTERFACE
VNET_PRIVINTERFACE
VNET_DHCPDAEMON
#VNET_DHCPUSER
VNET_SUBNET
VNET_NETMASK
VNET_DNS
VNET_ADDRSPERNET 
VNET_PUBLICIPS
#VNET_CLOUDIP
#VNET_LOCALIP

On each node:

VNET_MODE="MANAGED"
VNET_PUBINTERFACE
VNET_PRIVINTERFACE

The Eucalyptus administrator must configure the front-end machine's eucalyptus.conf file, for the appropriate Ethernet devices, as follows:

First, with a valid Ethernet device attached to the public network (let’s say eth0), configure as follows:

VNET_PUBINTERFACE="eth0"

Then, you must specify the Ethernet device on the physical network shared with the nodes. This could be the Ethernet device attached to the public network (eth0), or, if this is a second device, (let’s say eth1), you would configure as follows:

VNET_PRIVINTERFACE="eth1"

Nodes must have VNET_PUBINTERFACE set properly. For example, with current Xen versions, this parameter (when your node's Xen bridge is 'eth0') is typically:

VNET_PUBINTERFACE="eth0"

To configure Eucalyptus for your DHCP, see the above section: 1. Networking Features and Requirements, DHCP server.

It is also necessary to instruct Eucalyptus about the unused subnet. To do this, you must configure VNET_SUBNET, VNET_ADDRESSPERNET, etc., as described in the above section: About VNET_ options.

Caveats

In MANAGED mode, Eucalyptus will flush the front-end machine's iptables rules for both 'filter' and 'nat'. Next, it will set the default policy for the 'FORWARD' chain in 'filter' to 'DROP'. At run time, the front end will be adding and removing rules from 'FORWARD' as users add/remove ingress rules from their active security groups. In addition, the 'nat' table will be configured to allow VMs access to the external network using IP masquerading, and will dynamically add/remove rules in the 'nat' table as users assign/unassign public IPs to VMs at instance boot or run-time. If the administrator has some rules that they wish to apply on the front end, they should perform the following procedure on the front end, before Eucalyptus is started or while Eucalyptus is not running.

WARNING!

If the admin chooses to perform this operation to define special iptables rules that are loaded when Eucalyptus starts, they could inadvertently cause Eucalyptus VM networking to fail. It is suggested that you only do this if you are completely sure that it will not interfere with the operation of Eucalyptus.

[root@clc]# iptables-save > $EUCALYPTUS/var/run/eucalyptus/net/iptables-preload

Configuration Example

For our MANAGED mode example, we use the network configuration shown in Figure 2. To configure this mode properly, we must choose an unused private network for our instances. Since the 10.0.1.0/24 subnet is being used, we specify an alternate subnet 192.168.0.0/16.

The following IP addresses are available for use by instances in Eucalyptus: 173.205.188.131-173.205.188.150

On the front-end machine:

VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth1"
VNET_MODE="MANAGED"
VNET_SUBNET="192.168.0.0"
VNET_NETMASK="255.255.0.0"
VNET_DNS="173.205.188.129"
VNET_ADDRSPERNET="32"
VNET_PUBLICIPS="173.205.188.131-173.205.188.150"

On the nodes:

VNET_MODE="MANAGED"
VNET_PUBINTERFACE="eth0"

At this point, we show you how to determine the actual number of active security groups. In our example there are 65536 addresses available (192.168.0.0/16). If we divide by the number of addresses per network (set to 32 below), we find the maximum number of subnets is (65536 / 32 = 2048). We also need to know the number of VLANs available to Eucalyptus. You can find this in the Web UI under the configuration tab > Clusters > Use VLAN tags. Let's assume you have the default range of VLANS from 10 to 4095. You can now calculate the number of security groups:

#security groups = min(VLAN-end, #subnet) - VLAN-start

In our example,

min(4095, 2048) - 10 =  2038

Thus we can have 2038 active security groups at any given time.

3.4.2 MANAGED-NOVLAN Mode

In this mode, Eucalyptus will fully manage the local VM instance network and provides all of the networking features Eucalyptus currently supports, including security groups, elastic IPs, etc., but it does not provide VM network isolation.

Requirements

• There is an available range of IP addresses to be used for the virtual subnets that do not interfere with the physical network. Typically these IP addresses are selected from the private IP ranges: 192.168.x.x, 10.x.x.x, etc.
• You are not running a firewall on the front end (CC) or your firewall is compatible with the dynamic changes performed by Eucalyptus when working with security groups. (Note that Eucalyptus will flush the 'filter' and 'nat' tables upon boot).
• A range of public IP addresses must be available for use by Eucalyptus.
• Front end must have installed DHCP server daemon compatible with ISC DHCP Daemon version 3.0.X

Limitations

• Layer-3 only VM isolation

Configuration

The options in eucalyptus.conf that must be configured correctly in MANAGED-NOVLAN mode are as follows:

For configuration of the front-end machine, see the front-end configuration for MANAGED mode, which is identical to MANAGED-NOVLAN.

On each node:

VNET_MODE="MANAGED-NOVLAN"
VNET_BRIDGE

Nodes must have VNET_BRIDGE set properly:

VNET_BRIDGE="xenbr0"

Caveats

See the caveats for MANAGED mode in the preceding section.

Configuration Example

For this example, we refer to the network configuration shown in Figure 2. In this case the bridge on the node is “xenbr0.”

On the front-end machine:

VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth1"
VNET_MODE="MANAGED-NOVLAN"
VNET_SUBNET="192.168.0.0"
VNET_NETMASK="255.255.0.0"
VNET_DNS="173.205.188.129"
VNET_ADDRSPERNET="32"
VNET_PUBLICIPS="173.205.188.131-173.205.188.150"

On the node(s):

VNET_MODE="MANAGED-NOVLAN"
VNET_BRIDGE="xenbr0"

3.4.3 System Mode

There is very little Eucalyptus configuration required to use SYSTEM mode—Eucalyptus mostly stays 'out of the way' in terms of VM networking.

Requirements

• The Ethernet device on the nodes that communicates with the CC must be bridged.
• A pre-existing DHCP server must be running and configured.

Limitations

• No Elastic IPs
• No Security Groups
• No VM isolation

Configuration

The options in 'eucalyptus.conf' that must be configured correctly in SYSTEM mode are as follows:

On the front end and node(s)

VNET_MODE="SYSTEM" 

In each Eucalyptus node controller's (NC) 'eucalyptus.conf' file, make sure that the parameter 'VNET_BRIDGE' is set to the name of the bridge device that is connected to your local Ethernet:

VNET_BRIDGE="xenbr0"

Make sure that what you are specifying in this field is actually a bridge, and that it is the bridge that is connected to an Ethernet network that has a DHCP server running elsewhere that is configured to hand out IP addresses dynamically. Note that your front-end machine does not need to have any bridges (this is fine, as VNET_BRIDGE is only relevant for node controllers, and will be safely ignored by the front-end components).

Configuration Example

For our SYSTEM mode example we use the configuration illustrated in Figure 1 In this example, the node has a bridge called xenbr0. The following is the proper configuration parameters for eucalyptus.conf:

On front-end machine:

VNET_MODE="SYSTEM"

On node machine(s):

VNET_BRIDGE="xenbr0"
VNET_MODE="SYSTEM"

3.4.4 Static Mode

In this mode, Eucalyptus manages VM IP address assignment by maintaining its own DHCP server with one static entry per VM.

Requirements

• The Ethernet device on the nodes that communicates with the CC must be bridged.
• A range of IP addresses must be available for use by Eucalyptus.
• NO pre-existing DHCP server on subnet. (Or, existing DHCP server must be configured to not serve instances.)
• Front end must have installed (but not configured or running) DHCP server daemon compatible with ISC DHCP Daemon version 3.0.X.

Limitations

• No Elastic IPs
• No Security Groups
• No VM isolation

Configuration

The options in 'eucalyptus.conf' that must be configured correctly in 'STATIC' mode are as follows:

On the front end (options annotated with a '#' may not be required depending on your installation, see below for details):

VNET_MODE="STATIC" 
VNET_PRIVINTERFACE
VNET_DHCPDAEMON
#VNET_DHCPUSER
VNET_SUBNET
VNET_NETMASK
VNET_BROADCAST
VNET_ROUTER
VNET_DNS
VNET_MACMAP

On each node:

VNET_MODE="STATIC" 
VNET_BRIDGE

The Eucalyptus administrator must configure the front-end machine's eucalyptus.conf file first with a valid, configured Ethernet device that is attached to the same physical Ethernet as the Eucalyptus nodes:

VNET_PRIVINTERFACE="eth0"

It is also necessary to instruct Eucalyptus about the subnet being used by the CC and NC. To do this, you must configure VNET_SUBNET, etc., as described earlier in Section 3.3: About VNET_ options. (Note that VNET_ADDRESSPERNET is not used in this mode). In addition, you must specify the router on your subnet (VNET_ROUTER), and you must provide a list of static MAC addresses/IP addresses (VNET_MACMAP), as shown in the following configuration example:

Configuration Example:

For our STATIC mode example we refer to Figure 1. As with SYSTEM mode, the nodes have a bridge named xenbr0. The following IP addresses are available for use by instances in Eucalyptus: 173.205.188.133-173.205.188.135.

On the front-end machine:

VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_PRIVINTERFACE="eth0"
VNET_MODE="STATIC"
VNET_SUBNET="173.205.188.0"
VNET_NETMASK="255.255.255.0"
VNET_BROADCAST="173.205.188.255"
VNET_ROUTER="173.205.188.1"
VNET_DNS="173.205.188.129"
VNET_MACMAP="AA:DD:11:CE:FF:ED=173.205.188.133
AA:DD:11:CE:FF:EE=173.205.188.134 AA:DD:11:CE:FF:EF=173.205.188.135"

On the nodes:

VNET_BRIDGE="xenbr0"
VNET_MODE="STATIC"

4. Multi-cluster Networking

Eucalyptus EE supports multiple clusters within a single Eucalyptus cloud. This section briefly describes how Eucalyptus manages the networking aspect of a multi-cluster setup. First, in SYSTEM or STATIC networking modes, Eucalyptus does not perform any special configuration for a multi-cluster setup. In MANAGED and MANAGED-NOVLAN modes, Eucalyptus will set up layer-two tunnels between your clusters, so that virtual machines that are in the same security group, but distributed across clusters (potentially each in their own broadcast domain), can communicate with one another. We use the 'vtun' package to handle all layer-two tunneling between clusters.

In most cases, if 'vtun' is installed on each of your CCs, multi-cluster tunneling is automatically handled by each CC.

Caveats

Depending on your networking mode and network topology, you will want to keep the following caveats in mind with respect to your Eucalyptus network configuration:

MANAGED mode - During normal operation, you will see many tunnel interfaces being created and destroyed as virtual networks are constructed and torn down.

MANAGED-NOVLAN - Your CC will need to be configured with a bridge as its private interface (VNET_PRIVINTERFACE) in order for vtun tunneling to work in this mode.

MANAGEDand MANAGED-NOVLAN - The CC attempts to auto-discover it's list of local IP addresses upon startup, but if the IP that was used to register the CC is not locally available, you can override the CC's notion of 'self' by setting the 'VNET_LOCALIP' variable in eucalyptus.conf.

MANAGEDand MANAGED-NOVLAN modes - Do not run two CCs in the same broadcast domain with tunneling enabled, this will potentially lead to a broadcast storm as tunnels start forwarding packets in a loop on your local network.

If you wish to disable tunneling altogether, set 'VNET_LOCALIP=0.0.0.0' in eucalyptus.conf.

5. Network Configuration for Components on Separate Machines

If your cluster controller (CC) and cloud controller (CLC) are running on separate hosts, the following value needs to be set within the CC’s configuration file:

VNET_CLOUDIP="<ip-of-cloud-controller>"

If you are running multiple clusters, you may wish to explicitly specify the IP address that the CC used to register with the CLC. You may set the variable within the configuration file for each of the CCs.

VNET_LOCALIP="<ip-of-cluster-controller>"

If the VNET_LOCALIP value is not set, the CC will attempt to determine this value, automatically.