I have configured Eucalyptus on Ubuntu 9.10. I am able to run instances but the ipaddresses which the instances are getting in the range 172.19.1.x whereas I have set VNET_MOD="SYSTEM" in both CC and on Nodes. I don't know what wrong is happening that Eucalyptus is not able to pickup IPAddresses from the other DHCP server.
1.6-devel
Node eucalyptus.conf
EUCALYPTUS="/"
EUCA_USER="eucalyptus"
DISABLE_DNS="Y"
ENABLE_WS_SECURITY="Y"
LOGLEVEL="DEBUG"
CC_PORT="8774"
SCHEDPOLICY="ROUNDROBIN"
POWER_IDLETHRESH="300"
POWER_WAKETHRESH="300"
NODES=""
NC_SERVICE="axis2/services/EucalyptusNC"
NC_PORT="8775"
HYPERVISOR="kvm"
INSTANCE_PATH="/var/lib/eucalyptus/instances"
VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth0"
VNET_BRIDGE="br0"
VNET_MODE="SYSTEM"
Server eucalyptus.conf file
EUCALYPTUS="/"
EUCA_USER="eucalyptus"
DISABLE_DNS="Y"
ENABLE_WS_SECURITY="Y"
LOGLEVEL="DEBUG"
CC_PORT="8774"
SCHEDPOLICY="ROUNDROBIN"
POWER_IDLETHRESH="300"
POWER_WAKETHRESH="300"
NODES=" 172.16.4.101"
NC_SERVICE="axis2/services/EucalyptusNC"
NC_PORT="8775"
HYPERVISOR="kvm"
INSTANCE_PATH="/var/lib/eucalyptus/instances"
VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth0"
VNET_MODE="SYSTEM"
Regards
Ankush
cloud-debug.log
17:23:06 DEBUG [NetworkStateHandler:New I/O client worker #2-1] Adding node service tags: [http://172.16.4.101:8775/axis2/services/EucalyptusNC]
17:23:06 DEBUG [StatefulNamedRegistry:New I/O client worker #2-4] Network [availableNetworkIndexes=[3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], assignedNetworkIndexes=[2], name=admin-default, networkName=default, clusterTokens={ubuntuec1=NetworkToken [cluster=ubuntuec1, indexes=[2], name=admin-default, networkName=default, userName=admin, vlan=10]}, rules=[PacketFilterRule{destUserName='admin'destNetworkName='default', policy='firewall-open', protocol='tcp', portMin=22, portMax=22, sourceCidrs=[0.0.0.0/0], peers=[], sourceNetworkNames=[], sourceUserNames=[]}, PacketFilterRule{destUserName='admin'destNetworkName='default', policy='firewall-open', protocol='tcp', portMin=80, portMax=80, sourceCidrs=[0.0.0.0/0], peers=[], sourceNetworkNames=[], sourceUserNames=[]}], userName=admin]
17:23:06 DEBUG [ChannelStateMonitor:New I/O client worker #2-1] :1265716480.0000:ResourceStateHandler/ChannelStateMonitor.split.36:eucalyptus:eucalyptus:SOCKET_CLOSE:1265716386223:126:/172.16.4.107:57885:/172.16.4.107:8774:
17:23:06 DEBUG [ChannelStateMonitor:New I/O client worker #2-4] :1265716480.0000:NetworkStateHandler/ChannelStateMonitor.split.36:eucalyptus:eucalyptus:SOCKET_CLOSE:1265716386224:119:/172.16.4.107:57884:/172.16.4.107:8774:
17:23:06 DEBUG [EntityWrapper:New I/O client worker #2-2] :1265716480.0000:db/EntityWrapper..106:eucalyptus:eucalyptus:CREATE:END:1:eucalyptus:edu.ucsb.eucalyptus.cloud.cluster.VmTypes.update(VmTypes.java:111):fdc4bc50-5470-4d02-8cdf-50c021e9979b:
17:23:06 DEBUG [StatefulNamedRegistry:New I/O client worker #2-2] Network [availableNetworkIndexes=[3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], assignedNetworkIndexes=[2], name=admin-default, networkName=default, clusterTokens={ubuntuec1=NetworkToken [cluster=ubuntuec1, indexes=[2], name=admin-default, networkName=default, userName=admin, vlan=10]}, rules=[PacketFilterRule{destUserName='admin'destNetworkName='default', policy='firewall-open', protocol='tcp', portMin=22, portMax=22, sourceCidrs=[0.0.0.0/0], peers=[], sourceNetworkNames=[], sourceUserNames=[]}, PacketFilterRule{destUserName='admin'destNetworkName='default', policy='firewall-open', protocol='tcp', portMin=80, portMax=80, sourceCidrs=[0.0.0.0/0], peers=[], sourceNetworkNames=[], sourceUserNames=[]}], userName=admin]
17:23:06 DEBUG [Network:New I/O client worker #2-2] :1265716480.0000:Network/DelegatingMethodAccessorImpl.invoke.43:eucalyptus:eucalyptus:allocated:network=admin-default:cluster=ubuntuec1:networkIndex=2:
cc.log
[Tue Feb 9 17:23:48 2010][001322][EUCAWARN ] VNET_LOCALIP not set, tunneling is disabled
[Tue Feb 9 17:23:48 2010][001322][EUCADEBUG ] DescribeResources(): called 5
[Tue Feb 9 17:23:48 2010][001322][EUCADEBUG ] cache: i-43CE0948 0.0.0.0 172.19.1.2
[Tue Feb 9 17:23:48 2010][001322][EUCADEBUG ] refresh_resources(): called
[Tue Feb 9 17:23:48 2010][001322][EUCADEBUG ] calling http://172.16.4.101:8775/axis2/services/EucalyptusNC"
[Tue Feb 9 17:24:18 2010][001153][EUCADEBUG ] failed to attach tunnels for vlan 10 during maintainNetworkState()
[Tue Feb 9 17:24:18 2010][001153][EUCAERROR ] network state maintainance failed
[Tue Feb 9 17:24:18 2010][001214][EUCAERROR ] bad input params to vnetAttachTunnels()
[Tue Feb 9 17:24:18 2010][001214][EUCADEBUG ] failed to attach tunnels for vlan 10 during maintainNetworkState()
[Tue Feb 9 17:24:18 2010][001214][EUCAERROR ] network state maintainance failed
[Tue Feb 9 17:24:18 2010][001322][EUCAINFO ] node=172.16.4.101 mem=1793/1665 disk=66271/65757 cores=2/1
[Tue Feb 9 17:24:18 2010][001322][EUCADEBUG ] refresh_resources(): done
[Tue Feb 9 17:24:18 2010][001322][EUCADEBUG ] DescribeResources(): done
[Tue Feb 9 17:24:18 2010][001322][EUCAERROR ] bad input params to vnetAttachTunnels()
[Tue Feb 9 17:24:18 2010][001322][EUCADEBUG ] failed to attach tunnels for vlan 10 during maintainNetworkState()
[Tue Feb 9 17:24:18 2010][001322][EUCAERROR ] network state maintainance failed
cloud-error.log
17:13:57 [ClusterAllocator:New I/O client worker #2-2] ERROR Number of running VMs is greater than number of assigned addresses!
cloud-output.log
17:25:18 DEBUG tClusterMessageDispatcher | :1265716608.0000:AbstractClusterMessageDispatcher/AbstractClusterMessageDispatcher.operationComplete.133:eucalyptus:eucalyptus:MSG_SENT:
axis2c.log
[Tue Feb 9 17:25:54 2010] [error] error.c(94) OXS ERROR [x509.c:287 in openssl_x509_get_subject_key_identifier] oxs defualt error , The extenension index of NID_subject_key_identifier is not valid
httpd-cc_error_log
WARNING: Host declarations are global. They are not limited to the scope you declared them in.
Wrote 0 deleted host decls to leases file.
Wrote 0 new dynamic host decls to leases file.
Wrote 0 leases to leases file.
Listening on LPF/eth0/00:16:35:ab:4c:d3/euca
Sending on LPF/eth0/00:16:35:ab:4c:d3/euca
Sending on Socket/fallback/fallback-net
iptables-restore: line 11 failed
iptables-restore failed
nc.log
[Tue Feb 9 17:26:48 2010][001611][EUCAWARN ] could not execute arp cache populator script, check httpd log for errors
[Tue Feb 9 17:26:48 2010][001611][EUCAWARN ] could not execute arp cache populator script, check httpd log for errors
[Tue Feb 9 17:26:53 2010][001611][EUCAWARN ] could not execute arp cache populator script, check httpd log for errors
[Tue Feb 9 17:26:53 2010][001611][EUCAWARN ] could not execute arp cache populator script, check httpd log for errors
[Tue Feb 9 17:26:53 2010][001611][EUCADEBUG ] doDescribeResource() invoked
[Tue Feb 9 17:26:54 2010][001611][EUCADEBUG ] doDescribeInstances() invoked
[Tue Feb 9 17:26:58 2010][001611][EUCAWARN ] could not execute arp cache populator script, check httpd log for errors
[Tue Feb 9 17:26:58 2010][001611][EUCAWARN ] could not execute arp cache populator script, check httpd log for errors
[Tue Feb 9 17:27:00 2010][001611][EUCADEBUG ] doDescribeResource() invoked
[Tue Feb 9 17:27:00 2010][001611][EUCADEBUG ] doDescribeInstances() invoked
Hello,
first of all, if you changed network configuration you need to stop and start the CC using /etc/init.d/eucalyptus-cc cleanstop and then cleanstart or adding CLEAN=1 if you are using upstart. If you still have problem after, you may want to check that the dhcp request from the instances are making it to the dhcp server in your subnet.
cheers
graziano
Thanks for the suggestion. Even for doing the cleanstop and cleanstart of eucalyptus, problem did not get resolved. However, I find a workaround.
On the node add the below iptables rule
iptables -A FORWARD -p UDP --sport 123 --dport 123 -j LOG --log-level 0
Run the below command on the node whenever any new instance is turned on
/usr/share/eucalyptus/populate_arp.pl
Regards
Ankush
Hello,
this is interesting: can you check the httpd-nc.log? Also could you post the output iptables -L on the nc? Your iptables will log ntp messages coming in: I wonder why that works but you cannot intercept the dhcp requests ...
cheers
graziano
/bin/dd if=/dev/zero of=/var/lib/eucalyptus/instances/admin/i-56CE09B1/disk bs=1M seek=4110 count=1 >/dev/null 2>&1
/sbin/parted --script /var/lib/eucalyptus/instances/admin/i-56CE09B1/disk mkpartfs primary ext2 2050112s 7146112s
/sbin/parted --script /var/lib/eucalyptus/instances/admin/i-56CE09B1/disk mkpartfs primary linux-swap 7146113s 100%
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
/bin/dd if=/dev/zero of=/var/lib/eucalyptus/instances/admin/i-53150932/disk bs=1M seek=4110 count=1 >/dev/null 2>&1
/sbin/parted --script /var/lib/eucalyptus/instances/admin/i-53150932/disk mkpartfs primary ext2 2050112s 7146112s
/sbin/parted --script /var/lib/eucalyptus/instances/admin/i-53150932/disk mkpartfs primary linux-swap 7146113s 100%
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
ERROR: Disallowed command //usr/share/eucalyptus/populate_arp.pl
iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:bootps
ACCEPT tcp -- anywhere anywhere tcp dpt:bootps
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere 192.168.122.0/24 state RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
LOG udp -- anywhere anywhere udp spts:bootps:bootpc dpts:bootps:bootpc LOG level info
LOG udp -- anywhere anywhere udp spt:ntp dpt:ntp LOG level emerg
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Hello,
are you finding those errors on the httpd_nc.log? Can you run the command populate_arp.pl by hand as root? If so, can you then try to run /usr/lib/eucalyptus/euca_rootwrap /usr/share/eucalyptus/populate_arp.pl? How did you install eucalyptus? From source or using packages (and if so which ones)?
cheers
graziano
Hello,
We can use /usr/share/eucalyptus/populate_arp.pl to get IP address for the normal image, but for our own created image it doesn't work. Does type /usr/share/eucalyptus/populate_arp.pl the only method, no other way to fix this problem?
Thank you
Hello,
this is a weird problem: populate_arp.pl is a perl script which look for iptables log entries, and from that it extrapolate the IP assigned to the instance. Is your instance talking to your dhcp server? Can you login to it?
cheers
graziano
I have installed Ubuntu 9.10 64-bit with which Eucalyptus packages are bundled. However, after installing Ubuntu, I have upgraded the eucalyptus packages on both CC and NC.
Yes, those logs are from httpd_nc.log
The instance is talking to the DHCP server of the Organization and I am able to login onto the instance.
Regards
Ankush
Hello
there is this bug https://bugs.launchpad.net/ubuntu/+source/eucalyptus/+bug/461829 : are you using the packaged from Eucalyptus or from Ubuntu? Can you tell us the version number of the packages you are using? Also can you run populate* as the Eucalyptus user? And can you run it using /usr/share/eucalyptus/euca_rootwrap populate*?
cheers
graziano