Eucalyptus 2.0.0 on debian squeeze | Eucalyptus Community

Hello,

Since there are a number of problems, I thought I organize them on the same post.

Problem 1: MANAGED-NOVLAN mode seems to force a single vlan.

This is a mode that I've used successfully on the same cluster with eucalyptus 1.6.2 and never had any problems. However, using the same setup with eucalyptus 2.0.0, public IPs are assigned only when the VLAN in the UI Configuration page starts with the physical vlan on that the physical machines actually use (i.e. in my setup, the physical machines use VLAN 200, so the VLAN tag configuration HAS to be 200-4096, otherwise nothing happens...). The messages from the NC are confusing however, since kvm should know nothing about this... Attached is the eucalyptus.conf networking section, the nc.log and the libvirt log for a typical VM run:

VNET_MODE="MANAGED-NOVLAN" VNET_SUBNET="10.0.0.0" VNET_NETMASK="255.0.0.0" VNET_DNS="194.177.210.210" VNET_ADDRSPERNET="64" VNET_PUBLICIPS="62.217.120.100 62.217.120.101 62.217.120.102 62.217.120.103 62.217.120.104 62.217.120.105 62.217.120.106 62.217.120.107 62.217.120.108 62.217.120.109 62.217.120.110 62.217.120.111 62.217.120.112 62.217.120.113 62.217.120.114 62.217.120.115 62.217.120.116 62.217.120.117 62.217.120.118 62.217.120.119 62.217.120.120"

[Wed Sep 1 13:54:57 2010][001087][EUCADEBUG ] doStartNetwork() invoked [Wed Sep 1 13:54:57 2010][001087][EUCAINFO ] StartNetwork(): SUCCESS return from vnetStartNetwork 0 [Wed Sep 1 13:54:57 2010][001087][EUCAINFO ] StartNetwork(): done . . . [Wed Sep 1 13:54:57 2010][001087][EUCAINFO ] network started for instance i-526909D3 [Wed Sep 1 13:54:57 2010][001087][EUCAINFO ] retrieving images for instance i-526909D3 (disk limit=40960MB)... . . . [Wed Sep 1 13:54:59 2010][001087][EUCADEBUG ] doDescribeInstances() invoked [Wed Sep 1 13:54:59 2010][001087][EUCADEBUG ] doDescribeInstances(): instanceId=i-526909D3 publicIp=0.0.0.0 privateIp=10.0.25.5 mac=D0:0D:52:69:09:D3 vlan=200 networkIndex=5 [Wed Sep 1 13:55:05 2010][001087][EUCAINFO ] vrun(): [//usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/partition2disk /var/lib/eucalyptus/instances/admin/i-526909D3/disk 512 35529] [Wed Sep 1 13:55:07 2010][001087][EUCADEBUG ] doDescribeResource() invoked [Wed Sep 1 13:55:10 2010][001087][EUCADEBUG ] doDescribeInstances() invoked [Wed Sep 1 13:55:10 2010][001087][EUCADEBUG ] doDescribeInstances(): instanceId=i-526909D3 publicIp=0.0.0.0 privateIp=10.0.25.5 mac=D0:0D:52:69:09:D3 vlan=200 networkIndex=5 . . .

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 8192 -smp 4,sockets=4,cores=1,threads=1 -name i-526909D3 -uuid 74da4b9d-208b-f455-c789-3ea186213603 -nographic -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/i-526909D3.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -boot c -kernel /var/lib/eucalyptus/instances/admin/i-526909D3/kernel -initrd /var/lib/eucalyptus/instances/admin/i-526909D3/ramdisk -append root=/dev/sda1 console=ttyS0 -device lsi,id=scsi0,bus=pci.0,addr=0x3 -drive file=/var/lib/eucalyptus/instances/admin/i-526909D3/disk,if=none,id=drive-scsi0-0-0,boot=on,format=raw -device scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 -device e1000,vlan=0,id=net0,mac=d0:0d:52:69:09:d3,bus=pci.0,addr=0x2 -net tap,fd=103,vlan=0,name=hostnet0 -chardev file,id=serial0,path=/var/lib/eucalyptus/instances/admin/i-526909D3/console.log -device isa-serial,chardev=serial0 -usb -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

However, I cannot further troubleshoot this problem from inside a VM, due to problem 2.

Problem 2: VMs reach running state but host os cannot find root partition.

This is actually frustrating. All images used are exactly the same as those used in 1.6.2, where no problems were detected. From the NC I get:
[Wed Sep 1 13:56:21 2010][001087][EUCAERROR ] system(//usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/add_key.pl //usr/lib/eucalyptus/euca_mountwrap 32256 /var/lib/eucalyptus/instances/admin/i-526909D3/disk /tmp/sckey.4XwvEs) failed with 65280 [Wed Sep 1 13:56:21 2010][001087][EUCAERROR ] ERROR: key injection / tune2fs command failed [Wed Sep 1 13:56:21 2010][001087][EUCADEBUG ] system_output(): [//usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/gen_kvm_libvirt_xml --ramdisk --ephemeral]

Which is kind of reasonable since the host os never booted, as shown in the console log:

[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.32-5-amd64 (Debian 2.6.32-15) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-1) ) #1 SMP Tue Jun 1 04:34:03 UTC 2010 [ 0.000000] Command line: root=/dev/sda1 console=ttyS0 [ 0.000000] KERNEL supported cpus: [ 0.000000] Intel GenuineIntel [ 0.000000] AMD AuthenticAMD [ 0.000000] Centaur CentaurHauls [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009f000 (usable) [ 0.000000] BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved) [ 0.000000] BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved) [ 0.000000] BIOS-e820: 0000000000100000 - 00000000dfffb000 (usable) [ 0.000000] BIOS-e820: 00000000dfffb000 - 00000000e0000000 (reserved) [ 0.000000] BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved) [ 0.000000] BIOS-e820: 0000000100000000 - 0000000220000000 (usable) [ 0.000000] DMI 2.4 present. [ 0.000000] last_pfn = 0x220000 max_arch_pfn = 0x400000000 [ 0.000000] PAT not supported by CPU. [ 0.000000] last_pfn = 0xdfffb max_arch_pfn = 0x400000000 [ 0.000000] init_memory_mapping: 0000000000000000-00000000dfffb000 [ 0.000000] init_memory_mapping: 0000000100000000-0000000220000000 [ 0.000000] RAMDISK: 7f7a3000 - 7ffffa10 [ 0.000000] ACPI: RSDP 00000000000f8800 00014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 00000000dfffdd70 00034 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 00000000dffffdf0 00074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 00000000dfffdf50 01E22 (v01 BXPC BXDSDT 00000001 INTL 20090123) [ 0.000000] ACPI: FACS 00000000dffffd80 00040 [ 0.000000] ACPI: SSDT 00000000dfffdef0 0005E (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001) [ 0.000000] ACPI: APIC 00000000dfffddf0 0008A (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 00000000dfffddb0 00038 (v01 BOCHS BXPCHPET 00000001 BXPC 00000001) [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at 0000000000000000-0000000220000000 [ 0.000000] Bootmem setup node 0 0000000000000000-0000000220000000 [ 0.000000] NODE_DATA [0000000000011000 - 0000000000018fff] [ 0.000000] bootmap [0000000000019000 - 000000000005cfff] pages 44 [ 0.000000] (8 early reservations) ==> bootmem [0000000000 - 0220000000] [ 0.000000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000] [ 0.000000] #1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000] [ 0.000000] #2 [0001000000 - 0001680314] TEXT DATA BSS ==> [0001000000 - 0001680314] [ 0.000000] #3 [007f7a3000 - 007ffffa10] RAMDISK ==> [007f7a3000 - 007ffffa10] [ 0.000000] #4 [000009f000 - 0000100000] BIOS reserved ==> [000009f000 - 0000100000] [ 0.000000] #5 [0001681000 - 0001681071] BRK ==> [0001681000 - 0001681071] [ 0.000000] #6 [0000008000 - 000000c000] PGTABLE ==> [0000008000 - 000000c000] [ 0.000000] #7 [000000c000 - 0000011000] PGTABLE ==> [000000c000 - 0000011000] [ 0.000000] found SMP MP-table at [ffff8800000f8850] f8850 [ 0.000000] kvm-clock: cpu 0, msr 0:14eb5c1, boot clock [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000000 -> 0x00001000 [ 0.000000] DMA32 0x00001000 -> 0x00100000 [ 0.000000] Normal 0x00100000 -> 0x00220000 [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[3] active PFN ranges [ 0.000000] 0: 0x00000000 -> 0x0000009f [ 0.000000] 0: 0x00000100 -> 0x000dfffb [ 0.000000] 0: 0x00100000 -> 0x00220000 [ 0.000000] ACPI: PM-Timer IO Port: 0xb008 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled) [ 0.000000] ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000 [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000 [ 0.000000] PM: Registered nosave memory: 00000000000f0000 - 0000000000100000 [ 0.000000] PM: Registered nosave memory: 00000000dfffb000 - 00000000e0000000 [ 0.000000] PM: Registered nosave memory: 00000000e0000000 - 00000000fffbc000 [ 0.000000] PM: Registered nosave memory: 00000000fffbc000 - 0000000100000000 [ 0.000000] Allocating PCI resources starting at e0000000 (gap: e0000000:1ffbc000) [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 29 pages/cpu @ffff880008c00000 s90008 r8192 d20584 u524288 [ 0.000000] pcpu-alloc: s90008 r8192 d20584 u524288 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 1 2 3 [ 0.000000] kvm-clock: cpu 0, msr 0:8c155c1, primary cpu clock [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 2066477 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: root=/dev/sda1 console=ttyS0 [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] Initializing CPU#0 [ 0.000000] Checking aperture... [ 0.000000] No AGP bridge found [ 0.000000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 0.000000] Placing 64MB software IO TLB between ffff880008d9d000 - ffff88000cd9d000 [ 0.000000] software IO TLB at phys 0x8d9d000 - 0xcd9d000 [ 0.000000] Memory: 8189364k/8912896k available (3060k kernel code, 524696k absent, 198836k reserved, 1885k data, 584k init) [ 0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] NR_IRQS:4352 nr_irqs:440 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [ttyS0] enabled [ 0.000000] HPET: 3 timers in total, 0 timers will be used for per-cpu timer [ 0.000000] Detected 2266.898 MHz processor. [ 0.040001] Calibrating delay loop (skipped) preset value.. 4533.79 BogoMIPS (lpj=9067592) [ 0.040001] Security Framework initialized [ 0.040001] SELinux: Disabled at boot. [ 0.040073] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes) [ 0.048423] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.052853] Mount-cache hash table entries: 256 [ 0.053645] Initializing cgroup subsys ns [ 0.054198] Initializing cgroup subsys cpuacct [ 0.054711] Initializing cgroup subsys devices [ 0.055298] Initializing cgroup subsys freezer [ 0.056007] Initializing cgroup subsys net_cls [ 0.056629] CPU: L1 I cache: 32K, L1 D cache: 32K [ 0.057291] CPU: L2 cache: 4096K [ 0.057745] CPU 0/0x0 -> Node 0 [ 0.058120] mce: CPU supports 10 MCE banks [ 0.058711] Performance Events: unsupported p6 CPU model 2 no PMU driver, software events only. [ 0.063495] ACPI: Core revision 20090903 [ 0.064939] Setting APIC routing to flat [ 0.066679] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.067485] CPU0: Intel QEMU Virtual CPU version 0.12.5 stepping 03 [ 0.072001] Booting processor 1 APIC 0x1 ip 0x6000 [ 0.040001] Initializing CPU#1 [ 0.040001] CPU: L1 I cache: 32K, L1 D cache: 32K [ 0.040001] CPU: L2 cache: 4096K [ 0.040001] CPU 1/0x1 -> Node 0 [ 0.080208] CPU1: Intel QEMU Virtual CPU version 0.12.5 stepping 03 [ 0.083057] checking TSC synchronization [CPU#0 -> CPU#1]: passed. [ 0.040001] kvm-clock: cpu 1, msr 0:8c955c1, secondary cpu clock [ 0.092108] Booting processor 2 APIC 0x2 ip 0x6000 [ 0.040001] Initializing CPU#2 [ 0.040001] CPU: L1 I cache: 32K, L1 D cache: 32K [ 0.040001] CPU: L2 cache: 4096K [ 0.040001] CPU 2/0x2 -> Node 0 [ 0.103332] CPU2: Intel QEMU Virtual CPU version 0.12.5 stepping 03 [ 0.104681] checking TSC synchronization [CPU#0 -> CPU#2]: passed. [ 0.040001] kvm-clock: cpu 2, msr 0:8d155c1, secondary cpu clock [ 0.112173] Booting processor 3 APIC 0x3 ip 0x6000 [ 0.040001] Initializing CPU#3 [ 0.040001] CPU: L1 I cache: 32K, L1 D cache: 32K [ 0.040001] CPU: L2 cache: 4096K [ 0.040001] CPU 3/0x3 -> Node 0 [ 0.126590] CPU3: Intel QEMU Virtual CPU version 0.12.5 stepping 03 [ 0.128708] checking TSC synchronization [CPU#0 -> CPU#3]: passed. [ 0.040001] kvm-clock: cpu 3, msr 0:8d955c1, secondary cpu clock [ 0.136020] Brought up 4 CPUs [ 0.136022] Total of 4 processors activated (18135.18 BogoMIPS). [ 0.144202] devtmpfs: initialized [ 0.150878] regulator: core version 0.5 [ 0.152075] NET: Registered protocol family 16 [ 0.152826] ACPI: bus type pci registered [ 0.153486] PCI: Using configuration type 1 for base access [ 0.154472] mtrr: your CPUs had inconsistent variable MTRR settings [ 0.155348] mtrr: your CPUs had inconsistent MTRRdefType settings [ 0.156005] mtrr: probably your BIOS does not setup all CPUs. [ 0.156884] mtrr: corrected configuration. [ 0.157786] bio: create slab at 0 [ 0.162403] ACPI: Interpreter enabled [ 0.162957] ACPI: (supports S0 S3 S4 S5) [ 0.163544] ACPI: Using IOAPIC for interrupt routing [ 0.166542] ACPI: No dock devices found. [ 0.167106] ACPI: PCI Root Bridge [PCI0] (0000:00) [ 0.169422] pci 0000:00:01.3: quirk: region b000-b03f claimed by PIIX4 ACPI [ 0.170437] pci 0000:00:01.3: quirk: region b100-b10f claimed by PIIX4 SMB [ 0.176203] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11) [ 0.177233] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11) [ 0.178264] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11) [ 0.179184] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11) [ 0.180139] vgaarb: loaded [ 0.180638] PCI: Using ACPI for IRQ routing [ 0.181007] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.181731] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.192024] Switching to clocksource kvm-clock [ 0.193839] pnp: PnP ACPI init [ 0.194294] ACPI: bus type pnp registered [ 0.195527] pnp: PnP ACPI: found 6 devices [ 0.196140] ACPI: ACPI bus type pnp unregistered [ 0.202200] NET: Registered protocol family 2 [ 0.202980] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.206509] TCP established hash table entries: 524288 (order: 11, 8388608 bytes) [ 0.214779] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) [ 0.216609] TCP: Hash tables configured (established 524288 bind 65536) [ 0.217666] TCP reno registered [ 0.218155] NET: Registered protocol family 1 [ 0.218805] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.219623] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.220423] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.221385] Unpacking initramfs... [ 0.437914] Freeing initrd memory: 8562k freed [ 0.441100] audit: initializing netlink socket (disabled) [ 0.441824] type=2000 audit(1283338582.440:1): initialized [ 0.443276] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.446011] VFS: Disk quotas dquot_6.5.2 [ 0.446612] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.447527] msgmni has been set to 16011 [ 0.449151] alg: No test for stdrng (krng) [ 0.449839] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) [ 0.450787] io scheduler noop registered [ 0.451289] io scheduler anticipatory registered [ 0.451927] io scheduler deadline registered [ 0.452555] io scheduler cfq registered (default) [ 0.454453] Linux agpgart interface v0.103 [ 0.455073] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.456129] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 0.457256] input: Macintosh mouse button emulation as /devices/virtual/input/input0 [ 0.458285] PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 0.460109] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 0.460769] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 0.461546] mice: PS/2 mouse device common for all mice [ 0.462352] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0 [ 0.463230] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 0.464952] rtc0: alarms up to one day, 114 bytes nvram, hpet irqs [ 0.465916] cpuidle: using governor ladder [ 0.466452] cpuidle: using governor menu [ 0.467021] No iBFT detected. [ 0.467684] TCP cubic registered [ 0.468241] NET: Registered protocol family 10 [ 0.469322] lo: Disabled Privacy Extensions [ 0.470094] Mobile IPv6 [ 0.470415] NET: Registered protocol family 17 [ 0.471109] registered taskstats version 1 [ 0.472039] rtc_cmos 00:01: setting system clock to 2010-09-01 10:56:21 UTC (1283338581) [ 0.473096] Initalizing network drop monitor service [ 0.473933] Freeing unused kernel memory: 584k freed [ 0.474687] Write protecting the kernel read-only data: 4208k Loading, please wait... [ 0.494763] udev: starting version 157 [ 0.535776] SCSI subsystem initialized [ 0.542464] usbcore: registered new interface driver usbfs [ 0.543190] usbcore: registered new interface driver hub [ 0.544088] usbcore: registered new device driver usb [ 0.544423] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11 [ 0.544450] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11 [ 0.546179] Intel(R) PRO/1000 Network Driver - version 7.3.21-k5-NAPI [ 0.546180] Copyright (c) 1999-2006 Intel Corporation. [ 0.546344] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10 [ 0.546366] e1000 0000:00:02.0: PCI INT A -> Link[LNKB] -> GSI 10 (level, high) -> IRQ 10 [ 0.563700] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 0.564972] FDC 0 is a S82078B [ 0.567712] uhci_hcd: USB Universal Host Controller Interface driver [ 0.849491] e1000: 0000:00:02.0: e1000_probe: (PCI:33MHz:32-bit) d0:0d:52:69:09:d3 [ 0.892390] e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection [ 0.893488] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 [ 0.894231] sym53c8xx 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11 [ 0.896810] sym0: <895a> rev 0x0 at pci 0000:00:03.0 irq 11 [ 0.899738] sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking [ 0.904486] sym0: SCSI BUS has been reset. [ 0.913247] scsi0 : sym-2.2.3 [ 0.914287] scsi1 : ata_piix [ 0.914811] scsi2 : ata_piix [ 0.915237] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 irq 14 [ 0.916172] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 irq 15 [ 0.917203] uhci_hcd 0000:00:01.2: PCI INT D -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11 [ 0.918454] uhci_hcd 0000:00:01.2: UHCI Host Controller [ 0.919129] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 [ 0.920104] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c020 [ 0.920884] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001 [ 0.921920] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 0.922899] usb usb1: Product: UHCI Host Controller [ 0.923552] usb usb1: Manufacturer: Linux 2.6.32-5-amd64 uhci_hcd [ 0.924315] usb usb1: SerialNumber: 0000:00:01.2 [ 0.925080] usb usb1: configuration #1 chosen from 1 choice [ 0.925997] hub 1-0:1.0: USB hub found [ 0.926575] hub 1-0:1.0: 2 ports detected Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Waiting for root file system ... [ 3.937031] scsi 0:0:0:0: Direct-Access QEMU QEMU HARDDISK 0.12 PQ: 0 ANSI: 3 [ 3.938103] scsi target0:0:0: tagged command queuing enabled, command queue depth 16. [ 3.939074] scsi target0:0:0: Beginning Domain Validation [ 3.940630] scsi target0:0:0: Domain Validation skipping write tests [ 3.941422] scsi target0:0:0: Ending Domain Validation [ 3.952424] sd 0:0:0:0: [sda] 84107264 512-byte logical blocks: (43.0 GB/40.1 GiB) [ 3.953428] sd 0:0:0:0: [sda] Write Protect is off [ 3.954424] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 3.956187] sda: sda1 sda2 sda3 [ 3.958381] sd 0:0:0:0: [sda] Attached SCSI disk done. Begin: Running /scripts/local-premount ... done. mount: mounting /dev/sda1 on /root failed: No such file or directory Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory done. mount: mounting /sys on /root/sys failed: No such file or directory mount: mounting /proc on /root/proc failed: No such file or directory Target filesystem doesn't have /sbin/init. No init found. Try passing init= bootarg.

BusyBox v1.15.3 (Debian 1:1.15.3-1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

/bin/sh: can't access tty; job control turned off
(initramfs)

libvirt.xml is:

i-526909D3

hvm
/var/lib/eucalyptus/instances/admin/i-526909D3/kernel
/var/lib/eucalyptus/instances/admin/i-526909D3/ramdisk
root=/dev/sda1 console=ttyS0

8388608
4

/usr/bin/kvm

The same results were obtained with the stock Ubuntu image and some custom images from 1.6.2. Guest os is Debian Squeeze. As to the KVM/libvirt versions:

I 'll be extremely thankful for any advice. I had to upgrade the whole cluster from 1.6.2 due to a dependency problem with squeeze, and I am now stuck with a non-working eucalyptus 2.0.0 installation. Thanks!

Fri, 09/03/2010 - 22:38

graziano

Offline

Joined: 01/14/2010

Hello, in MANAGED-NOVLAN

in MANAGED-NOVLAN Eucalyptus shouldn't use any VLAN.. this is puzzling. Can you try to stop the cloud, restart the CC using cleanstop and cleanstart and see if you still have problem with VLAN?

For the second problem, can you post your eucalyptus.conf (sans the comments)? Have you looked into /var/log/libvirt/qemu/.log to see if there is any obvious error? Did you upload new images after upgrading of reuse the one already uploaded?

PS: thanks for starting a new thread!

cheers
graziano

Sat, 09/04/2010 - 02:37

vagoskar

Joined: 03/20/2010

Hi, Thanks for the reply. To

Hi,

Thanks for the reply. To answer one of your questions, the upgrade from 1.6.2 did not work for me, so I purged everything and started from scratch, with newly uploaded images. I took the whole day yesterday to figure out the second problem. The boot problem was quite puzzling, as when running images "by hand" (e.g. with: kvm -m 256 -drive file=image2.qcow,if=scsi,index=0,boot=on -boot c -net nic,vlan=0,model=e1000,macaddr=00:16:e3:de:ad:01 -net tap -nographic -vnc :0 or qemu -m 8192 -smp 4,sockets=4,cores=1,threads=1 -drive file=image.img,index=0,if=scsi,boot=on -boot c -net nic,vlan=0,model=e1000,macaddr=00:16:e3:de:ad:01 -net tap -nographic -vnc :0) there was no problem booting, but custom/eucalyptus libvirt XML files (as above) and virsh create were failing. After successive re-installs, upgrades of qemu and seabios on one of the nodes I finally figured out that libvirt 0.8.3 on Debian Squeeze cannot run qcow images. So, only raw images can be passed, which probably has something to do with qemu (finally in 0.12.5) respecting the options that libvirt passes. For example, the libvirt passes (copied from a eucalyptus qemu.log):

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 8192 -smp 4,sockets=4,cores=1,threads=1 -name i-423D07E1 -uuid 08a17b0e-3ea7-02fa-d8d4-a79657151cc9 -nographic -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/i-423D07E1.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -boot c -kernel /var/lib/eucalyptus/instances/admin/i-423D07E1/kernel -initrd /var/lib/eucalyptus/instances/admin/i-423D07E1/ramdisk -append root=/dev/sda1 console=ttyS0 -device lsi,id=scsi0,bus=pci.0,addr=0x3 -drive file=/var/lib/eucalyptus/instances/admin/i-423D07E1/disk,if=none,id=drive-scsi0-0-0,boot=on,format=raw -device scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 -device e1000,vlan=0,id=net0,mac=d0:0d:42:3d:07:e1,bus=pci.0,addr=0x2 -net tap,fd=103,vlan=0,name=hostnet0 -chardev file,id=serial0,path=/var/lib/eucalyptus/instances/admin/i-423D07E1/console.log -device isa-serial,chardev=serial0 -usb -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

I am guessing the problem is at the "format=raw" directive, so probably a more elaborate XML file needs to be passed by eucalyptus to ensure that the correct image format is used -or simply use raw images :-). Still, I will keep investigating, but at least my custom images are booting when converted to raw with qemu-img.

As to the MANAGED-NOVLAN problem, it still persists (I have done multiple cleanrestart cycles before posting - I am used to network misconfigurations :-D). Although eucalyptus assigns public IPs correctly, I still cannot access them (tried ssh and ping after euca-authorize, even tried setting a different group other than default). Maybe this is a combined problem (part eucalyptus - part image) - I will try to troubleshoot from within a VM.

Is there a way to alter the libvirt XML template that eucalyptus uses to boot instances? It would be a great help to have an interactive console for troubleshooting (using vnc preferably :-)).

Thank you again!

PS, this is my eucalyptus.conf sans comments:
EUCALYPTUS="/" EUCA_USER="eucalyptus" DISABLE_ISCSI="Y" DISABLE_DNS="Y" ENABLE_WS_SECURITY="Y" CLOUD_OPTS="" LOGLEVEL="DEBUG" CC_PORT="8774" SCHEDPOLICY="ROUNDROBIN" POWER_IDLETHRESH="300" POWER_WAKETHRESH="300" NODES="euca-node-30 euca-node-29 euca-node-27 euca-node-26 euca-node-24 euca-node-23 euca-node-22 euca-node-21 euca-node-20 euca-node-19 euca-node-18 euca-node-17 euca-node-16 euca-node-15 euca-node-14 euca-node-13 euca-node-12 euca-node-10 euca-node-9 euca-node-8 euca-node-7 euca-node-6 euca-node-5 euca-node-4 euca-node-3 euca-node-2 euca-node-1 euca-node-25" NC_SERVICE="axis2/services/EucalyptusNC" NC_PORT="8775" HYPERVISOR="kvm" USE_VIRTIO_DISK="0" USE_VIRTIO_ROOT="0" USE_VIRTIO_NET="0" INSTANCE_PATH="/var/lib/eucalyptus/instances" VNET_PUBINTERFACE="bond0" VNET_PRIVINTERFACE="bond0" VNET_BRIDGE="br-public" VNET_DHCPDAEMON="/usr/sbin/dhcpd3" VNET_DHCPUSER="dhcpd" VNET_MODE="MANAGED-NOVLAN" VNET_SUBNET="10.0.0.0" VNET_NETMASK="255.0.0.0" VNET_DNS="194.177.210.210" VNET_ADDRSPERNET="64" VNET_PUBLICIPS="62.217.120.100 62.217.120.101 62.217.120.102 62.217.120.103 62.217.120.104 62.217.120.105 62.217.120.106 62.217.120.107 62.217.120.108 62.217.120.109 62.217.120.110 62.217.120.111 62.217.120.112 62.217.120.113 62.217.120.114 62.217.120.115 62.217.120.116 62.217.120.117 62.217.120.118 62.217.120.119 62.217.120.120"