I am still unclear on a few things, Eucalyptus provides us with virtualized hardware... and can run instances, but what if the size of an instance is larger than memory available on any one single node? Also, how can one bring high performance computing using the eucalyptus cloud, what are the tools that will allow parallel processing.
is Rocks one such thing? what alternatives do i have?
Hello,
the question about the memory is really a question on what the hypervisor will do if it ran out of memory. Which depends on the hyperviros you use and the configuration of it. My first comment here is that HPC and working from swap doesn't really go well together: I highly doubts that you would get any decent performance once you are spilling our of memory.
Eucalyptus does provide you with a way to start multiple images: what you put into the images is up to you, so it really depends on the application you want to run. Also Eucalyptus is to be installed on an already configure cluster, where one machine will act as CLC/CC and the other will run as NC: so whatever you want to use as the underlying OS is really up to you. We do provide packages for CentOS, ubuntu/debian and opensuse. Alternatively you could compile it from source.
cheers
graziano
I did some tests with HPC on Eucalyptus. It works, you can put a grid middleware like gLite on VMs, you can use any sort of parallelizing libraries. We ended up using another system that integrates with existing infrastructure, but initial experiments with Eucalyptus proved it's a useful platform too. However, virtualization adds some overhead - CPU power stays almost the same, but network latency is severely degraded, so if your software require frequent communication between the nodes, it's better to use physical hardware (or at least PCI passthrough).
Are you new to parallel processing? I suggest to start with Hadoop. You can grasp the main concepts and start writing hello-world style apps in an hour or less. Hadoop is also immune to failure of worker nodes, something that often happens in large scale cloud deployments. Not all problems can be paralellized Hadoop-way though.
I know that OpenMP and OpenMPI are used in my place, although personally I don't develop with them. See wikipedia articles on both of these for intro, they look quite thorough. In our experiments the performance were degraded compared to physical hardware, but maybe it's the type of computations that matter, not the library.
From what i've read and understood, the solution i'm looking for is Identical to symmetric multiprocessing, however, porting it onto the cloud is not very well documented. would you know anything about this?
from what i could find out, only the proprietary vSMP is available for implementation of the SMP architecture onto the cloud.
please send in the papers to tejapratapb2005@vit.ac.in
i was busy the last week, too much workload, sorry for the late response... if you could please mail the papers to me i'd be really thankful.
Teja
Any way we can do some load balancing between nodes ? Similar to what Apache has for server load balancing ? Do these nodes have any File Extensions ? or are they treated like i-nodes (Linux) ?
Well I'm not sure about the alternative, but I have gone through Google and looked upon for some idea for Eucalyptus, but still didn't find any clue about it. What does Eucalyptus actually does? Is it a software same as Paycor ? Don't mind me, but I also need to find the answer for it.
If I have problems with my pc, I usually consult pc technicians to fix computer errors. And they can give an instant solution.
[quote=zacsifac]Any way we can do some load balancing between nodes ? Similar to what Apache has for server load balancing ? Do these nodes have any File Extensions ? or are they treated like i-nodes (Linux) ?[/quote]
I know there are at least 2 entries in the wiki on load balancing. It really depends on what you actually mean by 'load balancing'. Yes, I know that sounds like an obvious sort of question, but it really isn't. I wrote a wiki entry on the subject as a follow up to another individual's entry. Are you familiar with HAProxy?
Walt