newbie quesiton: practical applications for clusters?

Hello,
I have no experience with clusters, but I need to prepare a short
presentation on
them.

There is a ton of info out there on how to make a linux cluster,
beowulf, open Mosix,
knoppix, etc. There seems to be info on performance, costs, etc, as
well.

I was wondering though if anybody could offer an opinion on practical
applications
for clusters.

I have heard of clusters calculating primes and other mathematical
feats of strength.

Can you just throw a cluster together and have everything work faster?

What are some applications that cry out for a cluster?

Thanks,
Mike

On Fri, 12 May 2006 13:47:51 -0700, bsd_mike wrote:

>
>
> Hello,
> I have no experience with clusters, but I need to prepare a short
> presentation on
> them.
>
> There is a ton of info out there on how to make a linux cluster, beowulf,
> open Mosix,
> knoppix, etc. There seems to be info on performance, costs, etc, as
> well.
>
> I was wondering though if anybody could offer an opinion on practical
> applications
> for clusters.
>
> I have heard of clusters calculating primes and other mathematical feats
> of strength.
>
> Can you just throw a cluster together and have everything work faster?
>
> What are some applications that cry out for a cluster?
>
> Thanks,
> Mike

Hi.

Clusters can do wonder for you or can be expensive useless bricks.

It depend on the problems you are trying to solve.
Some problems can be solved faster by splitting them between may machines.

Examples : A weather simulation can be split : each computers nodes
calculate the values for it's own little cube of atmosphere.

But some problems cannot be separated. If each step of the calculations is
required to prepare for the next step, then this problem is not going to
be solved any faster by a cluster.
So no, some problems are not going to be solved faster on a cluster
solution.

Best problem for a cluster ? Any kind of problem that can be splitted in
little parts that can be calculated independently of each other parts.

Now: Not all cluster / system /server uses cluster technology in the same
way.

The problem solving cluster is a way of solving a mathematical problem.

Load balancing can be another.
Ex : Google : There is not a single computer powerful enough to re-index
the Google index of the web pages of the Internet. So each computer in a
Google cluster can do it's own re-indexing.

If a web server cannot do the job, maybe two or much more in a load
balanced way can do the job.

Now : not all program can be load balanced or cutted in small parts.
You can have 2 servers working in a cluster and they will all check on
each other. If one fails then the other will take over. In that case you
have a server working 100% and one working 0% until server A fails.

The load balancing vs spreading of task is not usually for you to decide.
it is the analyst/programmer who designed and coded the software that will
decide of the behaviour of the service.

Hope it help.

Hope it helps.

noEMA wrote:
> On Fri, 12 May 2006 13:47:51 -0700, bsd_mike wrote:
>
> >
> >
> > Hello,
> > I have no experience with clusters, but I need to prepare a short
> > presentation on
> > them.
> >
> > There is a ton of info out there on how to make a linux cluster, beowulf,
> > open Mosix,
> > knoppix, etc. There seems to be info on performance, costs, etc, as
> > well.
> >
> > I was wondering though if anybody could offer an opinion on practical
> > applications
> > for clusters.
> >
> > I have heard of clusters calculating primes and other mathematical feats
> > of strength.
> >
> > Can you just throw a cluster together and have everything work faster?
> >
> > What are some applications that cry out for a cluster?
> >
> > Thanks,
> > Mike
>
> Hi.
>
> Clusters can do wonder for you or can be expensive useless bricks.
>
> It depend on the problems you are trying to solve.
> Some problems can be solved faster by splitting them between may machines.
>
> Examples : A weather simulation can be split : each computers nodes
> calculate the values for it's own little cube of atmosphere.

I second that. Weather modeling is a very good application of clusters
where each processor takes up a set of parameters and runs the process
finally reporting back to the mother-node when finished. Protein/Gene
manipulation or comparision is another where there could be a set of
amino acids that we want to use for comparision in a set of proteins
(many sets compared to many other sets). This can be done by
distributing tasks to other processors, who independently work things
out and report back.

Another application is to get multiplication tables for prime numbers
between 1-10000. Mother node gets the prime numbers one after the other
and assigns it to different processors.


> But some problems cannot be separated. If each step of the calculations is
> required to prepare for the next step, then this problem is not going to
> be solved any faster by a cluster.

Yes, I think that is right. But again, if we write programs in a manner
that partial results can be used for processing with reliability -or-
if processes "know" how to wait for dependent results, we could still
make the whole process faster. For example, problem A has several ways
of being solved. We send each solution A1-A100 to different processors
but each solution has to wait for a calculation in-between. We can
write programs such that solutions wait till this intermediate
calculation is obtained. As soon as it appears, A1-A100 start working
again till the end. Some may output faster than the others, but still
its possible to speed up processing. That needs a lot of thinking and
coding, though.


> So no, some problems are not going to be solved faster on a cluster
> solution.

True, some problems wont work any faster on a cluster. For example,
find prime numbers between 1-500. Better is it to process it on a
single machine.


> Load balancing can be another.
> Ex : Google : There is not a single computer powerful enough to re-index
> the Google index of the web pages of the Internet. So each computer in a
> Google cluster can do it's own re-indexing.

Thats a good example. I never thought about that. Everyday there's more
to learn.


noEMA had some really good points and hope the discussion guided you in
a proper direction.

Chirag Shukla