Re: [PHP] LWP::Parallel in PHP

On Thu, 28 Aug 2003 20:25:05 +0300, you wrote:

>I am looking for PHP analog of Perl LWP::Parallel.
>I need to fetch several URL(pages) from PHP at the same time.
>I have a script which fetch 5-10 URL, each URL fetched 0.5 - 2 sec.
>Totally it's about 10 seconds in average.
>I suppose if I fetched them parallel I would speed up the script 5 times.
>I am looking mostly for PHP solutions.
>(Hope they exist :)

Think you're out of luck. Yes, it's a problem I've run up against more than
once. There's no thread support in PHP [4. Anyone know if it's in 5?].

I suppose you might be able to hack something together by spawning external
processes, but... might as well just do it all externally then.

php.net/pcntl_fork



On Thursday 28 August 2003 11:09 am, David Otton wrote:
> On Thu, 28 Aug 2003 20:25:05 +0300, you wrote:
> >I am looking for PHP analog of Perl LWP::Parallel.
> >I need to fetch several URL(pages) from PHP at the same time.
> >I have a script which fetch 5-10 URL, each URL fetched 0.5 - 2 sec.
> >Totally it's about 10 seconds in average.
> >I suppose if I fetched them parallel I would speed up the script 5 times.
> >I am looking mostly for PHP solutions.
> >(Hope they exist :)
>
> Think you're out of luck. Yes, it's a problem I've run up against more than
> once. There's no thread support in PHP [4. Anyone know if it's in 5?].
>
> I suppose you might be able to hack something together by spawning external
> processes, but... might as well just do it all externally then.

--
"The missionaries go forth to Christianize the savages- as if the savages
weren't dangerous enough already."

-Edward Abbey

This isn't the best solution but it might help bring down the total
time. Can you set up a shell script to retrieve the content from a URL
(in PHP if you wish) and then from your web app spawn 5 processes, with
destination temporary files for the data which you can then poll for
completion (microsleep here and there :)?

Cheers,
Rob.

On Thu, 2003-08-28 at 18:49, Evan Nemerson wrote:
> php.net/pcntl_fork
>
>
>
> On Thursday 28 August 2003 11:09 am, David Otton wrote:
> > On Thu, 28 Aug 2003 20:25:05 +0300, you wrote:
> > >I am looking for PHP analog of Perl LWP::Parallel.
> > >I need to fetch several URL(pages) from PHP at the same time.
> > >I have a script which fetch 5-10 URL, each URL fetched 0.5 - 2 sec.
> > >Totally it's about 10 seconds in average.
> > >I suppose if I fetched them parallel I would speed up the script 5 times.
> > >I am looking mostly for PHP solutions.
> > >(Hope they exist :)
> >
> > Think you're out of luck. Yes, it's a problem I've run up against more than
> > once. There's no thread support in PHP [4. Anyone know if it's in 5?].
> >
> > I suppose you might be able to hack something together by spawning external
> > processes, but... might as well just do it all externally then.
>
> --
> "The missionaries go forth to Christianize the savages- as if the savages
> weren't dangerous enough already."
>
> -Edward Abbey
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
--
..---------------------------------------------.
| Worlds of Carnage - http://www.wocmud.org |
:---------------------------------------------:
| Come visit a world of myth and legend where |
| fantastical creatures come to life and the |
| stuff of nightmares grasp for your soul. |
`---------------------------------------------'

On Thu, 28 Aug 2003 15:49:15 -0700, you wrote:

>> Think you're out of luck. Yes, it's a problem I've run up against more than
>> once. There's no thread support in PHP [4. Anyone know if it's in 5?].

>php.net/pcntl_fork

Interesting - that's new.

Unix-only and not in the default install. Not avaialble when PHP is
installed as a mod, either, if I read the manual right. Which explains why
it isn't in common use...

Getting the data back from the child process is gonna be pretty messy (temp
files I guess), but... hmm. Still, may be the best solution in some
circumstances.

For the sake of completeness, I did some searching this time and there
appears to be experimental thread support in PECL.

hi,

David is right, you will not find an equivalent. My advise is to mix
PHP and perl. You can get a perl script to handle the URL retrieval
stuff and pass handlign back to php when you are done. It's this
approach that i took when creating the mega upload progress bar for php.

all the best

David Otton wrote:

>On Thu, 28 Aug 2003 20:25:05 +0300, you wrote:
>
>
>
>>I am looking for PHP analog of Perl LWP::Parallel.
>>I need to fetch several URL(pages) from PHP at the same time.
>>I have a script which fetch 5-10 URL, each URL fetched 0.5 - 2 sec.
>>Totally it's about 10 seconds in average.
>>I suppose if I fetched them parallel I would speed up the script 5 times.
>>I am looking mostly for PHP solutions.
>>(Hope they exist :)
>>
>>
>
>Think you're out of luck. Yes, it's a problem I've run up against more than
>once. There's no thread support in PHP [4. Anyone know if it's in 5?].
>
>I suppose you might be able to hack something together by spawning external
>processes, but... might as well just do it all externally then.
>
>
>


--
http://www.raditha.com/php/progress.php
A progress bar for PHP file uploads.

Just fork the web application. That would probably be a much better
solution...



On Thursday 28 August 2003 04:13 pm, Robert Cummings wrote:
> This isn't the best solution but it might help bring down the total
> time. Can you set up a shell script to retrieve the content from a URL
> (in PHP if you wish) and then from your web app spawn 5 processes, with
> destination temporary files for the data which you can then poll for
> completion (microsleep here and there :)?
>
> Cheers,
> Rob.
>
> On Thu, 2003-08-28 at 18:49, Evan Nemerson wrote:
> > php.net/pcntl_fork
> >
> > On Thursday 28 August 2003 11:09 am, David Otton wrote:
> > > On Thu, 28 Aug 2003 20:25:05 +0300, you wrote:
> > > >I am looking for PHP analog of Perl LWP::Parallel.
> > > >I need to fetch several URL(pages) from PHP at the same time.
> > > >I have a script which fetch 5-10 URL, each URL fetched 0.5 - 2 sec.
> > > >Totally it's about 10 seconds in average.
> > > >I suppose if I fetched them parallel I would speed up the script 5
> > > > times. I am looking mostly for PHP solutions.
> > > >(Hope they exist :)
> > >
> > > Think you're out of luck. Yes, it's a problem I've run up against more
> > > than once. There's no thread support in PHP [4. Anyone know if it's in
> > > 5?].
> > >
> > > I suppose you might be able to hack something together by spawning
> > > external processes, but... might as well just do it all externally
> > > then.
> >
> > --
> > "The missionaries go forth to Christianize the savages- as if the savages
> > weren't dangerous enough already."
> >
> > -Edward Abbey
> >
> > --
> > PHP General Mailing List (http://www.php.net/)
> > To unsubscribe, visit: http://www.php.net/unsub.php