why cURL is so slow?

Hi all,

Here is a part of my code. The goal is to connect to a website, keep the
session by using cookies and navigate through the different pages of the
website then.
I noticed that it needs about 15 to 20sec to retrieve one page. I do believe
this isn't normal, it should be much more faster. I have a 2 Mbps
connection.

1) First you can see the two methods of my class that i want to debug:

public function connect($url, $login='', $pwd='')
{
global $log;

$log->Write("Connection");
$this->ch = curl_init();
curl_setopt($this->ch, CURLOPT_URL, $url);
curl_setopt($this->ch, CURLOPT_USERAGENT, $this->user_agent);

if(!empty($login) && !empty($pwd))
{
curl_setopt ($this->ch, CURLOPT_POST, 1);
curl_setopt ($this->ch, CURLOPT_POSTFIELDS, $this->loginformName.'='
..$login. '&' .$this->loginformPwd. '=' .$pwd);
}

if($this->ssl === true)
{
curl_setopt($this->ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, FALSE);
}

curl_setopt($this->ch, CURLOPT_COOKIEJAR, $this->cookieFile);
curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, 1);
$ret = curl_exec($this->ch);

if($ret != FALSE)
$log->Write("Authentication successfull");
else
$log->Write("authentication failed");
}


protected function GoToPage($page)
{
global $log;

$log->Write("Retrieving page: ".$page);
curl_setopt ($this->ch, CURLOPT_POST, 0);
curl_setopt($this->ch, CURLOPT_COOKIEFILE, $this->cookieFile);

if($this->ssl == true)
{
curl_setopt($this->ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, FALSE);
}

curl_setopt($this->ch, CURLOPT_URL,$page);

$content = curl_exec($this->ch);
if($content != false)
$log->Write("Page retrieved: ".$page);
else
$log->Write("Error retrieving page: ".$page);
return $content;
}

2) and here is how i instanciate my object and call the functions

$curlTest = new $curlTest();
$curlTest->connect('https://xxx.com/login.do', $login, $pwd);
$curlTest ->ListAdvertisers("https://xxx.com/page.php");

I hope you can help me, i'm running out of time, i've searched a lot but
couldn't find any clue.
thanks,

Paske

Nioulou wrote:
> Hi all,
>
> Here is a part of my code. The goal is to connect to a website, keep the
> session by using cookies and navigate through the different pages of the
> website then.
> I noticed that it needs about 15 to 20sec to retrieve one page. I do believe
> this isn't normal, it should be much more faster. I have a 2 Mbps
> connection.
>
> 1) First you can see the two methods of my class that i want to debug:
>
> public function connect($url, $login='', $pwd='')
> {
> global $log;
>
> $log->Write("Connection");
> $this->ch = curl_init();
> curl_setopt($this->ch, CURLOPT_URL, $url);
> curl_setopt($this->ch, CURLOPT_USERAGENT, $this->user_agent);
>
> if(!empty($login) && !empty($pwd))
> {
> curl_setopt ($this->ch, CURLOPT_POST, 1);
> curl_setopt ($this->ch, CURLOPT_POSTFIELDS, $this->loginformName.'='
> .$login. '&' .$this->loginformPwd. '=' .$pwd);
> }
>
> if($this->ssl === true)
> {
> curl_setopt($this->ch, CURLOPT_SSL_VERIFYHOST, 0);
> curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, FALSE);
> }
>
> curl_setopt($this->ch, CURLOPT_COOKIEJAR, $this->cookieFile);
> curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, 1);
> $ret = curl_exec($this->ch);
>
> if($ret != FALSE)
> $log->Write("Authentication successfull");
> else
> $log->Write("authentication failed");
> }
>
>
> protected function GoToPage($page)
> {
> global $log;
>
> $log->Write("Retrieving page: ".$page);
> curl_setopt ($this->ch, CURLOPT_POST, 0);
> curl_setopt($this->ch, CURLOPT_COOKIEFILE, $this->cookieFile);
>
> if($this->ssl == true)
> {
> curl_setopt($this->ch, CURLOPT_SSL_VERIFYHOST, 0);
> curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, FALSE);
> }
>
> curl_setopt($this->ch, CURLOPT_URL,$page);
>
> $content = curl_exec($this->ch);
> if($content != false)
> $log->Write("Page retrieved: ".$page);
> else
> $log->Write("Error retrieving page: ".$page);
> return $content;
> }
>
> 2) and here is how i instanciate my object and call the functions
>
> $curlTest = new $curlTest();
> $curlTest->connect('https://xxx.com/login.do', $login, $pwd);
> $curlTest ->ListAdvertisers("https://xxx.com/page.php");
>
> I hope you can help me, i'm running out of time, i've searched a lot but
> couldn't find any clue.
> thanks,
>
> Paske
>
>

I haven't found cURL to be overly slow, but there are a lot of
possibilities. How long does it take for your browser to load the same
pages? (minus images, of course).


You'll probably need to run a tcp/ip trace (and know how to read it) to
find out what the problem might be. Of course, since you're using
https:, you won't be able to see the data, but the headers will tell you
what's going on.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

Nioulou wrote:

> I noticed that it needs about 15 to 20sec to retrieve one page. I do
> believe this isn't normal, it should be much more faster. I have a 2 Mbps
> connection.

Bandwith is not an issue here. Round-trip time *and* SSL negotiation are.

Try to use keep-alive HTTP connections whenever possible, it should
alleviate the problem if you're requesting several pages in a row.

--
----------------------------------
Iván Sánchez Ortega -ivansanchez-algarroba-escomposlinux-punto-org-

I fell asleep reading a dull book, and I dreamt that I was reading on,
so I woke up from sheer boredom.

On May 27, 1:26 am, Iván Sánchez Ortega <ivansanchez-...@rroba-
escomposlinux.-.punto.-.org> wrote:
> Nioulou wrote:
> > I noticed that it needs about 15 to 20sec to retrieve one page. I do
> > believe this isn't normal, it should be much more faster. I have a 2 Mbps
> > connection.
>
> Bandwith is not an issue here. Round-trip time *and* SSL negotiation are.
>
> Try to use keep-alive HTTP connections whenever possible, it should
> alleviate the problem if you're requesting several pages in a row.
>

1) lack of Keep-alive does not explain the performance issues being
described
2) Keep-alives with SSL is a whole different can of worms, indeed it
might be wrth checking that they are actiually disabled in this case.

C.