limiting download size with curl

Hi!

I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end. Since the serfver doesn't support HTTP range to
use it in curl like curl_setopt($ch, CURLOPT_RANGE, "0-5000"); I
started looking around and found that using the custom write function
I might be able to do the same.

As I understand it, closing the handle and thus terminating the
transfer from within the callback function is not possible, so I'm a
bit stuck..

How could I achieve this? Could someone paste me a fewliner example?

Thanks in advance
Andrew

shikakaa@gmail.com wrote:

> I have a server I'd like to download hundreds of pages from, and only
> get the first few kilobytes of every page to save resources on both my
> and the remote end.

Have you though using a HTTP HEAD request instead of a HTTP GET request?

Cheers,
--
----------------------------------
Iván Sánchez Ortega -ivansanchez-algarroba-escomposlinux-punto-org-

No es culpable el asesino, sino el asesinado.

shikakaa@gmail.com wrote:

> As I understand it, closing the handle and thus terminating the transfer
> from within the callback function is not possible, so I'm a bit stuck..
>
> How could I achieve this?

TCP Sockets. GET requests are not *that* difficult to implement.


--
Toby A Inkster BSc (Hons) ARCS
[Geek of HTML/SQL/Perl/PHP/Python/Apache/Linux]
[OS: Linux 2.6.17.14-mm-desktop-9mdvsmp, up 27 days, 23:43.]

Bottled Water
http://tobyinkster.co.uk/blog/2008/02/18/bottled-water/

On Feb 26, 6:06*pm, Iván Sánchez Ortega <ivansanchez-...@rroba-
escomposlinux.-.punto.-.org> wrote:
> shika...@gmail.com wrote:
> > I have a server I'd like to download hundreds of pages from, and only
> > get the first few kilobytes of every page to save resources on both my
> > and the remote end.
>
> Have you though using a HTTP HEAD request instead of a HTTP GET request?
>
> Cheers,

As I understand it, the HTTP HEAD request would get me a response that
doesn't have the body of the file in question. The target of my file
downloads are about 20% into body of the file, so this approach
wouldn't work.

Thank you for the suggestion though!

On Feb 26, 6:26*pm, Toby A Inkster <usenet200...@tobyinkster.co.uk>
wrote:
> shika...@gmail.com wrote:
> > As I understand it, closing the handle and thus terminating the transfer
> > from within the callback function is not possible, so I'm a bit stuck..
>
> > How could I achieve this?
>
> TCP Sockets. GET requests are not *that* difficult to implement.

Could you please show me an example of this? I'm not sure what I'm
supposed to be googling for at this point.

Hello,

on 02/26/2008 04:07 PM shikakaa@gmail.com said the following:
> On Feb 26, 6:26 pm, Toby A Inkster <usenet200...@tobyinkster.co.uk>
> wrote:
>> shika...@gmail.com wrote:
>>> As I understand it, closing the handle and thus terminating the transfer
>>> from within the callback function is not possible, so I'm a bit stuck..
>>> How could I achieve this?
>> TCP Sockets. GET requests are not *that* difficult to implement.
>
> Could you please show me an example of this? I'm not sure what I'm
> supposed to be googling for at this point.

You can use this HTTP client class that uses TCP sockets with the PHP
fsockopen and only retrieve the amount of data that you want. It figures
what is the response body and headers so you do not have to write
additional code to parse the headers and skip them:

http://www.phpclasses.org/httpclient


--

Regards,
Manuel Lemos

PHP professionals looking for PHP jobs
http://www.phpclasses.org/professionals/

PHP Classes - Free ready to use OOP components written in PHP
http://www.phpclasses.org/

Manuel Lemos wrote:
> Hello,
>
> on 02/26/2008 04:07 PM shikakaa@gmail.com said the following:
>> On Feb 26, 6:26 pm, Toby A Inkster <usenet200...@tobyinkster.co.uk>
>> wrote:
>>> shika...@gmail.com wrote:
>>>> As I understand it, closing the handle and thus terminating the transfer
>>>> from within the callback function is not possible, so I'm a bit stuck..
>>>> How could I achieve this?
>>> TCP Sockets. GET requests are not *that* difficult to implement.
>> Could you please show me an example of this? I'm not sure what I'm
>> supposed to be googling for at this point.
>
> You can use this HTTP client class that uses TCP sockets with the PHP
> fsockopen and only retrieve the amount of data that you want. It figures
> what is the response body and headers so you do not have to write
> additional code to parse the headers and skip them:
>
> http://www.phpclasses.org/httpclient
>
>

Manuel,

Is the only reason you're on this newsgroup is to promote your junky
classes?


--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================