Why I need to set the memory limit for large file upload?

Suppose the file is stored in "upload_tmp_dir ", so why I need to
increase the memory limit?

If I want to upload 100 MB, how large should I set?

Thanks.

howa wrote:
> Suppose the file is stored in "upload_tmp_dir ", so why I need to
> increase the memory limit?
>
> If I want to upload 100 MB, how large should I set?
>
> Thanks.
Because all files are held in memory before being output to disk.

howa wrote:
> Suppose the file is stored in "upload_tmp_dir ", so why I need to
> increase the memory limit?
>
> If I want to upload 100 MB, how large should I set?
>
> Thanks.
>

You shouldn't have to increase the memory limit, but you'll probably
have to increase file upload size (in your webserver or html).

PHP doesn't get involved in the actual upload. Just after the upload
has completed. And moving the file doesn't require reading it into memory.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

On 11$B7n(B30$BF|(B, $B>e8a(B9$B;~(B19$BJ,(B, Jerry Stuckle <jstuck...@attglobal.net> wrote:
> howa wrote:
> > Suppose the file is stored in "upload_tmp_dir ", so why I need to
> > increase the memory limit?
>
> > If I want to upload 100 MB, how large should I set?
>
> > Thanks.
>
> You shouldn't have to increase the memory limit, but you'll probably
> have to increase file upload size (in your webserver or html).
>

In the doc: http://us3.php.net/manual/en/feature...n-pitfalls.php

it said: if a memory limit is enabled, a larger memory_limit may be
needed. Make sure you set memory_limit large enough.

this is quite confusing, as it doesn't said how much you need to set
the memory limit

Thanks anyway.
howa

howa pisze:
> On 11$B7n(B30$BF|(B, $B>e8a(B9$B;~(B19$BJ,(B, Jerry Stuckle <jstuck...@attglobal.net> wrote:
>> howa wrote:
>>> Suppose the file is stored in "upload_tmp_dir ", so why I need to
>>> increase the memory limit?
>>> If I want to upload 100 MB, how large should I set?
>>> Thanks.
>> You shouldn't have to increase the memory limit, but you'll probably
>> have to increase file upload size (in your webserver or html).
>>
>
> In the doc: http://us3.php.net/manual/en/feature...n-pitfalls.php
>
> it said: if a memory limit is enabled, a larger memory_limit may be
> needed. Make sure you set memory_limit large enough.
>
> this is quite confusing, as it doesn't said how much you need to set
> the memory limit


depends on your config

for example
if you have "always poputale raw post data" enabled you need enough to
fit "raw post data" to one of the variables.



--
Piotr Siudak

howa wrote:
> On 11$B7n(B30$BF|(B, $B>e8a(B9$B;~(B19$BJ,(B, Jerry Stuckle <jstuck...@attglobal.net> wrote:
>> howa wrote:
>>> Suppose the file is stored in "upload_tmp_dir ", so why I need to
>>> increase the memory limit?
>>> If I want to upload 100 MB, how large should I set?
>>> Thanks.
>> You shouldn't have to increase the memory limit, but you'll probably
>> have to increase file upload size (in your webserver or html).
>>
>
> In the doc: http://us3.php.net/manual/en/feature...n-pitfalls.php
>
> it said: if a memory limit is enabled, a larger memory_limit may be
> needed. Make sure you set memory_limit large enough.
>
> this is quite confusing, as it doesn't said how much you need to set
> the memory limit
>
> Thanks anyway.
> howa
>

Quite frankly, I don't know. The file itself isn't posted; it comes in
a separate request. And AFAIK, the server handles the upload into the
temporary directory. Even if it doesn't, I wouldn't expect PHP to read
the entire file in before saving it.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

On Fri, 30 Nov 2007 19:44:20 +0100, Jerry Stuckle
<jstucklex@attglobal.net> wrote:

> howa wrote:
>> On 11月30日, 上午9時19分, Jerry Stuckle <jstuck...@attglobal.net> wrote:
>>> howa wrote:
>>>> Suppose the file is stored in "upload_tmp_dir ", so why I need to
>>>> increase the memory limit?
>>>> If I want to upload 100 MB, how large should I set?
>>>> Thanks.
>>> You shouldn't have to increase the memory limit, but you'll probably
>>> have to increase file upload size (in your webserver or html).
>>>
>>
>> In the doc:
>> http://us3.php.net/manual/en/feature...n-pitfalls.php
>>
>> it said: if a memory limit is enabled, a larger memory_limit may be
>> needed. Make sure you set memory_limit large enough.
>>
>> this is quite confusing, as it doesn't said how much you need to set
>> the memory limit
>>
>> Thanks anyway.
>> howa
>>
>
> Quite frankly, I don't know. The file itself isn't posted; it comes in
> a separate request. And AFAIK, the server handles the upload into the
> temporary directory. Even if it doesn't, I wouldn't expect PHP to read
> the entire file in before saving it.

Indeed. The mentioned 'raw post data' is also not populated with HTTP
file-uploads:
<http://nl2.php.net/manual/en/ini.core.php#ini.always-populate-raw-post-data>
"Always populate the $HTTP_RAW_POST_DATA containing the raw POST data.
Otherwise, the variable is populated only with unrecognized MIME type of
the data. However, the preferred method for accessing the raw POST data is
php://input. $HTTP_RAW_POST_DATA is not available with
enctype="multipart/form-data".
Nor is php://input or STDIN:
<http://nl2.php.net/manual/en/wrappers.php.php>
--
Rik Wasmus

Jerry Stuckle wrote:
> howa wrote:
>> On 11$B7n(B30$BF|(B, $B>e8a(B9$B;~(B19$BJ,(B, Jerry Stuckle <jstuck...@attglobal.net> wrote:
>>> howa wrote:
>>>> Suppose the file is stored in "upload_tmp_dir ", so why I need to
>>>> increase the memory limit?
>>>> If I want to upload 100 MB, how large should I set?
>>>> Thanks.
>>> You shouldn't have to increase the memory limit, but you'll probably
>>> have to increase file upload size (in your webserver or html).
>>>
>> In the doc: http://us3.php.net/manual/en/feature...n-pitfalls.php
>>
>> it said: if a memory limit is enabled, a larger memory_limit may be
>> needed. Make sure you set memory_limit large enough.
>>
>> this is quite confusing, as it doesn't said how much you need to set
>> the memory limit
>>
>> Thanks anyway.
>> howa
>>
>
> Quite frankly, I don't know. The file itself isn't posted; it comes in
> a separate request.

I think that is exactly incorrect. I think the file and its headers ARE
posted.

PHP docs for move_uploaded_file says:-
"a valid upload file (meaning that it was uploaded via PHP's HTTP POST
upload mechanism)".

> And AFAIK, the server handles the upload into the
> temporary directory.

No, it does not.

No file exists in the temporary directory till PHP exits. In fact no
file EVER exists in the temporary directory if you move it first.

From my experience what basically happens is that the form sends the
file size, mime type, input name, file name AND CONTENTS as part of the
'post' string..why not? no need to reinvent the wheel.

That gets passed to the PHP interpreter as stdin..(I guess) by apache etc.

The php chucks it all into memory and set up pointers to it, which is
what you extract with the $__FILES[] stuff. And makes up a usable
temporary name for it..but it DOES NOT WRITE IT TO DISK.

Even move_uploaded_file() does not (immediately) write it to disk.

In order for it to be flushed to disk, *you have to exit the PHP
session*, OR as I found by dint of trying everything else, do a straight
copy() on the temporary file and unlink the original. There may be other
methods, but I didn't find them.

You may remember that I stumbled on all this attempting to use the mysql
LOAD_FILE() command: it simply would not work till I copy()'ed the file
to a new location. Using the temporary filename or using
move_uploaded_file() did NOT work.

Although the files duly were there after php exited..;-)


> Even if it doesn't, I wouldn't expect PHP to read
> the entire file in before saving it.
>

Expectations are seldom fulfilled: there is no file to be read. There is
data coming in from a POST operation down stdin..it needs to be stored
in memory - virtual or otherwise, before it can be placed on disk.

In reality, given that it never does exist as a file in the first place,
php's decision NOT to stuff it onto a disk makes sense, speed wise. Why
PHP does not have dynamic unlimited access to memory is more an issue I
would say.

The fact remains you need PHP's memory availability to be somewhat
larger than the sum of all the files you may choose to upload in a
single post operation.