'grep' a file - output to browser not working
This is a discussion on 'grep' a file - output to browser not working within the PHP Language forums, part of the PHP Programming Forums category; I cannot figure why this works fine at the command-line of the linux server, but will not output anything ...
|
|||||||
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
11-01-2007
|
|||
|
|||
'grep' a file - output to browser not working
I cannot figure why this works fine at the command-line of the linux
server, but will not output anything to the browser, and no errors in the error_log (syslog). The goal here is to have a page to enter a search term and grep or zgrep pattern matches in the file (will be a log file). Probably there is a better way to do this using php, but this is what I was able to come up with using my limited php- knowledge. Can anyone help me debug this? <HTML> <HEAD> <TITLE>Search Log</TITLE> </HEAD> <BODY> <form action="<?php echo "$_SERVER[PHP_SELF]"; ?>" method="POST"> <input type="text" name="sstr" value="" size="20" maxlength="30"/> <input type="submit" value="Search!"/> </form> <?php //$test=$_POST['sstr']; $test="foo"; if (! empty($test)) { $cmdstr = "grep -i $test <somefile>"; $fp = popen($cmdstr, 'r'); while ($buffer = fread($fp, 8096)) { echo "<br><hr><pre>"; print($buffer); echo "</pre>"; flush(); } pclose($fp); } else { echo "Please enter a search-term and try again!"; } ?> </BODY> </HTML> 
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
I should point out that I know this has something to do with the file-
size. With a small file it works OK. With a large file it fails to the browser. In this case, a large file is up to 3.5GB of uncompressed ASCII text. There may be up to 100 files to search - I can do that part I think, once I get it to actually search the file! :-) Any help GREATLY appreciated. Probably I should mention that I am not a full-time developer ... I'm quite good with shell scripts, REGEX and procedural languages ... I'm a hacker at best with PHP :-) I'm more of a systems person. On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote: > I cannot figure why this works fine at the command-line of the linux > server, but will not output anything to the browser, and no errors in > the error_log (syslog). The goal here is to have a page to enter a > search term and grep or zgrep pattern matches in the file (will be a > log file). Probably there is a better way to do this using php, but > this is what I was able to come up with using my limited php- > knowledge. Can anyone help me debug this?
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
Phil wrote:
> I should point out that I know this has something to do with the file- > size. With a small file it works OK. With a large file it fails to the > browser. In this case, a large file is up to 3.5GB of uncompressed > ASCII text. There may be up to 100 files to search - I can do that > part I think, once I get it to actually search the file! :-) Any help > GREATLY appreciated. > > Probably I should mention that I am not a full-time developer ... I'm > quite good with shell scripts, REGEX and procedural languages ... I'm > a hacker at best with PHP :-) I'm more of a systems person. > > On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote: >> I cannot figure why this works fine at the command-line of the linux >> server, but will not output anything to the browser, and no errors in >> the error_log (syslog). The goal here is to have a page to enter a >> search term and grep or zgrep pattern matches in the file (will be a >> log file). Probably there is a better way to do this using php, but >> this is what I was able to come up with using my limited php- >> knowledge. Can anyone help me debug this? > > > I should point out that I know this has something to do with the file- > size. With a small file it works OK. With a large file it fails to the > browser. In this case, a large file is up to 3.5GB of uncompressed > ASCII text. There may be up to 100 files to search - I can do that > part I think, once I get it to actually search the file! :-) Any help > GREATLY appreciated. > > Probably I should mention that I am not a full-time developer ... I'm > quite good with shell scripts, REGEX and procedural languages ... I'm > a hacker at best with PHP :-) I'm more of a systems person. > (Top posting fixed) Well, some code would help. But are you possibly running out of memory and/or execution time? Those are the two main things which cause problems with large amounts of data but not small. Anything in your PHP error log? > > -- ================== Remove the "x" from my email address Jerry Stuckle JDS Computer Training Corp. jstucklex@attglobal.net ==================
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
Phil wrote:
P.S. Please don't top post. Thanks. > -- ================== Remove the "x" from my email address Jerry Stuckle JDS Computer Training Corp. jstucklex@attglobal.net ==================
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
On Oct 31, 9:58 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote:
> Phil wrote: > > I should point out that I know this has something to do with the file- > > size. With a small file it works OK. With a large file it fails to the > > browser. In this case, a large file is up to 3.5GB of uncompressed > > ASCII text. There may be up to 100 files to search - I can do that > > part I think, once I get it to actually search the file! :-) Any help > > GREATLY appreciated. > > > Probably I should mention that I am not a full-time developer ... I'm > > quite good with shell scripts, REGEX and procedural languages ... I'm > > a hacker at best with PHP :-) I'm more of a systems person. > > > On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote: > >> I cannot figure why this works fine at the command-line of the linux > >> server, but will not output anything to the browser, and no errors in > >> the error_log (syslog). The goal here is to have a page to enter a > >> search term and grep or zgrep pattern matches in the file (will be a > >> log file). Probably there is a better way to do this using php, but > >> this is what I was able to come up with using my limited php- > >> knowledge. Can anyone help me debug this? > > > I should point out that I know this has something to do with the file- > > size. With a small file it works OK. With a large file it fails to the > > browser. In this case, a large file is up to 3.5GB of uncompressed > > ASCII text. There may be up to 100 files to search - I can do that > > part I think, once I get it to actually search the file! :-) Any help > > GREATLY appreciated. > > > > Probably I should mention that I am not a full-time developer ... I'm > > quite good with shell scripts, REGEX and procedural languages ... I'm > > a hacker at best with PHP :-) I'm more of a systems person. > > > (Top posting fixed) > > Well, some code would help. But are you possibly running out of memory > and/or execution time? Those are the two main things which cause > problems with large amounts of data but not small. > > Anything in your PHP error log? there is squat from logs :( This really seems to be an issue of file size. Even this simple read/echo for a file of only 9200 lines it fails, say nothing of my other files of over 2million lines. I have max_execution set to 3000 sec, so i don't think that is it. I don't see too much in php.ini that controls memory handling. "memory limit" is 32MB and post limit is 8PM (default). This is Apache/2.0.52 and PHP 4.3.9 by the way on a dual dual-core P4 3Ghz with 4GB RAM and CentOS 4.4 (32bit) with a 2.6.9-42.0.3.ELsmp kernel $file="/var/log/httpd/access_log"; $handle = @fopen("$file", "r"); if ($handle) { echo "<pre>"; while (!feof($handle)) { $buffer = fgets($handle, 2048); print_r ($buffer); ob_flush(); } echo "</pre>"; fclose($handle); } Sorry for Top Post. See GP for original code snip.
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
top posting is fine. it's really a matter of individual preference. here are
some posting tips for you: http://www.robertnyman.com/2007/03/2...-is-just-fine/ http://en.wikipedia.org/wiki/Posting_style and the quote in your message is missing completely. "Jerry Stuckle" <jstucklex@attglobal.net> wrote in message news:2tqdnSFVda_L0rTanZ2dnUVZ_tDinZ2d@comcast.com. .. > Phil wrote: > > P.S. Please don't top post. Thanks. >> > > > -- > ================== > Remove the "x" from my email address > Jerry Stuckle > JDS Computer Training Corp. > jstucklex@attglobal.net > ================== >
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
Phil wrote:
> On Oct 31, 9:58 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote: >> Phil wrote: >>> I should point out that I know this has something to do with the file- >>> size. With a small file it works OK. With a large file it fails to the >>> browser. In this case, a large file is up to 3.5GB of uncompressed >>> ASCII text. There may be up to 100 files to search - I can do that >>> part I think, once I get it to actually search the file! :-) Any help >>> GREATLY appreciated. >>> Probably I should mention that I am not a full-time developer ... I'm >>> quite good with shell scripts, REGEX and procedural languages ... I'm >>> a hacker at best with PHP :-) I'm more of a systems person. >>> On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote: >>>> I cannot figure why this works fine at the command-line of the linux >>>> server, but will not output anything to the browser, and no errors in >>>> the error_log (syslog). The goal here is to have a page to enter a >>>> search term and grep or zgrep pattern matches in the file (will be a >>>> log file). Probably there is a better way to do this using php, but >>>> this is what I was able to come up with using my limited php- >>>> knowledge. Can anyone help me debug this? >> > I should point out that I know this has something to do with the file- >> > size. With a small file it works OK. With a large file it fails to the >> > browser. In this case, a large file is up to 3.5GB of uncompressed >> > ASCII text. There may be up to 100 files to search - I can do that >> > part I think, once I get it to actually search the file! :-) Any help >> > GREATLY appreciated. >> > >> > Probably I should mention that I am not a full-time developer ... I'm >> > quite good with shell scripts, REGEX and procedural languages ... I'm >> > a hacker at best with PHP :-) I'm more of a systems person. >> > >> (Top posting fixed) >> >> Well, some code would help. But are you possibly running out of memory >> and/or execution time? Those are the two main things which cause >> problems with large amounts of data but not small. >> >> Anything in your PHP error log? > > there is squat from logs :( > > This really seems to be an issue of file size. > Even this simple read/echo for a file of only 9200 lines it fails, say > nothing of my other > files of over 2million lines. > > I have max_execution set to 3000 sec, so i don't think that is it. I > don't see too much in > php.ini that controls memory handling. "memory limit" is 32MB and post > limit is 8PM (default). > > This is Apache/2.0.52 and PHP 4.3.9 by the way on a dual dual-core P4 > 3Ghz with 4GB RAM and CentOS 4.4 (32bit) with a 2.6.9-42.0.3.ELsmp > kernel > > $file="/var/log/httpd/access_log"; > $handle = @fopen("$file", "r"); > if ($handle) { > echo "<pre>"; > while (!feof($handle)) { > $buffer = fgets($handle, 2048); > print_r ($buffer); > ob_flush(); > } > echo "</pre>"; > fclose($handle); > } > > Sorry for Top Post. See GP for original code snip. > > OK, in your php.ini file, ensure you have error_reporting=E_ALL display_errors=On for testing purposes. If this is a production system, I don't recommend having display_errors=On. Rather, I use log_errors=On and set a log file. P.S. Don't mind the trolls here. Someone is starved for attention. -- ================== Remove the "x" from my email address Jerry Stuckle JDS Computer Training Corp. jstucklex@attglobal.net ==================
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
On 1 Nov, 05:25, "Jackie Silva" <maritanl...@yahoo.com> wrote:
> top posting is fine. it's really a matter of individual preference. here are > some posting tips for you: > > http://www.robertnyman.com/2007/03/2.../Posting_style > > and the quote in your message is missing completely. > > "Jerry Stuckle" <jstuck...@attglobal.net> wrote in message > > news:2tqdnSFVda_L0rTanZ2dnUVZ_tDinZ2d@comcast.com. .. > > > > > Phil wrote: > > > P.S. Please don't top post. Thanks. > > > -- > > ================== > > Remove the "x" from my email address > > Jerry Stuckle > > JDS Computer Training Corp. > > jstuck...@attglobal.net > > ==================- Hide quoted text - > > - Show quoted text - I have a posting tip for you. Don't bother!
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
On Nov 1, 5:30 am, Jerry Stuckle <jstuck...@attglobal.net> wrote:
> Phil wrote: > > On Oct 31, 9:58 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote: > >> Phil wrote: > >>> I should point out that I know this has something to do with the file- > >>> size. With a small file it works OK. With a large file it fails to the > >>> browser. In this case, a large file is up to 3.5GB of uncompressed > >>> ASCII text. There may be up to 100 files to search - I can do that > >>> part I think, once I get it to actually search the file! :-) Any help > >>> GREATLY appreciated. > >>> Probably I should mention that I am not a full-time developer ... I'm > >>> quite good with shell scripts, REGEX and procedural languages ... I'm > >>> a hacker at best with PHP :-) I'm more of a systems person. > >>> On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote: > >>>> I cannot figure why this works fine at the command-line of the linux > >>>> server, but will not output anything to the browser, and no errors in > >>>> the error_log (syslog). The goal here is to have a page to enter a > >>>> search term and grep or zgrep pattern matches in the file (will be a > >>>> log file). Probably there is a better way to do this using php, but > >>>> this is what I was able to come up with using my limited php- > >>>> knowledge. Can anyone help me debug this? > >> > I should point out that I know this has something to do with the file- > >> > size. With a small file it works OK. With a large file it fails to the > >> > browser. In this case, a large file is up to 3.5GB of uncompressed > >> > ASCII text. There may be up to 100 files to search - I can do that > >> > part I think, once I get it to actually search the file! :-) Any help > >> > GREATLY appreciated. > > >> > Probably I should mention that I am not a full-time developer ... I'm > >> > quite good with shell scripts, REGEX and procedural languages ... I'm > >> > a hacker at best with PHP :-) I'm more of a systems person. > > >> (Top posting fixed) > > >> Well, some code would help. But are you possibly running out of memory > >> and/or execution time? Those are the two main things which cause > >> problems with large amounts of data but not small. > > >> Anything in your PHP error log? > > > there is squat from logs :( > > > This really seems to be an issue of file size. > > Even this simple read/echo for a file of only 9200 lines it fails, say > > nothing of my other > > files of over 2million lines. > > > I have max_execution set to 3000 sec, so i don't think that is it. I > > don't see too much in > > php.ini that controls memory handling. "memory limit" is 32MB and post > > limit is 8PM (default). > > > This is Apache/2.0.52 and PHP 4.3.9 by the way on a dual dual-core P4 > > 3Ghz with 4GB RAM and CentOS 4.4 (32bit) with a 2.6.9-42.0.3.ELsmp > > kernel > > > $file="/var/log/httpd/access_log"; > > $handle = @fopen("$file", "r"); > > if ($handle) { > > echo "<pre>"; > > while (!feof($handle)) { > > $buffer = fgets($handle, 2048); > > print_r ($buffer); > > ob_flush(); > > } > > echo "</pre>"; > > fclose($handle); > > } > > > Sorry for Top Post. See GP for original code snip. > > OK, in your php.ini file, ensure you have > > error_reporting=E_ALL > display_errors=On > > for testing purposes. > > If this is a production system, I don't recommend having > display_errors=On. Rather, I use log_errors=On and set a log file. > > P.S. Don't mind the trolls here. Someone is starved for attention. I already have those options set. Actually, I partly found the answer, and it was a beginners mistake ... file permissions, as the logs are root-owned and apache/ php runs as apache - a little 'is_readable' helped me out there. Now I have to decide if i want to have apache run as root, or modify system log files. It is 'production' system, but it's an enterprise internal only single-purpose and with Change Control I can do anything I need for this. I'd still be interested in quid-pro-quo on using external grep versus some internal preg_grep (and how to make that one work ... I've been stumped so far). Thanks! phil
|
|
11-01-2007
|
|||
|
|||
|
Re: 'grep' a file - output to browser not working
On Thu, 01 Nov 2007 06:25:07 +0100, Jackie Silva <maritanlito@yahoo.com>
wrote: > top posting is fine. it's really a matter of individual preference. here > are > some posting tips for you: > > http://www.robertnyman.com/2007/03/2...-is-just-fine/ Yes, topposting is fine, and actually my preferred method, for email (sometimes interleaved, though rarely). This however is usenet, not personal email. -- Rik Wasmus
|
Linear Mode
