This is a discussion on use ipchains to block all ports > 60,000 within the Linux Security forums, part of the System Security and Security Related category; In article <a3b71eb1.0308271156.63e57ff9@posting.google.com >, rx@rxlist.com says... > Checking `lkm'... You have 27 ...
|
|||||||
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
|
|||
|
In article <a3b71eb1.0308271156.63e57ff9@posting.google.com >,
rx@rxlist.com says... > Checking `lkm'... You have 27 process hidden for ps command > Warning: Possible LKM Trojan installed > > When I run ./chkrootkit -x lkm it shows me the processes and when I > check the output of ps (ps -edf or ps aux) I see all the same > processes id's. I've read that you can get false positives for short > lived processes that may have finished while chkrootkit is running but > these are all stable running processes such as init, sshd, mingetty, > etc. I'm not sure why I'm getting this error since ps and chkproc > both show identical process id's. -Neil > chkrootkit's feedback and your ISP's request (strange as it was...) would indicate that you have almost certainly been rooted. None of your system's utilities (ls, ps, netstat, etc.) can be trusted to give true indication of what is really going on. FWIW, you might start by running "ls -la" and "lsattr" on the contents of /sbin and /bin. Note the ownership of the files in the ls output. Note any odd attributes in the lsattr output. My bet is that you'll notice something out of place, but if you don't, its still no guarantee that you're clean. Since its almost impossible to know what's been cracked and compromised, your only safe recourse is to wipe and reinstall. |
|
|||
|
Tim Haynes wrote: > rx@rxlist.com (Neil Sandow) writes: > > >>It's sounding like blocking the ports is an unwise idea and checking for >>signs of compromise is the way to go. Thanks for the suggestion to run >>chkrootkit as I've never used it before. After running it, I find one >>interesting message: >> >>Checking `lkm'... You have 27 process hidden for ps command Warning: >>Possible LKM Trojan installed > > > Ach, pants. I'd be worried, if I were you. I'm starting to be. > > Next question: does that persist on subsequent runs? Does the number vary, > if so, by how much? Pretty much the same every time, might vary by 1 or 2 > > >>When I run ./chkrootkit -x lkm it shows me the processes and when I check >>the output of ps (ps -edf or ps aux) I see all the same processes id's. >>I've read that you can get false positives for short lived processes that >>may have finished while chkrootkit is running but these are all stable >>running processes such as init, sshd, mingetty, etc. I'm not sure why I'm >>getting this error since ps and chkproc both show identical process id's. > > > How about `ls /proc/[0-9]* | wc -l' and compare that with `ps auxww | wc -l' > several times over? > Hmmm very different. `ls /proc/[0-9]* | wc -l' returns 377 (every time) `ps auxww | wc -l' returns 28 (every time) > Do both a > netstat -plant | grep LISTEN I get: tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 480/sshd > and nmap your box remotely - all ports - and compare what it thinks are > listening. I'm not sure how to do that.... > > Also you need to monitor traffic going out - preferably by inserting a hub > just adjacent to the machine with another box listening in promiscuous mode > to what's going past. Also, snort would be an idea. If you can't place a > sniffing box in the way, run snort on the box itself and configure it to > look for port-scans and dodgy outgoing traffic. > Now I'm way over my head. Drowning...... -ns |
|
|||
|
Neil Sandow wrote: > > > Tim Haynes wrote: > >> rx@rxlist.com (Neil Sandow) writes: >> >> >>> It's sounding like blocking the ports is an unwise idea and checking for >>> signs of compromise is the way to go. Thanks for the suggestion to run >>> chkrootkit as I've never used it before. After running it, I find one >>> interesting message: >>> >>> Checking `lkm'... You have 27 process hidden for ps command Warning: >>> Possible LKM Trojan installed >> >> >> >> Ach, pants. I'd be worried, if I were you. > > > I'm starting to be. > >> >> Next question: does that persist on subsequent runs? Does the number >> vary, >> if so, by how much? > > > Pretty much the same every time, might vary by 1 or 2 > >> >> >>> When I run ./chkrootkit -x lkm it shows me the processes and when I >>> check >>> the output of ps (ps -edf or ps aux) I see all the same processes id's. >>> I've read that you can get false positives for short lived processes >>> that >>> may have finished while chkrootkit is running but these are all stable >>> running processes such as init, sshd, mingetty, etc. I'm not sure why >>> I'm >>> getting this error since ps and chkproc both show identical process >>> id's. >> >> >> >> How about `ls /proc/[0-9]* | wc -l' and compare that with `ps auxww | >> wc -l' >> several times over? >> > > Hmmm very different. > > `ls /proc/[0-9]* | wc -l' returns 377 (every time) > > > `ps auxww | wc -l' returns 28 (every time) ls -d /proc/[0-9]* | wc -l returns 27 almost every time, sometimes 28 ps auxww | wc -l still returns 28 every time > > > >> Do both a netstat -plant | grep LISTEN > > I get: > tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 480/sshd > >> and nmap your box remotely - all ports - and compare what it thinks are >> listening. > > > I'm not sure how to do that.... > I ran nmap from one of the realservers and it only showed one open port (22) >> >> Also you need to monitor traffic going out - preferably by inserting a >> hub >> just adjacent to the machine with another box listening in promiscuous >> mode >> to what's going past. Also, snort would be an idea. If you can't place a >> sniffing box in the way, run snort on the box itself and configure it to >> look for port-scans and dodgy outgoing traffic. > > > > Now I'm way over my head. Drowning...... > > > -ns > -- Neil Sandow, Pharm.D President & CEO - RxList LLC http://www.rxlist.com - The Internet Drug Index rx@rxlist.com |
|
|||
|
On Thu, 28 Aug 2003, Chris McDonald wrote: > In comp.os.linux.security you write: > > >Hmmm very different. > > >`ls /proc/[0-9]* | wc -l' returns 377 (every time) > > >`ps auxww | wc -l' returns 28 (every time) > > > Try: ls -d /proc/[0-9]* | wc -l' > > > Thanks, now I'm getting 27's and 28's should they always match? -ns |
|
|||
|
Neb Revod <spammers.biteme@flatline.com> writes:
> chkrootkit's feedback and your ISP's request (strange as it was...) would > indicate that you have almost certainly been rooted. None of your > system's utilities (ls, ps, netstat, etc.) can be trusted to give true > indication of what is really going on. FWIW, you might start by running > "ls -la" and "lsattr" on the contents of /sbin and /bin. Note the > ownership of the files in the ls output. Note any odd attributes in the > lsattr output. My bet is that you'll notice something out of place, but > if you don't, its still no guarantee that you're clean. Amongst other things, if this is an RPM-based distribution, it should be possible to run rpm -Vva 2>&1 | tee somefile.log and check for suspicious files being changed in bad ways - anything in a bin/ directory, anything in /etc/init.d/ or otherwise related to the startup process, would be major places to start. > Since its almost impossible to know what's been cracked and compromised, > your only safe recourse is to wipe and reinstall. Eventually, maybe, but not based on the say-so of just chkrootkit and some dubious advice from an ISP (50-50 whether it's for marketing or technical reasons). Far more research is required yet. ~Tim -- Ideologies come, ideologies go |piglet@stirfried.vegetable.org.uk A waste of words, and endless flow |http://spodzone.org.uk/ |
|
|||
|
Neil Sandow <rx@rxlist.com> writes:
>> Next question: does that persist on subsequent runs? Does the number >> vary, if so, by how much? > > Pretty much the same every time, might vary by 1 or 2 OK. So there's a consistent difference, quite a gross one. A variation of 1 or 2 is experimental error; in a false-positive report, I'd expect chkrootkit not to be able to discover between 0-2 pids, once every quite-a-few runs, if the box is heavily fork()ing processes left right & centre. A near-as-dammit-constant large-number like 27 going AWOL isn't cool. >>>When I run ./chkrootkit -x lkm it shows me the processes and when I check >>>the output of ps (ps -edf or ps aux) I see all the same processes id's. >>>I've read that you can get false positives for short lived processes that >>>may have finished while chkrootkit is running but these are all stable >>>running processes such as init, sshd, mingetty, etc. I'm not sure why I'm >>>getting this error since ps and chkproc both show identical process id's. >> How about `ls /proc/[0-9]* | wc -l' and compare that with `ps auxww | wc >> -l' >> several times over? >> > > Hmmm very different. > > `ls /proc/[0-9]* | wc -l' returns 377 (every time) > > > `ps auxww | wc -l' returns 28 (every time) OK, I cocked-up on the first of those. Woops :( I should've requested: ls -1d /proc/[0-9]* | wc -l instead - a simple count of how many second-level directories comprised solely of pid#s there are, contrasted with how many processes are running. Only 28 processes on the box? What is its role? Give us a `whereis ps` output as well. >> Do both a netstat -plant | grep LISTEN > I get: > tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 480/sshd Just sshd? That's possibly a good thing. Now what version of ssh is that? (Telnet into port 22 and give us the banner string.) >> and nmap your box remotely - all ports - and compare what it thinks are >> listening. > > I'm not sure how to do that.... On a remote box with nmap, nmap -sS -P0 -p1-65535 myboxip -O -o somelogfile.nmap (syn/fin scan, ignoring failure to respond to pings, all ports, your box's externally-visible IP#, with OS-detection, outputs to file as well as stdout). >> Also you need to monitor traffic going out - preferably by inserting a >> hub just adjacent to the machine with another box listening in >> promiscuous mode to what's going past. Also, snort would be an idea. If >> you can't place a sniffing box in the way, run snort on the box itself >> and configure it to look for port-scans and dodgy outgoing traffic. > > Now I'm way over my head. Drowning...... Your box -----------ethernet--------> outside world right? Now consider: Your box ----HUB------ethernet--------> outside world | | `-------ethernet---> sniffing host By temporarily breaking the network connection and inserting a hub there, you can plug another box into that hub that'll see all the traffic coming & going with tcpdump, ethereal or snort, as long as its network device is in promiscuous mode. One idea you might want to consider: get pen and paper, and list all the functions and services this box provides, all the critical data stored on it, all the tiresome configuration-tweaks present on it. List the cron-jobs it runs. List the users on it and how it gets them. Now, in the background, you can start work on building a replacement box with clean, uptodate, trusted CDs offline, apply distribution's errata, and be vaguely confident of being able to switch the two in the eventuality that you discover (a) a crack has been perpetrated or (b) you never get certain knowlege[0] but want a clean slate. [0] Finding a running pid that shouldn't be, an LKM, a network socket open & listening that netstat doesn't show, evidence of massive scans emanating from that machine, evidence of users you know not of appearing on the box, - these things are concrete. Without one or more of these, or something very similar, you're more or less clean. People, please provide more clues to look for, by all means. ~Tim -- Then came the churches |piglet@stirfried.vegetable.org.uk Then came the schools, |http://spodzone.org.uk/ Then came the lawyers | Then came the rules | |
|
|||
|
Neil Sandow wrote:
> > > erik wrote: >> Neil Sandow wrote: >> >> >>>David <thunderbolt01@netscape.net> wrote in message >>>news:<s2S2b.105978$2x.30473@rwcrnsc52.ops.asp.a tt.net>... >>> >>>>Neil Sandow wrote: >>>> >>>>>Thanks, David! I'll wait a little while to see if anybody pops up >>>>>and says, wait, do that and this will happen, otherwise I'll put >>>>>this in >>>>>place tonight. -Neil >>>> >>>>If the system has been cracked then you need to fix it instead of >>>>blocking ports. Though you probably already know that. >>>>You might want to run chkrootkit on it to see if it finds >>>>anything if you haven't already. >>> >>>It's sounding like blocking the ports is an unwise idea and checking >>>for signs of compromise is the way to go. Thanks for the suggestion >>>to run chkrootkit as I've never used it before. After running it, I >>>find one interesting message: >>> >>>Checking `lkm'... You have 27 process hidden for ps command >>>Warning: Possible LKM Trojan installed >>> >>>When I run ./chkrootkit -x lkm it shows me the processes and when I >>>check the output of ps (ps -edf or ps aux) I see all the same >>>processes id's. I've read that you can get false positives for short >>>lived processes that may have finished while chkrootkit is running >>>but these are all stable running processes such as init, sshd, >>>mingetty, >>>etc. I'm not sure why I'm getting this error since ps and chkproc >>>both show identical process id's. -Neil >> >> >> Given the fact that you run an old distribution chances are that you >> did not update all running services on your machine. I would guess >> that your machine does have problems. Disconnect now. Investigate >> further. Assume that hosts behind your machine are also compromised. >> That would account for the masqueraded hosts. >> >> EJ > > > If I disconnect now my entire site goes down and I'd only do that > under extreme circumstances knowing that I had something terrible > going on. Ok. Please run the nmap as suggested by Tim. Mail me if you want me to do it. I suppose Tim could also do that for you. Of course I would need your ip address. Don't publish that here and now. > It's a high traffic public website. This linux box was built from > scratch a couple of years ago using Mandrake 2.2.17 and the kernel was > rebuilt to include LVS which is its sole purpose. (Maybe it's time > to build a new, clean LVS with the latest code.) > > I've checked the freebsd servers behind the linux box with chkrootkit > and everything seems fine with them. First you need to establish the integrity of the firewall. > > I'm still not getting why chkproc is complaining about processes that > are clearly visible under ps. -Neil So, nmap could tell you more. EJ -- Remove the obvious part (including the dot) for my email address |
|
|||
|
Tim Haynes wrote:
> Neb Revod <spammers.biteme@flatline.com> writes: > >> chkrootkit's feedback and your ISP's request (strange as it was...) >> would indicate that you have almost certainly been rooted. None of >> your system's utilities (ls, ps, netstat, etc.) can be trusted to >> give true indication of what is really going on. FWIW, you might >> start by running "ls -la" and "lsattr" on the contents of /sbin and >> /bin. Note the ownership of the files in the ls output. Note any odd >> attributes in the lsattr output. My bet is that you'll notice >> something out of place, but if you don't, its still no guarantee that >> you're clean. > > Amongst other things, if this is an RPM-based distribution, it should > be possible to run > rpm -Vva 2>&1 | tee somefile.log > and check for suspicious files being changed in bad ways - anything in > a bin/ directory, anything in /etc/init.d/ or otherwise related to the > startup process, would be major places to start. > But also be aware that this when showing nothing is by no means conclusive. There are trojans out there intercepting rpm checksums. >> Since its almost impossible to know what's been cracked and >> compromised, your only safe recourse is to wipe and reinstall. > > Eventually, maybe, but not based on the say-so of just chkrootkit and > some dubious advice from an ISP (50-50 whether it's for marketing or > technical reasons). Far more research is required yet. Correct. EJ -- Remove the obvious part (including the dot) for my email address |
|
|||
|
Tim Haynes wrote: > Neil Sandow <rx@rxlist.com> writes: > > >>>Next question: does that persist on subsequent runs? Does the number >>>vary, if so, by how much? >> >>Pretty much the same every time, might vary by 1 or 2 > > > OK. So there's a consistent difference, quite a gross one. A variation of 1 > or 2 is experimental error; in a false-positive report, I'd expect > chkrootkit not to be able to discover between 0-2 pids, once every > quite-a-few runs, if the box is heavily fork()ing processes left right & > centre. A near-as-dammit-constant large-number like 27 going AWOL isn't > cool. Yes, but running ./chkrootkit -x lkm shows all the process id's and they are no different than what ps shows. It's not like there are hidden processes going on that shouldn't be running (as far as I can tell) > > >>>>When I run ./chkrootkit -x lkm it shows me the processes and when I check >>>>the output of ps (ps -edf or ps aux) I see all the same processes id's. >>>>I've read that you can get false positives for short lived processes that >>>>may have finished while chkrootkit is running but these are all stable >>>>running processes such as init, sshd, mingetty, etc. I'm not sure why I'm >>>>getting this error since ps and chkproc both show identical process id's. >>> >>>How about `ls /proc/[0-9]* | wc -l' and compare that with `ps auxww | wc >>>-l' >>>several times over? >>> >> >>Hmmm very different. >> >>`ls /proc/[0-9]* | wc -l' returns 377 (every time) >> >> >>`ps auxww | wc -l' returns 28 (every time) > > > OK, I cocked-up on the first of those. Woops :( I should've requested: > > ls -1d /proc/[0-9]* | wc -l > > instead - a simple count of how many second-level directories comprised > solely of pid#s there are, contrasted with how many processes are running. > > Only 28 processes on the box? What is its role? It's an LVS (director) for a web site (load balancer) As the LVS is compiled into the kernel it doesn't even show as a process. Nothing else going on here except sshd which allows me to log in and monitor the load. That's all it does. > > Give us a `whereis ps` output as well. > [root@www chkrootkit]# whereis ps ps: /bin/ps /usr/share/man/man1/ps.1.bz2 > >>>Do both a netstat -plant | grep LISTEN >> >>I get: >>tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 480/sshd > > > Just sshd? That's possibly a good thing. Now what version of ssh is that? > (Telnet into port 22 and give us the banner string.) > Telnet not running but here's the ouput of ssh -V and sshd -V OpenSSH_3.0.1p1, SSH protocols 1.5/2.0, OpenSSL 0x0090581f sshd version OpenSSH_3.0.1p1 > >>>and nmap your box remotely - all ports - and compare what it thinks are >>>listening. >> >>I'm not sure how to do that.... > > > On a remote box with nmap, > > nmap -sS -P0 -p1-65535 myboxip -O -o somelogfile.nmap > > (syn/fin scan, ignoring failure to respond to pings, all ports, your box's > externally-visible IP#, with OS-detection, outputs to file as well as > stdout). > OK, ran that from an external box and it showed open ports 22, 80, plus the masqueraded ports which have been set up to allow ssh into the realservers behind the LVS. Should I be concerned that netstat -plant | grep listen only returned port 22? > >>>Also you need to monitor traffic going out - preferably by inserting a >>>hub just adjacent to the machine with another box listening in >>>promiscuous mode to what's going past. Also, snort would be an idea. If >>>you can't place a sniffing box in the way, run snort on the box itself >>>and configure it to look for port-scans and dodgy outgoing traffic. >> >>Now I'm way over my head. Drowning...... > > > > Your box -----------ethernet--------> outside world > > right? > > Now consider: > > Your box ----HUB------ethernet--------> outside world > | > | > `-------ethernet---> sniffing host > > By temporarily breaking the network connection and inserting a hub there, > you can plug another box into that hub that'll see all the traffic coming & > going with tcpdump, ethereal or snort, as long as its network device is in > promiscuous mode. > Would I have to get off my couch? :> > > One idea you might want to consider: get pen and paper, and list all the > functions and services this box provides, all the critical data stored on > it, all the tiresome configuration-tweaks present on it. List the cron-jobs > it runs. List the users on it and how it gets them. > Now, in the background, you can start work on building a replacement box > with clean, uptodate, trusted CDs offline, apply distribution's errata, and > be vaguely confident of being able to switch the two in the eventuality > that you discover (a) a crack has been perpetrated or (b) you never get > certain knowlege[0] but want a clean slate. > That certainly sounds reasonable! > [0] Finding a running pid that shouldn't be, an LKM, a network socket open > & listening that netstat doesn't show, evidence of massive scans emanating > from that machine, evidence of users you know not of appearing on the box, > - these things are concrete. Without one or more of these, or something > very similar, you're more or less clean. People, please provide more clues > to look for, by all means. > > ~Tim My ISP looked for evidence of massive scans emanating from my ip address and found none. No unknown users, no unknown pid's. I'm reluctant to take heroic measures without, as you say, concrete evidence of a real problem. -Neil -- Neil Sandow, Pharm.D President & CEO - RxList LLC http://www.rxlist.com - The Internet Drug Index rx@rxlist.com |
|
|||
|
Tim Haynes wrote: > Neil Sandow <rx@rxlist.com> writes: > > >>> Next question: does that persist on subsequent runs? Does the number >>> vary, if so, by how much? >> >> >> Pretty much the same every time, might vary by 1 or 2 > > > > OK. So there's a consistent difference, quite a gross one. A variation of 1 > or 2 is experimental error; in a false-positive report, I'd expect > chkrootkit not to be able to discover between 0-2 pids, once every > quite-a-few runs, if the box is heavily fork()ing processes left right & > centre. A near-as-dammit-constant large-number like 27 going AWOL isn't > cool. Yes, but running ./chkrootkit -x lkm shows all the process id's and they are no different than what ps shows. It's not like there are hidden processes going on that shouldn't be running (as far as I can tell) > > >>>> When I run ./chkrootkit -x lkm it shows me the processes and when I check >>>> the output of ps (ps -edf or ps aux) I see all the same processes id's. >>>> I've read that you can get false positives for short lived processes that >>>> may have finished while chkrootkit is running but these are all stable >>>> running processes such as init, sshd, mingetty, etc. I'm not sure why I'm >>>> getting this error since ps and chkproc both show identical process id's. >>> >>> >>> How about `ls /proc/[0-9]* | wc -l' and compare that with `ps auxww | wc >>> -l' >>> several times over? >>> >> >> Hmmm very different. >> >> `ls /proc/[0-9]* | wc -l' returns 377 (every time) >> >> >> `ps auxww | wc -l' returns 28 (every time) > > > > OK, I cocked-up on the first of those. Woops I should've requested: > > ls -1d /proc/[0-9]* | wc -l > > instead - a simple count of how many second-level directories comprised > solely of pid#s there are, contrasted with how many processes are running. > > Only 28 processes on the box? What is its role? It's an LVS (director) for a web site (load balancer) As the LVS is compiled into the kernel it doesn't even show as a process. Nothing else going on here except sshd which allows me to log in and monitor the load. That's all it does. > > Give us a `whereis ps` output as well. > [root@www chkrootkit]# whereis ps ps: /bin/ps /usr/share/man/man1/ps.1.bz2 > >>> Do both a netstat -plant | grep LISTEN >> >> >> I get: >> tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 480/sshd > > > > Just sshd? That's possibly a good thing. Now what version of ssh is that? > (Telnet into port 22 and give us the banner string.) > Telnet not running but here's the ouput of ssh -V and sshd -V OpenSSH_3.0.1p1, SSH protocols 1.5/2.0, OpenSSL 0x0090581f sshd version OpenSSH_3.0.1p1 > >>> and nmap your box remotely - all ports - and compare what it thinks are >>> listening. >> >> >> I'm not sure how to do that.... > > > > On a remote box with nmap, > > nmap -sS -P0 -p1-65535 myboxip -O -o somelogfile.nmap > > (syn/fin scan, ignoring failure to respond to pings, all ports, your box's > externally-visible IP#, with OS-detection, outputs to file as well as > stdout). > OK, ran that from an external box and it showed open ports 22, 80, plus the masqueraded ports which have been set up to allow ssh into the realservers behind the LVS. Should I be concerned that netstat -plant | grep listen only returned port 22? > >>> Also you need to monitor traffic going out - preferably by inserting a >>> hub just adjacent to the machine with another box listening in >>> promiscuous mode to what's going past. Also, snort would be an idea. If >>> you can't place a sniffing box in the way, run snort on the box itself >>> and configure it to look for port-scans and dodgy outgoing traffic. >> >> >> Now I'm way over my head. Drowning...... > > > > > Your box -----------ethernet--------> outside world > > right? > > Now consider: > > Your box ----HUB------ethernet--------> outside world > | > | > `-------ethernet---> sniffing host > > By temporarily breaking the network connection and inserting a hub there, > you can plug another box into that hub that'll see all the traffic coming & > going with tcpdump, ethereal or snort, as long as its network device is in > promiscuous mode. > Would I have to get off my couch? :> > > One idea you might want to consider: get pen and paper, and list all the > functions and services this box provides, all the critical data stored on > it, all the tiresome configuration-tweaks present on it. List the cron-jobs > it runs. List the users on it and how it gets them. > Now, in the background, you can start work on building a replacement box > with clean, uptodate, trusted CDs offline, apply distribution's errata, and > be vaguely confident of being able to switch the two in the eventuality > that you discover (a) a crack has been perpetrated or (b) you never get > certain knowlege[0] but want a clean slate. > That certainly sounds reasonable! > [0] Finding a running pid that shouldn't be, an LKM, a network socket open > & listening that netstat doesn't show, evidence of massive scans emanating > from that machine, evidence of users you know not of appearing on the box, > - these things are concrete. Without one or more of these, or something > very similar, you're more or less clean. People, please provide more clues > to look for, by all means. > > ~Tim My ISP looked for evidence of massive scans emanating from my ip address and found none. No unknown users, no unknown pid's. I'm reluctant to take heroic measures without, as you say, concrete evidence of a real problem. -Neil |