LAMP performance issue- where is bottleneck?

The following is a lot of information about a specific performance
tuning related problem I am having with a LAMP web server. I'm not sure
if this group is appropriate for such discussions, please advise if
otherwise.

I'm having a performance problem with a web application that is hitting
a performance ceiling at 34 pages per second with no obvious bottleneck.

I've factored out calls to the database and used the loopback device for
this test.

My basic premis is this:

During testing there appears to be 50% of CPU resouces in idle state,
with most dedicated to user, some in sys, and almost non in iowait
states. This tells me that I should have room to process more
concurrent requests without adversely affecting the response time of the
web application. Unfortunately, this is not the case- as I increase the
number of concurrent users during subsequent testing, the pages per
second remain the same while the average response times grow larger.
This tells me there is indeed a bottleneck, and based on the output from
vmstat and iostat I am hopeful it is not a bottleneck based on hardware
saturation.

One thing interesting to note that I'm not sure if indicates a problem
is that I have ~ 3 times the number of context switches (3000) than I
have interrupts per second (1000.) Since cpu sys utilization is so low,
I am assuming this is not a problem.

I've also included a dump of my sysctl settings in case there is a
problem with tunable parameters.

The environment is mod_php 5.2.1 with apache 2.0.52 running RHEL 4.5
(2.6 kernel) on an Intel Xeon 2.4G Dual Core with 2G ram and a SCSI raid
5 and 1Gb NIC.


The following data was captured during a stress test. It includes
output from apachebench, vmstat, iostat, syscntl and httpd.conf for a
period of one minute.

I'm hoping someone can take a look and offer some suggestions on
potential bottlenecks that I have overlooked.

Thanks.


This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation,
http://www.apache.org/

Benchmarking localhost (be patient)


Server Software: Apache/2.0.52
Server Hostname: localhost
Server Port: 80

Document Path: /ppc/rs.php?view=nmr
Document Length: 3108 bytes

Concurrency Level: 30
Time taken for tests: 59.437493 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 6762000 bytes
HTML transferred: 6216000 bytes
Requests per second: 33.65 [#/sec] (mean)
Time per request: 891.562 [ms] (mean)
Time per request: 29.719 [ms] (mean, across all concurrent requests)
Transfer rate: 111.09 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.8 0 9
Processing: 38 875 1004.3 361 5760
Waiting: 36 875 1004.3 361 5759
Total: 39 875 1004.3 361 5760

Percentage of the requests served within a certain time (ms)
50% 361
66% 847
75% 1726
80% 1937
90% 2387
95% 2743
98% 3373
99% 3997
100% 5760 (longest request)
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us
sy id wa
3 0 208 661128 74548 1169612 0 0 0 4 3 3 0
2 98 0
2 0 208 655920 74548 1169612 0 0 10 0 1486 3482 47
3 51 0
2 0 208 649264 74548 1169612 0 0 0 100 1478 3142 47
23 31 0
2 0 208 617288 74556 1169604 0 0 0 8 1522 3307 47
35 19 0
2 0 208 613848 74556 1169604 0 0 0 50 1546 3898 47
3 51 0
4 0 208 613720 74556 1169604 0 0 0 0 1482 3716 47
2 51 0
2 0 208 613480 74564 1169596 0 0 0 8 1487 3717 47
2 50 0
1 0 208 613416 74564 1169596 0 0 0 60 1485 3753 47
2 51 0
4 0 208 613368 74572 1169588 0 0 0 8 1492 3678 47
3 51 0
2 0 208 613368 74580 1169580 0 0 0 38 1489 3707 47
2 51 0
2 0 208 613256 74588 1169572 0 0 0 88 1487 3691 47
3 50 0
3 0 208 613256 74596 1169564 0 0 0 8 1486 3759 48
2 51 0
2 0 208 613256 74596 1169564 0 0 0 30 1487 3738 47
2 50 0
2 0 208 613128 74604 1169816 0 0 0 8 1483 3723 47
2 51 0
4 0 208 613512 74604 1169816 0 0 0 4 1487 3796 47
2 51 0
4 0 208 613256 74604 1169816 0 0 0 32 1489 3792 47
3 51 0
1 0 208 613128 74612 1169808 0 0 0 8 1490 3729 47
2 50 0
1 0 208 613128 74612 1169808 0 0 0 144 1495 3786 47
3 51 0
4 0 208 613512 74620 1169800 0 0 0 8 1487 3664 47
3 51 0
2 0 208 613064 74620 1169800 0 0 0 0 1483 3732 47
2 51 0
1 0 208 613000 74620 1169800 0 0 0 32 1484 3669 47
2 51 0
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us
sy id wa
0 0 208 613000 74628 1169792 0 0 0 8 1488 3817 47
2 50 0
2 0 208 612936 74632 1169788 0 0 0 32 1488 3729 47
3 50 0
2 0 208 612808 74640 1170040 0 0 0 8 1487 3727 47
2 51 0
2 0 208 612808 74640 1170040 0 0 0 0 1483 3762 47
2 51 0
0 0 208 612808 74640 1170040 0 0 0 36 1485 3770 46
3 51 0
0 0 208 612744 74648 1170032 0 0 0 8 1489 3851 47
2 51 0
2 0 208 612744 74652 1170028 0 0 0 36 1489 3813 47
2 51 0
2 0 208 612744 74660 1170020 0 0 0 8 1486 3789 47
2 51 0
2 0 208 612808 74660 1170020 0 0 0 0 1483 3776 47
3 51 0
Linux 2.6.9-55.ELsmp (124011-stgweb.rmgconnect.com) 07/01/2007

avg-cpu: %user %nice %sys %iowait %idle
0.30 0.00 1.77 0.01 97.93

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.51 0.95 17.71 1661688 30912784
sda1 0.00 0.00 0.00 676 832
sda2 0.00 0.00 0.00 276 16
sda3 0.03 0.00 0.27 7094 468184
sda4 2.21 0.95 17.44 1652386 30442008

avg-cpu: %user %nice %sys %iowait %idle
47.00 0.00 2.50 0.00 50.50

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.50 20.00 0.00 40 0
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 0.50 20.00 0.00 40 0

avg-cpu: %user %nice %sys %iowait %idle
46.50 0.00 23.00 0.00 30.50

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 3.50 0.00 200.00 0 400
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 25.00 0.00 200.00 0 400

avg-cpu: %user %nice %sys %iowait %idle
46.62 0.00 34.84 0.00 18.55

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.49 0.00 15.92 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 1.99 0.00 15.92 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
46.87 0.00 2.51 0.00 50.63

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 4.02 0.00 100.50 0 200
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 12.56 0.00 100.50 0 200

avg-cpu: %user %nice %sys %iowait %idle
46.88 0.00 2.49 0.00 50.62

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
47.12 0.00 2.26 0.00 50.63

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.50 0.00 16.00 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 2.00 0.00 16.00 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
47.13 0.00 2.24 0.00 50.62

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.49 0.00 119.40 0 240
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 14.93 0.00 119.40 0 240

avg-cpu: %user %nice %sys %iowait %idle
46.75 0.00 2.50 0.00 50.75

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.50 0.00 16.00 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 2.00 0.00 16.00 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
47.00 0.00 2.50 0.00 50.50

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.50 0.00 76.00 0 152
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 9.50 0.00 76.00 0 152

avg-cpu: %user %nice %sys %iowait %idle
46.63 0.00 3.49 0.00 49.88

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 5.03 0.00 176.88 0 352
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 22.11 0.00 176.88 0 352

avg-cpu: %user %nice %sys %iowait %idle
47.62 0.00 1.75 0.00 50.63

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.49 0.00 15.92 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 1.99 0.00 15.92 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
46.88 0.00 2.49 0.00 50.62

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.50 0.00 60.00 0 120
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 7.50 0.00 60.00 0 120

avg-cpu: %user %nice %sys %iowait %idle
47.25 0.00 2.25 0.00 50.50

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.50 0.00 16.00 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 2.00 0.00 16.00 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
46.88 0.00 2.49 0.00 50.62

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.50 0.00 8.00 0 16
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 1.00 0.00 8.00 0 16

avg-cpu: %user %nice %sys %iowait %idle
46.87 0.00 2.51 0.00 50.63

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.50 0.00 64.00 0 128
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 8.00 0.00 64.00 0 128

avg-cpu: %user %nice %sys %iowait %idle
47.38 0.00 2.24 0.00 50.37

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.49 0.00 15.92 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 1.99 0.00 15.92 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
46.62 0.00 2.51 0.00 50.88

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 24.62 0.00 289.45 0 576
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.50 0.00 4.02 0 8
sda4 35.68 0.00 285.43 0 568

avg-cpu: %user %nice %sys %iowait %idle
46.38 0.00 2.49 0.00 51.12

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.50 0.00 16.00 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 2.00 0.00 16.00 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
46.87 0.00 2.51 0.00 50.63

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
47.13 0.00 2.24 0.00 50.62

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.51 0.00 64.32 0 128
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 8.04 0.00 64.32 0 128

avg-cpu: %user %nice %sys %iowait %idle
47.25 0.00 2.25 0.00 50.50

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.49 0.00 15.92 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 1.99 0.00 15.92 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
47.13 0.00 2.99 0.00 49.88

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.50 0.00 64.00 0 128
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 8.00 0.00 64.00 0 128

avg-cpu: %user %nice %sys %iowait %idle
47.25 0.00 2.00 0.00 50.75

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.50 0.00 16.00 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 2.00 0.00 16.00 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
47.13 0.00 2.24 0.00 50.62

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
46.37 0.00 2.76 0.00 50.88

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.49 0.00 71.64 0 144
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 8.96 0.00 71.64 0 144

avg-cpu: %user %nice %sys %iowait %idle
47.38 0.00 2.24 0.00 50.37

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.50 0.00 16.00 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 2.00 0.00 16.00 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
47.00 0.00 2.25 0.00 50.75

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.50 0.00 72.00 0 144
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 9.00 0.00 72.00 0 144

avg-cpu: %user %nice %sys %iowait %idle
47.00 0.00 2.25 0.00 50.75

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.51 0.00 16.08 0 32
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 2.01 0.00 16.08 0 32
sda4 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %sys %iowait %idle
46.75 0.00 2.75 0.00 50.50

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0

dev.scsi.logging_level = 0
dev.raid.speed_limit_max = 200000
dev.raid.speed_limit_min = 1000
dev.cdrom.check_media = 0
dev.cdrom.lock = 1
dev.cdrom.debug = 0
dev.cdrom.autoeject = 0
dev.cdrom.autoclose = 1
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17
dev.cdrom.info =
dev.cdrom.info = drive name: hda
dev.cdrom.info = drive speed: 24
dev.cdrom.info = drive # of slots: 1
dev.cdrom.info = Can close tray: 1
dev.cdrom.info = Can open tray: 1
dev.cdrom.info = Can lock tray: 1
dev.cdrom.info = Can change speed: 1
dev.cdrom.info = Can select disk: 0
dev.cdrom.info = Can read multisession: 1
dev.cdrom.info = Can read MCN: 1
dev.cdrom.info = Reports media changed: 1
dev.cdrom.info = Can play audio: 1
dev.cdrom.info = Can write CD-R: 0
dev.cdrom.info = Can write CD-RW: 0
dev.cdrom.info = Can read DVD: 0
dev.cdrom.info = Can write DVD-R: 0
dev.cdrom.info = Can write DVD-RAM: 0
dev.cdrom.info = Can read MRW: 1
dev.cdrom.info = Can write MRW: 1
dev.cdrom.info = Can write RAM: 1
dev.cdrom.info =
dev.cdrom.info =
dev.rtc.max-user-freq = 64
net.unix.max_dgram_qlen = 10
net.token-ring.rif_timeout = 600000
net.ipv4.conf.eth1.force_igmp_version = 0
net.ipv4.conf.eth1.disable_policy = 0
net.ipv4.conf.eth1.disable_xfrm = 0
net.ipv4.conf.eth1.arp_ignore = 0
net.ipv4.conf.eth1.arp_announce = 0
net.ipv4.conf.eth1.arp_filter = 0
net.ipv4.conf.eth1.tag = 0
net.ipv4.conf.eth1.log_martians = 0
net.ipv4.conf.eth1.bootp_relay = 0
net.ipv4.conf.eth1.medium_id = 0
net.ipv4.conf.eth1.proxy_arp = 0
net.ipv4.conf.eth1.accept_source_route = 0
net.ipv4.conf.eth1.send_redirects = 1
net.ipv4.conf.eth1.rp_filter = 1
net.ipv4.conf.eth1.shared_media = 1
net.ipv4.conf.eth1.secure_redirects = 1
net.ipv4.conf.eth1.accept_redirects = 1
net.ipv4.conf.eth1.mc_forwarding = 0
net.ipv4.conf.eth1.forwarding = 0
net.ipv4.conf.eth0.force_igmp_version = 0
net.ipv4.conf.eth0.disable_policy = 0
net.ipv4.conf.eth0.disable_xfrm = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.eth0.arp_announce = 0
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.tag = 0
net.ipv4.conf.eth0.log_martians = 0
net.ipv4.conf.eth0.bootp_relay = 0
net.ipv4.conf.eth0.medium_id = 0
net.ipv4.conf.eth0.proxy_arp = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.eth0.send_redirects = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.eth0.shared_media = 1
net.ipv4.conf.eth0.secure_redirects = 1
net.ipv4.conf.eth0.accept_redirects = 1
net.ipv4.conf.eth0.mc_forwarding = 0
net.ipv4.conf.eth0.forwarding = 0
net.ipv4.conf.lo.force_igmp_version = 0
net.ipv4.conf.lo.disable_policy = 0
net.ipv4.conf.lo.disable_xfrm = 0
net.ipv4.conf.lo.arp_ignore = 0
net.ipv4.conf.lo.arp_announce = 0
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.tag = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.lo.bootp_relay = 0
net.ipv4.conf.lo.medium_id = 0
net.ipv4.conf.lo.proxy_arp = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.lo.send_redirects = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.lo.shared_media = 1
net.ipv4.conf.lo.secure_redirects = 1
net.ipv4.conf.lo.accept_redirects = 1
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.lo.forwarding = 0
net.ipv4.conf.default.force_igmp_version = 0
net.ipv4.conf.default.disable_policy = 0
net.ipv4.conf.default.disable_xfrm = 0
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.default.arp_announce = 0
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.tag = 0
net.ipv4.conf.default.log_martians = 0
net.ipv4.conf.default.bootp_relay = 0
net.ipv4.conf.default.medium_id = 0
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.shared_media = 1
net.ipv4.conf.default.secure_redirects = 1
net.ipv4.conf.default.accept_redirects = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 0
net.ipv4.conf.all.force_igmp_version = 0
net.ipv4.conf.all.disable_policy = 0
net.ipv4.conf.all.disable_xfrm = 0
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_announce = 0
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.tag = 0
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.medium_id = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.send_redirects = 1
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.all.shared_media = 1
net.ipv4.conf.all.secure_redirects = 1
net.ipv4.conf.all.accept_redirects = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.all.forwarding = 0
net.ipv4.neigh.eth1.locktime = 99
net.ipv4.neigh.eth1.proxy_delay = 79
net.ipv4.neigh.eth1.anycast_delay = 99
net.ipv4.neigh.eth1.proxy_qlen = 64
net.ipv4.neigh.eth1.unres_qlen = 3
net.ipv4.neigh.eth1.gc_stale_time = 60
net.ipv4.neigh.eth1.delay_first_probe_time = 5
net.ipv4.neigh.eth1.base_reachable_time = 30
net.ipv4.neigh.eth1.retrans_time = 99
net.ipv4.neigh.eth1.app_solicit = 0
net.ipv4.neigh.eth1.ucast_solicit = 3
net.ipv4.neigh.eth1.mcast_solicit = 3
net.ipv4.neigh.eth0.locktime = 99
net.ipv4.neigh.eth0.proxy_delay = 79
net.ipv4.neigh.eth0.anycast_delay = 99
net.ipv4.neigh.eth0.proxy_qlen = 64
net.ipv4.neigh.eth0.unres_qlen = 3
net.ipv4.neigh.eth0.gc_stale_time = 60
net.ipv4.neigh.eth0.delay_first_probe_time = 5
net.ipv4.neigh.eth0.base_reachable_time = 30
net.ipv4.neigh.eth0.retrans_time = 99
net.ipv4.neigh.eth0.app_solicit = 0
net.ipv4.neigh.eth0.ucast_solicit = 3
net.ipv4.neigh.eth0.mcast_solicit = 3
net.ipv4.neigh.lo.locktime = 99
net.ipv4.neigh.lo.proxy_delay = 79
net.ipv4.neigh.lo.anycast_delay = 99
net.ipv4.neigh.lo.proxy_qlen = 64
net.ipv4.neigh.lo.unres_qlen = 3
net.ipv4.neigh.lo.gc_stale_time = 60
net.ipv4.neigh.lo.delay_first_probe_time = 5
net.ipv4.neigh.lo.base_reachable_time = 30
net.ipv4.neigh.lo.retrans_time = 99
net.ipv4.neigh.lo.app_solicit = 0
net.ipv4.neigh.lo.ucast_solicit = 3
net.ipv4.neigh.lo.mcast_solicit = 3
net.ipv4.neigh.default.gc_thresh3 = 1024
net.ipv4.neigh.default.gc_thresh2 = 512
net.ipv4.neigh.default.gc_thresh1 = 128
net.ipv4.neigh.default.gc_interval = 30
net.ipv4.neigh.default.locktime = 99
net.ipv4.neigh.default.proxy_delay = 79
net.ipv4.neigh.default.anycast_delay = 99
net.ipv4.neigh.default.proxy_qlen = 64
net.ipv4.neigh.default.unres_qlen = 3
net.ipv4.neigh.default.gc_stale_time = 60
net.ipv4.neigh.default.delay_first_probe_time = 5
net.ipv4.neigh.default.base_reachable_time = 30
net.ipv4.neigh.default.retrans_time = 99
net.ipv4.neigh.default.app_solicit = 0
net.ipv4.neigh.default.ucast_solicit = 3
net.ipv4.neigh.default.mcast_solicit = 3
net.ipv4.tcp_bic_beta = 819
net.ipv4.tcp_tso_win_divisor = 8
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_bic_low_window = 14
net.ipv4.tcp_bic_fast_convergence = 1
net.ipv4.tcp_bic = 1
net.ipv4.tcp_vegas_gamma = 2
net.ipv4.tcp_vegas_beta = 6
net.ipv4.tcp_vegas_alpha = 2
net.ipv4.tcp_vegas_cong_avoid = 0
net.ipv4.tcp_westwood = 0
net.ipv4.tcp_no_metrics_save = 0
net.ipv4.ipfrag_secret_interval = 600
net.ipv4.tcp_low_latency = 0
net.ipv4.tcp_frto = 0
net.ipv4.tcp_tw_reuse = 0
net.ipv4.icmp_ratemask = 6168
net.ipv4.icmp_ratelimit = 1000
net.ipv4.tcp_adv_win_scale = 2
net.ipv4.tcp_app_win = 31
net.ipv4.tcp_rmem = 65535 131072 4194304
net.ipv4.tcp_wmem = 65535 131072 194304
net.ipv4.tcp_mem = 8192 9216 10150
net.ipv4.tcp_dsack = 1
net.ipv4.tcp_ecn = 0
net.ipv4.tcp_reordering = 3
net.ipv4.tcp_fack = 1
net.ipv4.tcp_orphan_retries = 0
net.ipv4.inet_peer_gc_maxtime = 120
net.ipv4.inet_peer_gc_mintime = 10
net.ipv4.inet_peer_maxttl = 600
net.ipv4.inet_peer_minttl = 120
net.ipv4.inet_peer_threshold = 65664
net.ipv4.igmp_max_msf = 10
net.ipv4.igmp_max_memberships = 20
net.ipv4.route.secret_interval = 600
net.ipv4.route.min_adv_mss = 256
net.ipv4.route.min_pmtu = 552
net.ipv4.route.mtu_expires = 600
net.ipv4.route.gc_elasticity = 8
net.ipv4.route.error_burst = 5000
net.ipv4.route.error_cost = 1000
net.ipv4.route.redirect_silence = 20480
net.ipv4.route.redirect_number = 9
net.ipv4.route.redirect_load = 20
net.ipv4.route.gc_interval = 60
net.ipv4.route.gc_timeout = 300
net.ipv4.route.gc_min_interval = 0
net.ipv4.route.max_size = 2097152
net.ipv4.route.gc_thresh = 131072
net.ipv4.route.max_delay = 10
net.ipv4.route.min_delay = 2
net.ipv4.icmp_errors_use_inbound_ifaddr = 0
net.ipv4.icmp_ignore_bogus_error_responses = 0
net.ipv4.icmp_echo_ignore_broadcasts = 0
net.ipv4.icmp_echo_ignore_all = 0
net.ipv4.ip_local_port_range = 32768 61000
net.ipv4.tcp_max_syn_backlog = 1024
net.ipv4.tcp_rfc1337 = 0
net.ipv4.tcp_stdurg = 0
net.ipv4.tcp_abort_on_overflow = 0
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_fin_timeout = 5
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_keepalive_intvl = 75
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 7200
net.ipv4.ipfrag_time = 30
net.ipv4.ip_dynaddr = 0
net.ipv4.ipfrag_low_thresh = 196608
net.ipv4.ipfrag_high_thresh = 262144
net.ipv4.tcp_max_tw_buckets = 180000
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_synack_retries = 5
net.ipv4.tcp_syn_retries = 5
net.ipv4.ip_nonlocal_bind = 0
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.ip_autoconfig = 0
net.ipv4.ip_default_ttl = 64
net.ipv4.ip_forward = 0
net.ipv4.tcp_retrans_collapse = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.core.somaxconn = 128
net.core.divert_version = 0.46
net.core.optmem_max = 10240
net.core.message_burst = 10
net.core.message_cost = 5
net.core.mod_cong = 290
net.core.lo_cong = 100
net.core.no_cong = 20
net.core.no_cong_thresh = 10
net.core.netdev_max_backlog = 300
net.core.dev_weight = 64
net.core.rmem_default = 110592
net.core.wmem_default = 110592
net.core.rmem_max = 131071
net.core.wmem_max = 131071
vm.percpu_pagelist_fraction = 0
vm.max_queue_depth = 0
vm.oom-kill = 1
vm.legacy_va_layout = 0
vm.vfs_cache_pressure = 100
vm.block_dump = 0
vm.laptop_mode = 0
vm.max_map_count = 65536
vm.min_free_kbytes = 949
vm.lower_zone_protection = 0
vm.hugetlb_shm_group = 0
vm.nr_hugepages = 0
vm.swappiness = 60
vm.nr_pdflush_threads = 2
vm.dirty_expire_centisecs = 3000
vm.dirty_writeback_centisecs = 500
vm.dirty_ratio = 40
vm.dirty_background_ratio = 10
vm.page-cluster = 3
vm.overcommit_ratio = 50
vm.overcommit_memory = 0
kernel.panic_on_unrecovered_nmi = 0
kernel.wake_balance = 1
kernel.suid_dumpable = 0
kernel.unknown_nmi_panic = 0
kernel.ngroups_max = 65536
kernel.printk_ratelimit_burst = 10
kernel.printk_ratelimit = 5
kernel.panic_on_oops = 1
kernel.pid_max = 32768
kernel.sercons_esc = -1
kernel.overflowgid = 65534
kernel.overflowuid = 65534
kernel.pty.nr = 3
kernel.pty.max = 4096
kernel.random.uuid = 97b16d47-b158-4899-b1d8-0e85a81445bb
kernel.random.boot_id = 7fb1d140-3dfe-42a3-9783-0b80ba08e95b
kernel.random.write_wakeup_threshold = 128
kernel.random.read_wakeup_threshold = 64
kernel.random.entropy_avail = 3456
kernel.random.poolsize = 512
kernel.threads-max = 65525
kernel.cad_pid = 1
kernel.sysrq = 0
kernel.sem = 250 32000 32 128
kernel.msgmnb = 16384
kernel.msgmni = 16
kernel.msgmax = 8192
kernel.shmmni = 4096
kernel.shmall = 2097152
kernel.shmmax = 33554432
kernel.acct = 4 2 30
kernel.hotplug = /sbin/hotplug
kernel.modprobe = /sbin/modprobe
kernel.printk = 6 4 1 7
kernel.ctrl-alt-del = 0
kernel.real-root-dev = 0
kernel.cap-bound = -257
kernel.tainted = 0
kernel.core_pattern = /root/core
kernel.core_uses_pid = 1
kernel.vdso = 0
kernel.print-fatal-signals = 0
kernel.exec-shield-randomize = 1
kernel.exec-shield = 1
kernel.panic = 0
kernel.domainname = (none)
kernel.hostname = 124011-stgweb.rmgconnect.com
kernel.version = #1 SMP Fri Apr 20 17:03:35 EDT 2007
kernel.osrelease = 2.6.9-55.ELsmp
kernel.ostype = Linux
fs.mqueue.msgsize_max = 8192
fs.mqueue.msg_max = 10
fs.mqueue.queues_max = 256
fs.quota.syncs = 18
fs.quota.free_dquots = 0
fs.quota.allocated_dquots = 0
fs.quota.cache_hits = 0
fs.quota.writes = 0
fs.quota.reads = 0
fs.quota.drops = 0
fs.quota.lookups = 0
fs.aio-max-nr = 65536
fs.aio-nr = 0
fs.lease-break-time = 45
fs.dir-notify-enable = 1
fs.leases-enable = 1
fs.overflowgid = 65534
fs.overflowuid = 65534
fs.dentry-state = 69912 60758 45 0 0 0
fs.file-max = 206095
fs.file-nr = 1220 0 206095
fs.inode-state = 62569 10 0 0 0 0 0
fs.inode-nr = 62569 10



ServerTokens OS

ServerRoot "/etc/httpd"

PidFile run/httpd.pid

Timeout 5

KeepAlive Off

MaxKeepAliveRequests 100

KeepAliveTimeout 2


<IfModule prefork.c>
StartServers 10
MinSpareServers 10
MaxSpareServers 10
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
</IfModule>

<IfModule worker.c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>

Listen 80

LoadModule access_module modules/mod_access.so
LoadModule auth_module modules/mod_auth.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule cern_meta_module modules/mod_cern_meta.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule asis_module modules/mod_asis.so
LoadModule info_module modules/mod_info.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule cache_module modules/mod_cache.so
LoadModule suexec_module modules/mod_suexec.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule file_cache_module modules/mod_file_cache.so

Include conf.d/*.conf


ExtendedStatus On


User nobody
Group nobody

ServerAdmin root@localhost


UseCanonicalName Off

DocumentRoot "/var/www/html"

<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>


<Directory "/var/www/html">


# Insert filter
SetOutputFilter DEFLATE

# Don't compress images
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary

# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary


#AuthName "restricted area"
#AuthType Basic
#AuthUserFile /etc/httpd/conf/passwd

#require valid-user

Options Indexes FollowSymLinks

AllowOverride None

Order allow,deny
Allow from all

</Directory>

<IfModule mod_userdir.c>
#
# UserDir is disabled by default since it can confirm the presence
# of a username on the system (depending on home directory
# permissions).
#
UserDir disable

#
# To enable requests to /~user/ to serve the user's public_html
# directory, remove the "UserDir disable" line above, and uncomment
# the following line instead:
#
#UserDir public_html

</IfModule>


DirectoryIndex index.html index.html.var index.php

AccessFileName .htaccess

<Files ~ "^\.ht">
Order allow,deny
Deny from all
</Files>

TypesConfig /etc/mime.types

DefaultType text/plain

<IfModule mod_mime_magic.c>
MIMEMagicFile conf/magic
</IfModule>

HostnameLookups Off



ErrorLog logs/error_log

LogLevel warn

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent



CustomLog logs/access_log combined

ServerSignature On

Alias /icons/ "/var/www/icons/"

<Directory "/var/www/icons">
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>

<IfModule mod_dav_fs.c>
# Location of the WebDAV lock database.
DAVLockDB /var/lib/dav/lockdb
</IfModule>

ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

<Directory "/var/www/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>



IndexOptions FancyIndexing VersionSort NameWidth=*

AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip

AddIconByType (TXT,/icons/text.gif) text/*
AddIconByType (IMG,/icons/image2.gif) image/*
AddIconByType (SND,/icons/sound2.gif) audio/*
AddIconByType (VID,/icons/movie.gif) video/*

AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core

AddIcon /icons/back.gif ..
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^

DefaultIcon /icons/unknown.gif


ReadmeName README.html
HeaderName HEADER.html

IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t

AddLanguage ca .ca
AddLanguage cs .cz .cs
AddLanguage da .dk
AddLanguage de .de
AddLanguage el .el
AddLanguage en .en
AddLanguage eo .eo
AddLanguage es .es
AddLanguage et .et
AddLanguage fr .fr
AddLanguage he .he
AddLanguage hr .hr
AddLanguage it .it
AddLanguage ja .ja
AddLanguage ko .ko
AddLanguage ltz .ltz
AddLanguage nl .nl
AddLanguage nn .nn
AddLanguage no .no
AddLanguage pl .po
AddLanguage pt .pt
AddLanguage pt-BR .pt-br
AddLanguage ru .ru
AddLanguage sv .sv
AddLanguage zh-CN .zh-cn
AddLanguage zh-TW .zh-tw

LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn
no pl pt pt-BR ru sv zh-CN zh-TW

ForceLanguagePriority Prefer Fallback

AddDefaultCharset UTF-8

AddCharset ISO-8859-1 .iso8859-1 .latin1
AddCharset ISO-8859-2 .iso8859-2 .latin2 .cen
AddCharset ISO-8859-3 .iso8859-3 .latin3
AddCharset ISO-8859-4 .iso8859-4 .latin4
AddCharset ISO-8859-5 .iso8859-5 .latin5 .cyr .iso-ru
AddCharset ISO-8859-6 .iso8859-6 .latin6 .arb
AddCharset ISO-8859-7 .iso8859-7 .latin7 .grk
AddCharset ISO-8859-8 .iso8859-8 .latin8 .heb
AddCharset ISO-8859-9 .iso8859-9 .latin9 .trk
AddCharset ISO-2022-JP .iso2022-jp .jis
AddCharset ISO-2022-KR .iso2022-kr .kis
AddCharset ISO-2022-CN .iso2022-cn .cis
AddCharset Big5 .Big5 .big5
AddCharset WINDOWS-1251 .cp-1251 .win-1251
AddCharset CP866 .cp866
AddCharset KOI8-r .koi8-r .koi8-ru
AddCharset KOI8-ru .koi8-uk .ua
AddCharset ISO-10646-UCS-2 .ucs2
AddCharset ISO-10646-UCS-4 .ucs4
AddCharset UTF-8 .utf8

AddCharset GB2312 .gb2312 .gb
AddCharset utf-7 .utf7
AddCharset utf-8 .utf8
AddCharset big5 .big5 .b5
AddCharset EUC-TW .euc-tw
AddCharset EUC-JP .euc-jp
AddCharset EUC-KR .euc-kr
AddCharset shift_jis .sjis



AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz



AddHandler imap-file map

AddHandler type-map var

AddType text/html .shtml
AddOutputFilter INCLUDES .shtml




Alias /error/ "/var/www/error/"

<IfModule mod_negotiation.c>
<IfModule mod_include.c>
<Directory "/var/www/error">
AllowOverride None
Options IncludesNoExec
AddOutputFilter Includes html
AddHandler type-map var
Order allow,deny
Allow from all
LanguagePriority en es de fr
ForceLanguagePriority Prefer Fallback
</Directory>


</IfModule>
</IfModule>

BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4\.0" force-response-1.0
BrowserMatch "Java/1\.0" force-response-1.0
BrowserMatch "JDK/1\.0" force-response-1.0

BrowserMatch "Microsoft Data Access Internet Publishing Provider"
redirect-carefully
BrowserMatch "^WebDrive" redirect-carefully
BrowserMatch "^WebDAVFS/1.[012]" redirect-carefully
BrowserMatch "^gnome-vfs" redirect-carefully











ï„*Ĩï„*LoadModule php5_module /usr/local/Zend/Core/modules/apache2/libphp5.so




AddType application/x-httpd-php .php
DirectoryIndex index.php
Alias /ZendCore /usr/local/Zend/Core/GUI

<Location /ZendCore>
Options FollowSymLinks
AllowOverride None
Order deny,allow
Allow from all
</Location>

ExtendedStatus On
<Location /server-status>
SetHandler server-status
Order deny,allow
Deny from all
Allow from localhost
Allow from 127.0.0.1
</Location>

<Location /server-info>
SetHandler server-info
Order deny,allow
Deny from all
Allow from localhost
Allow from 127.0.0.1
</Location>