Re: [squid-users] cache dir limits

Hi,

2 recommendations-

1- Install SCSI raid instead (smaller files will be accessed faster)

2- Format the partition that has the cache as ReiserFS.

These can be accomplished *easily* by adding a SCSI Raid controller to
your current system, create a new partition on said controller, format
it and tell squid where to put it. :)

Hope it helps,

Dave



On Fri, 2003-12-05 at 06:57, Victor Ivanov wrote:
> Hello,
>
> I'm running squid 2.5 stable 3. Actualy it should be stable4 (freebsd
> squid port 2.5_4), it uses lots of patches.
> Anyway, the behavior is the same with squid 2.4.
>
> The problem is my cache dir became too big and now it takes about
> five minutes to write the cache. In the meantime there's no service.
>
> The other problem is that when squid starts it takes too long to
> rebuild the store, and while it rebuilds it there's heavy disk usage.
>
> I'm asking if I've reached the limits of squid on this hardware.
> CPU: AMD XP 2200+ (CPU load is low)
> RAM: 512M + 1G swap
> HDD: 2x Maxtor UDMA133 using RAID0 (that obviously was a mistake)
>
> It seems this raid has awfully low speed.
>
> Anyway, here's some from the log:
> (some lines were omitted for readability)
>
> 2003/12/05 12:14:50| storeDirWriteCleanLogs: Starting...
> 2003/12/05 12:14:57| 65536 entries written so far.
> ...
> 2003/12/05 12:16:06| 720896 entries written so far.
> ...
> 2003/12/05 12:20:03| 3014656 entries written so far.
> 2003/12/05 12:22:53| Finished. Wrote 4594867 entries.
> 2003/12/05 12:22:53| Took 482.9 seconds (9515.9 entries/sec).
>
> (That was the last shutdown from 2.4STABLE7, after this
> it's 2.5STABLE3 with the patches)
>
> 2003/12/05 12:22:54| Store rebuilding is 0.1% complete
> 2003/12/05 12:23:09| Store rebuilding is 15.9% complete
> 2003/12/05 12:23:24| Store rebuilding is 30.3% complete
> 2003/12/05 12:23:39| Store rebuilding is 42.3% complete
> 2003/12/05 12:23:54| Store rebuilding is 53.9% complete
> 2003/12/05 12:24:09| Store rebuilding is 65.1% complete
> 2003/12/05 12:24:24| Store rebuilding is 75.6% complete
> 2003/12/05 12:24:39| Store rebuilding is 83.3% complete
> 2003/12/05 12:24:58| Store rebuilding is 84.9% complete
> 2003/12/05 12:25:20| Store rebuilding is 85.0% complete
> 2003/12/05 12:25:36| Store rebuilding is 85.3% complete
> 2003/12/05 12:25:52| Store rebuilding is 85.9% complete
> 2003/12/05 12:26:08| Store rebuilding is 86.1% complete
> 2003/12/05 12:26:26| Store rebuilding is 86.3% complete
> 2003/12/05 12:26:43| Store rebuilding is 86.4% complete
> 2003/12/05 12:26:59| Store rebuilding is 86.5% complete
> 2003/12/05 12:27:15| Store rebuilding is 86.6% complete
> 2003/12/05 12:27:30| Store rebuilding is 86.6% complete
> ...and then it gets even slower, while the disk is
> at its limits... actually the array is at its limits...
> 2003/12/05 12:30:07| Store rebuilding is 87.4% complete
> ...
> 2003/12/05 12:54:06| Store rebuilding is 90.0% complete
> ...
> 2003/12/05 13:51:52| Store rebuilding is 96.9% complete
> FATAL: xcalloc: Unable to allocate 1 blocks of 4104 bytes!
>
> And then it starts all over. Now this xcalloc error is
> probably due to ulimit configuration and I'm going to
> extend it. This is not the problem I'm asking about.
>
> I know I misconfigured the whole thing. Any pointers
> what should I fix? Here are some lines from the config:
>
> cache_mem 384 MB
> maximum_object_size 32768 KB
> maximum_object_size_in_memory 32 KB
> cache_replacement_policy heap LFUDA
> cache_dir ufs /usr/local/squid/cache 100000 128 256
>
> The rest is about the default. There are four delay pools,
> and some ...long regex acl's, but the CPU's fast enough :)
>
> P.S. The last used dir in the cache is 46/1F
> P.P.S. I don't use diskd anymore. It's more stable this way.
> Actually, I don't remember why I turned to direct ufs access,
> but it saved me at the time :)
>
> Regards