Re: [Samba] Overloaded samba server. Is it a bug?

--===============0016867548==
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature";
boundary="=-XsIUQlg/Ajx6xRsLZpmm"


--=-XsIUQlg/Ajx6xRsLZpmm
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2005-11-02 at 18:53 -0300, Martin wrote:
> On Monday 31 October 2005 18:27, Andrew Bartlett wrote:
> > > >Now we are testing this configuration and waiting for the results.
> > >
> > > Bad luck... the load exploited again :(
> >
> > My gut feeling is that this is an e-Directory bug, but have you posted
> > logfiles and traffic somewhere? (Warning: unencrypted LDAP traffic wil=
l
> > include passwords).
> >
>=20
> Dear Andrew:
> =20
> We've been trying to test our environment as recommended in the last=20
> posts. We switched the backend to openLDAP trying to discard the idea of=20
> a bug in eDirectory. And here... the conclusion:
> =20
> =20
> Backend switch:
> =20
> =20
> 1) Sadly, Samba server still getting overloaded. The server doesn't=20
> hang as in the previous scenario, but it gets extremely slow and there's=20
> no way to provide service with it (load grows up to 60 or 70). It stops=20
> responding a couple of minutes after the load gets to the limit.
> =20
> 2) There's an incredible amount of smbd childs in "D" state=20
> (uninterruptable sleep), when the load starts to raise. It happens with=20
> both backends (it's softer with openLDAP, but still unusable).

This is a *very* important clue. If this were an LDAP issue, then Samba
should be in S state, and the ldap processes should be going nuts. =20

> 3) The number of sleeping processes is considerably lower with openLD=
AP.
> =20
> It seems that, something is beating the samba server because of a=20
> bug perhaps, or a misconfiguration. The system is a little (but not=20
> much) tolerant when openLDAP is used as backend (instead of eDir), but=20
> the problems still no matter the directory service being used.
> What do you think about a client triggering this behaviour some way?

I now suspect the LDAP angle is a red herring, and I'm instead thinking
'kernel issue'. =20

> =20
> Weird things found:
> =20
> I'll comment some lines about a couple of strange things i saw. They=20
> may be completely unrelated to the main problem, but here they go just=20
> in case.
> =20
> 1) Some times (according to what an strace attached to the parent=20
> smbd process shows us), a user working on an XLS file starts a curious=20
> behaviour in which the server tries to find a file that no longer exist=20
> in a periodically basis (i.e: loop). We think the user deleted the file,=20
> still an smbd process kept trying to access it. (it was complaining with=20
> "file does not exist" messages permanently) (A few minutes later when the=
loop=20
> was happening we went to the user desktop and found out he has already tu=
rned=20
> off his machine!)
> =20
> We've captured service logs, straces, ps aux snapshots during the=20
> load issue and a couple of lsofs. (The whole samba's logs are more than =
1G=20
> and is impossible to determinate a fail, because the server still respond=
ing=20
> until the load is too high to continue serving files)
> =20
> Is there any way to get some more verbosity? any different way of=20
> debugging (gdb maybe?)?
=20
What would be interesting is to find out where each of those smbd
processes is waiting. Ie, what call is causing the kernel to put the
process in D state. Is it the same call, or a lot of different calls?=20

Given the load problems, I presume the processes are chewing CPU time?

Andrew Bartlett

--=20
Andrew Bartlett http://samba.org/~abartlet/
Samba Developer, SuSE Labs, Novell Inc. http://suse.de
Authentication Developer, Samba Team http://samba.org
Student Network Administrator, Hawker College http://hawkerc.net

--=-XsIUQlg/Ajx6xRsLZpmm
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQBDaT75z4A8Wyi0NrsRAhL3AJ4u6RFbXKnQWg2ctMYeH6 k0q6G24ACgmgCP
h4Uzmqbv6Bfec/bEKkyN+o8=
=F81I
-----END PGP SIGNATURE-----

--=-XsIUQlg/Ajx6xRsLZpmm--


--===============0016867548==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

--
To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/listinfo/samba
--===============0016867548==--