Dictionary sshd attacks

Is it possible to have sshd or some other daemon recognize
a dictionary attack in progress, and to "shun" that IP for
(say) an hour? The machanics seem simple to me: A log file
reader spots the pattern in real time and issues an iptables
command to DROP port 22 packets from the identified IP address,
sends a syslog message, and creates an "at" job to remove the
iptabes rule after some period of time. A more sophisticated
tool could continue to monitor the logs for the attack and only
reactivate the port after X minutes after the attack ends.

I know I've read of such tools someplace before, but I can't
remember any details.

Some of my servers have been under attack for many months now.
Sometimes twice a day, hundreds to thousands of ssh connections
are attempted using a dictionary of common usernames.

I have been saving the logs but no pattern has emerged yet
from the IP addresses. Of course they don't get in (yet),
but I'd like a more automatic way to respond to such attacks.

Lessons learned:

Have account naming policies that forbid usernames that commonly
appear in dictionary attacks.

Make sure *all* system accounts are disabled, and where possible
have invalid shells (such as /bin/false or /sbin/nologin).

Never allow sshd (or other) root logins.

Configure the PAM "su" policy to only allow a few select users to
succeed with the su command. (The members of group "wheel".) Other
users who attempt "su" will fail even if they know the password.

-Wayne

Wayne wrote:
> Is it possible to have sshd or some other daemon recognize
> a dictionary attack in progress, and to "shun" that IP for
> (say) an hour? The machanics seem simple to me: A log file
> reader spots the pattern in real time and issues an iptables
> command to DROP port 22 packets from the identified IP address,
> sends a syslog message, and creates an "at" job to remove the
> iptabes rule after some period of time. A more sophisticated
> tool could continue to monitor the logs for the attack and only
> reactivate the port after X minutes after the attack ends.
>
> I know I've read of such tools someplace before, but I can't
> remember any details.

Why not shut things down for a while after so many failed
attempts from the IP? I would think that would be sufficient.

>
> Some of my servers have been under attack for many months now.
> Sometimes twice a day, hundreds to thousands of ssh connections
> are attempted using a dictionary of common usernames.

Make sure you protect your SSH to allow only key'd access, or
at least minimize the number of valid users.

>
> I have been saving the logs but no pattern has emerged yet
> from the IP addresses. Of course they don't get in (yet),
> but I'd like a more automatic way to respond to such attacks.
>
> Lessons learned:
>
> Have account naming policies that forbid usernames that commonly
> appear in dictionary attacks.
>
> Make sure *all* system accounts are disabled, and where possible
> have invalid shells (such as /bin/false or /sbin/nologin).
>
> Never allow sshd (or other) root logins.
>
> Configure the PAM "su" policy to only allow a few select users to
> succeed with the su command. (The members of group "wheel".) Other
> users who attempt "su" will fail even if they know the password.

These are all good ideas.

Wayne <wayne@nospam.4me.invalid> wrote:
> Is it possible to have sshd or some other daemon recognize
> a dictionary attack in progress, and to "shun" that IP for
> (say) an hour? The machanics seem simple to me: A log file
> reader spots the pattern in real time

"Spot the pattern" can becom arbitrarily complex. That's hump
number one you'll face.

> and issues an iptables
> command to DROP port 22 packets from the identified IP address,
> sends a syslog message, and creates an "at" job to remove the
> iptabes rule after some period of time.

I don't know about you, but I always feel uneasy when some
script messes with my Netfilter rules autopmagically. Besides
zeroing counters, that is. That (to me) would be hump number two.

> I know I've read of such tools someplace before, but I can't
> remember any details.

The places I'd look for such things would be Freshmeat,
Bugtraq/Focus-Linux and Google, in that order.

> Some of my servers have been under attack for many months now.
> Sometimes twice a day, hundreds to thousands of ssh connections
> are attempted using a dictionary of common usernames.
>
> I have been saving the logs but no pattern has emerged yet
> from the IP addresses. Of course they don't get in (yet),
> but I'd like a more automatic way to respond to such attacks.

Well, I can understand that. I've taken to nailing my Netfilter
rules shut for SSH (besides a handful low-risk addresses). Makes
me sleep much better :)

> Lessons learned:
>
> Have account naming policies that forbid usernames that commonly
> appear in dictionary attacks.

Agreed. Cracklib can do a nice job here.

> Make sure *all* system accounts are disabled, and where possible
> have invalid shells (such as /bin/false or /sbin/nologin).

And shun software that needs accounts with shells (or uses nobody
as hardcoded run-as user).

> Never allow sshd (or other) root logins.

I allow root logins on the console. If physical security is
compromised in the co-lo, nearly all bets are off.

> Configure the PAM "su" policy to only allow a few select users to
> succeed with the su command. (The members of group "wheel".) Other
> users who attempt "su" will fail even if they know the password.

My distribution of choice does this by default.

Regards,
Tobias

--
export DISPLAY=vt100

In comp.os.linux.security Wayne <wayne@nospam.4me.invalid>:
> Is it possible to have sshd or some other daemon recognize
> a dictionary attack in progress, and to "shun" that IP for
> (say) an hour? The machanics seem simple to me: A log file
> reader spots the pattern in real time and issues an iptables
> command to DROP port 22 packets from the identified IP address,
> sends a syslog message, and creates an "at" job to remove the
> iptabes rule after some period of time. A more sophisticated
> tool could continue to monitor the logs for the attack and only
> reactivate the port after X minutes after the attack ends.

> I know I've read of such tools someplace before, but I can't
> remember any details.

Taking a short look:

http://denyhosts.sourceforge.net/
http://www.csc.liv.ac.uk/~greg/sshdfilter/
http://www.hexten.net/sw/pam_abl/index.mhtml
http://fail2ban.sourceforge.net/

And probably dozen others look promising. Even if the easiest
would just be to restrict ssh access to a few trusted
hosts/networks.

[..]

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 389: /dev/clue was linked to /dev/null

On 2005-07-17, Michael Heiming wrote:
> In comp.os.linux.security Wayne <wayne@nospam.4me.invalid>:
>> Is it possible to have sshd or some other daemon recognize
>> a dictionary attack in progress, and to "shun" that IP for

There was a recent posting about this on slashdot, and several people
posted iptables methods of dealing with this, using the "recent" rule.

http://it.slashdot.org/article.pl?sid=05/07/16/1615233
"Rundown on SSH Brute Force Attacks"

What I'd like to see is to get the "tarpit" target incorporated into
the mainline kernel (currently it is only in the extra patch-o-matic).
Then everyone could easily use recent and tarpit to slow the scans.

I'd love to tie up these idiots' ssh scanners as much as possible!

This is what was posted (there was a more complicated one that allowed
for ssh IP whitelisting):

-A PREROUTING -m tcp -p tcp -d $EXTERNAL --dport 22 -m recent --rcheck
--hitcount 3 --seconds 600 -j LOG --log-prefix "SSH attack: "
-A PREROUTING -m tcp -p tcp -d $EXTERNAL --dport 22 -m recent --rcheck
--hitcount 3 --seconds 600 -j DROP
-A PREROUTING -m tcp -p tcp -d $EXTERNAL --dport 22 -m recent --set -j DNAT
--to-destination $INTERNAL:22
-A OUTPUT -m tcp -p tcp -d $EXTERNAL --dport 22 -j DNAT --to-destination
$INTERNAL:22

So if you have the tarpit patch added, I guess all that needs to be done
is to replace "-j DROP" with "-j TARPIT".

--
Brian Hall
Linux Consultant
http://pcisys.net/~brihall

On Sat, 16 Jul 2005 20:38:04 +0000, Wayne wrote:

> Is it possible to have sshd or some other daemon recognize
> a dictionary attack in progress, and to "shun" that IP for
> (say) an hour? The machanics seem simple to me: A log file
> reader spots the pattern in real time and issues an iptables
> command to DROP port 22 packets from the identified IP address,
> sends a syslog message, and creates an "at" job to remove the
> iptabes rule after some period of time. A more sophisticated
> tool could continue to monitor the logs for the attack and only
> reactivate the port after X minutes after the attack ends.
>
> I know I've read of such tools someplace before, but I can't
> remember any details.
>
> Some of my servers have been under attack for many months now.
> Sometimes twice a day, hundreds to thousands of ssh connections
> are attempted using a dictionary of common usernames.
>
> I have been saving the logs but no pattern has emerged yet
> from the IP addresses. Of course they don't get in (yet),
> but I'd like a more automatic way to respond to such attacks.
>
> Lessons learned:
>
> Have account naming policies that forbid usernames that commonly
> appear in dictionary attacks.
>
> Make sure *all* system accounts are disabled, and where possible
> have invalid shells (such as /bin/false or /sbin/nologin).
>
> Never allow sshd (or other) root logins.
>
> Configure the PAM "su" policy to only allow a few select users to
> succeed with the su command. (The members of group "wheel".) Other
> users who attempt "su" will fail even if they know the password.
>
> -Wayne


I have elected to take a different approach.

1 - in the SSH config file, I restrict login ability to specific grouped
users, specific usernames or specific IP/subnets (this depends upon the
type of system I'm protection and who uses it)

2 - I have a perl script cronned to run once a minute which tails the SSH
log and looks for login failures. If a certain number of failures are
seen on a specific IP address or against a specific username, it puts the
user's IP address(es) in the hosts.deny file (this restricting their login
access). It also logs the blocking, and emails me info about the
offending IP address or username being used - so I can review further if I
want to.

3 - If I should want to unblock the blocked IPs on a specific box, I have
another script which can be cronned to auto-unblock IPs after X time or
can be used manually to unblock a specific IP/subnet.

With this home grown solution in place, I never have to worry about those
types of ssh attacks. I mostly address unblock requests manually.
With the amount of users on my servers, it is not much work at all. I can
also customize the solution as things situations change.

I hope this helps give you an alternate solution to your issue.

James

Wayne napsal(a):
> Is it possible to have sshd or some other daemon recognize
> a dictionary attack in progress, and to "shun" that IP for
> (say) an hour?

Try pam_abl.

--Petr

Petr Pisar wrote:
> Wayne napsal(a):
>
>> Is it possible to have sshd or some other daemon recognize
>> a dictionary attack in progress, and to "shun" that IP for
>> (say) an hour?
>
>
> Try pam_abl.
>
> --Petr

That is exactly what I was looking for! Thanks!!

-Wayne Pollock

On Sat, 16 Jul 2005, Wayne wrote:

> Is it possible to have sshd or some other daemon recognize
> a dictionary attack in progress, and to "shun" that IP for
> (say) an hour?

This has been a problem I've had some interest in... I implemented a
little daemon in perl to deal with it. I hope posting this much code
isn't frowned on... if so, sorry. Otherwise I'd welcome comments on this.

It doesn't worry about removing an ip from the rules: for me, periodically
running my rc.iptables startup script to set things back to the baseline
is sufficient.

--skip--

#!/usr/bin/perl -w
#
# sshd_monitor.pl
#
# A daemon for monitoring sshd's log to detect root or illegal user
# login attempts and to block access from the source ip of such
# attempts using iptables.
#
# SYNOPSIS:
#
# sshd_monitor.pl log_file_to_monitor

use strict;
use POSIX;
use Sys::Syslog;

#
# System environment sensitive values, change as needed:
#

my $pid_file_name = $> == 0 ? "/var/run/" : "./";
$pid_file_name .= "sshd_monitor.pid";
# The file our pid is written to.

my $logger_facility = $> == 0 ? "authpriv" : "user";
my $logger_priority = "warning";
# The syslog parameters for logging activity.

my $monitoring_command = "/usr/bin/tail -n 0 --follow=name --retry ";
# The call used to provide input from the monitored log file. Must
# accept a bare, final command-line arg designating the file name.


my $usage = "usage: $0 logfile_to_monitor";

if (scalar(@ARGV) != 1)
{
die "$usage\n";
}

my $log_file_name = shift(@ARGV);

# Daemonize
my $pid = fork;
exit if $pid;
close STDIN;
close STDOUT;
close STDERR;
POSIX::setsid() or die "Can't start new session: $!\n";

open(RUNFILE, ">$pid_file_name");
print RUNFILE "$$";
close(RUNFILE);

my $tail_pid;
my $terminating = 0;

$SIG{TERM} =
sub
{
logger("Received sig TERM");
kill(15, $tail_pid);
$terminating = 1;
};

$SIG{HUP} =
sub
{
logger("Received sig HUP");
kill(15, $tail_pid);
};

while (!$terminating)
{
$tail_pid = open(LOG, "$monitoring_command $log_file_name |");
logger("(Re)start monitoring %s", $log_file_name);

my $prev_ip; # could get a few before this kicks in
while (my $line = <LOG>)
{
if ($line =~ /sshd/)
{
if ($line =~ /Failed password for (.*) from (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/)
{
my $user = $1;
my $ip = $2;
if ($ip ne $prev_ip
&& ($user eq "ROOT" || $user =~ /illegal user/))
{
$prev_ip = $ip;
logger("Blocking access from %s", $ip);
my $command =
"/sbin/iptables -I INPUT --source $ip -j REJECT";

if ($> == 0)
{
system $command;
}
else
{
logger($command);
}
}
}
}
}

# Should only be here if the monitor process is terminated so this
# isn't expected to want to block.
#
if ($tail_pid != waitpid($tail_pid, WNOHANG))
{
logger("PID=%d is unxepectedly not dead yet", $tail_pid);
}
}

logger("Shutting down");
unlink($pid_file_name);

sub logger
{
my ($format, @args) = @_;

openlog("sshd monitor", "pid", $logger_facility);
syslog($logger_priority, $format, @args);
closelog();
}


0;

> 2 - I have a perl script cronned to run once a minute which tails the SSH

many attempt to login can occour in one minute
this solution seems a bit too static to me

> log and looks for login failures. If a certain number of failures are
> seen on a specific IP address or against a specific username, it puts the
> user's IP address(es) in the hosts.deny file (this restricting their login
> access). It also logs the blocking, and emails me info about the
> offending IP address or username being used - so I can review further if I
> want to.

though, can you provide your script to me/this group?
--
lg niko
icq:# 129022192
--
Nur ein Sieg, für den es keinen Besiegten gibt,
ist wirklich ein Sieg
(Hans Margolius)