Last update: $Date: 2010/11/16 09:50:59 $
This deals with some work I did back in April '05, but people ask about it often enough that I thought I'd make a technote out of it. There are three parts:
All these emails were originally sent to the Linux Managers' mailing list.Please read all the way to the bottom; as I have refined the solution, I've posted updates down there.
Date: Mon, 4 Apr 2005 11:38:27 +0100 (BST) From: Tom Yates <madlists-AT-teaparty.net> To: Linux Managers <linuxmanagers-AT-linuxmanagers.org> Subject: sshd password-guessing attacksi'm seeing a lot of sshd password-guessing attacks in my logfiles. it's been ramping up over the past few months, but it's now getting seriously frequent.
i've already got sshd tied down, inasmuch as users have to be a member of a particular group to ssh in, and i'm about to start running a password-guessing tool. but sshd is very useful to me, and i'm not prepared to shut it down or limit its accessibility.
what i want is a way to limit the number of new connections any one IP address can make, particularly if it starts issuing an awful lot of bad passwords. an iptables-based solution would be fine, as would an sshd-based solution, but i'll entertain anything that works.
does anyone else have this problem, and if so, what are people doing about it? reply offlist, please, and i'll summarise later this week.
Here's a note from after the event: this is the sort of logfile entry
that worried me:
Mar 20 19:49:30 www sshd(pam_unix)[27300]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=dubioushostname.dubiouscompany.com
Mar 20 19:49:32 www sshd(pam_unix)[27302]: check pass; user unknown
Mar 20 19:49:32 www sshd(pam_unix)[27302]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=dubioushostname.dubiouscompany.com
Mar 20 19:49:34 www sshd(pam_unix)[27304]: check pass; user unknown
Mar 20 19:49:34 www sshd(pam_unix)[27304]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=dubioushostname.dubiouscompany.com
Mar 20 19:49:35 www sshd(pam_unix)[27306]: check pass; user unknown
Date: Tue, 5 Apr 2005 08:27:44 +0100 (BST) From: Tom Yates <madlists-AT-teaparty.net> To: Linux Managers <linuxmanagers-AT-linuxmanagers.org> Subject: SUMMARY: sshd password-guessing attacksA number of answers came in, for which many thanks.
The most common solution is to run sshd on a non-standard port. As Martin Maney noted, 'this offers no protection if someone is actually targeting you and "notices" the odd port(s), but against dumb, scripted shotgun attacks this should be highly effective'.
Another, less-common solution is to use "port-knocking", where you have to make a connection to another port (or ports) before trying sshd. The first attempt produces no visible results, but your access to port 22 is opened up as a result of this. A knock daemon, which I believe to be Franky's, is documented at http://www.zeroflux.org/cgi-bin/cvstrac/knock/wiki; Jeffrey points to another implementation at http://www.soloport.com/iptables.html.
Many recommend disabling password-based authentication entirely, forcing the use of RSA/DSA keys instead.
Bjorn Tore Sund has an elegant solution using the "recent" modules in iptables. This module does appear to be in my kernel (2.4.29), so I'm in the process of recompiling and, if this works, I will post code fragments after testing.
Some use tcp_wrappers, others block netblocks that correspond to countries full of script kiddies, but those doesn't meet my requirements (still open to everyone, but less vulnerable to guessing attempts).
Mike Brodbelt recommends using the sshd configuration option "MaxStartups", although this won't allow me to limit by connecting host, only the total number of unauthenticated connections. This seems to me to make it quite easy for someone else to DoS me by SYN flooding, and individual password guessers seem to be more serial than parallel, so it probably won't stop them. Lance Levsen suggests an iptables "limit"-based solution which seems to have the same drawbacks.
Mike (and others) also recommends one-time passwords, which is an extremely good idea that I'm too lazy to implement (and one I don't think some of my users are quite ready for).
On a tangent, I'm still looking for an open-source version of the RSA SecurID tokens; an implementation which allows me access to all the code which verifies the token, so I can plug it in how I like. Noone heard of any such, have they? Note from much later: I have since found one, and have verified I can use it to secure ssh access; read about it in this technote, if you want.
Greg Dick has a small daemon that adjusts iptables rules in real-time, based on grepping denies from the relevant logfile.
I'm really pleased by the number of responses. Firstly, I no longer feel like I'm the only person worrying about this (although I see I was a complete fool to think I was!). Secondly, different people are trying a lot of imaginative and elegant solutions, and heterogeneous approaches are usually good for security. Some really interesting ideas, things I'd never have thought of, like port-knocking, have emerged as a result of this query. Thanks to all of you!
Specifically, many thanks to Franky Liedeke (I'm guessing the last name. I hope it's right), Mike Kercher, Wayne E Goodrich, Will H. Backman, David Boutcher, Mike Brodbelt, Greg Dick, Jeffrey L. Taylor, Martin Maney, Lance Levsen, Cary Penniman, Clif Smith, and especially to Bjorn Tore Sund, who did a fair amount of digging around for version numbers for me.
Date: Fri, 8 Apr 2005 11:05:42 +0100 (BST) From: Tom Yates <madlists-AT-teaparty.net> To: Linux Managers <linuxmanagers-AT-linuxmanagers.org> Subject: SUMMARY: sshd password-guessing attacks
On Tue, 5 Apr 2005, Tom Yates wrote:
> Bjorn Tore Sund has an elegant solution using the "recent" modules in
> iptables. This module does appear to be in my kernel (2.4.29), so I'm in
> the process of recompiling and, if this works, I will post code fragments
> after testing.
it proved necessary to update my kernel (2.4.29 has 'recent', but i wasn't compiling it, so i went the whole hog and upgraded to 2.4.30 as well as turning on the module), and iptables (1.2.8 doesn't have the hooks for 'recent'; there is evidence that later versions of 1.2 do, but i just went to 1.3.1 on a general principle of "oooh, shiny!").
i now have the following rules in place:
IPT=/usr/local/iptables/sbin/iptables # [many lines later] $IPT -A INPUT -p tcp --syn --dport 22 -m recent --name sshattack --set $IPT -A INPUT -p tcp --dport 22 --syn -m recent --name sshattack \ --rcheck --seconds 60 --hitcount 3 -j LOG --log-prefix 'SSH REJECT: ' $IPT -A INPUT -p tcp --dport 22 --syn -m recent --name sshattack \ --rcheck --seconds 60 --hitcount 3 -j REJECT --reject-with tcp-resetwhich means that if, in a rolling 60-second window, two SYNs have been received on port 22 from a particular IP address, the third and subsequent SYNs are rejected with a tcp reset. after an amusing incident during testing where i shut down all my ssh sessions to the box (fortunately, i was testing on a local machine), i decided only to reject further SYNs, so existing sessions are not dropped, even from a host which triggers the trap. note that these rules will need to come before any blanket permit to port 22.
it's been very effective so far. logwatch no longer reports lengthy fishing expeditions, but none of my users has reported any problems. examination of the kernel logfiles suggests that some fishing tools get a single refusal and bugger off, whereas others sputter on, blindly assuming that since they were allowed to connect twice, they will ever be. in neither case, though, are they likely to compromise my security or waste much of my bandwidth.
i post this as it may be of particular use to people running distros later than mine, for which no recompilation may be necessary. thanks again to all, especially bjorn.
# allow return packets from connections we initiated $IPT -A INPUT -i eth0 -p tcp -m state --state ESTABLISHED -j ACCEPT # rate-limit connections to sshd $IPT -A INPUT -p tcp --syn --dport 22 -m recent --name sshattack --set $IPT -A INPUT -p tcp --dport 22 --syn -m recent --name sshattack --rcheck --seconds 60 --hitcount 3 -j LOG --log-prefix 'SSH REJECT: ' $IPT -A INPUT -p tcp --dport 22 --syn -m recent --name sshattack --rcheck --seconds 60 --hitcount 3 -j REJECT --reject-with tcp-reset
A little testing with GNU
netcat reveals that continuous reuse of the same source port number on
connection attempts bypasses the rate-limiter (nc -t -p 33334
www.teaparty.net 22 is allowed to get an ssh banner over and over
again), because the iptables state mechanism is synchronicity-based.
The solution is to move the ssh rate-limiting code right to the top of one's firewall script.
I don't know why this happens; I suspect it has to do with extra SYNs being in the connection stream due to weirdly-configured firewalls. In any case, changing the rules to use the state module to detect new connections seems to have fixed things.
$IPT -A INPUT -p tcp -m state --state NEW --dport 22 -m recent --name sshattack --set $IPT -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --name sshattack --rcheck --seconds 60 --hitcount 3 -j LOG --log-prefix 'SSH REJECT: ' $IPT -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --name sshattack --rcheck --seconds 60 --hitcount 3 -j REJECT --reject-with tcp-reset