Author |
Message |
Antimon
Joined: Jan 18, 2005 Posts: 4145 Location: Sweden
Audio files: 371
G2 patch files: 100
|
Posted: Fri May 29, 2009 2:47 pm Post subject:
How do I know if my server is being hijacked? |
|
|
So me and Eike (Noodulator) are trying out a little collaboration, and I have set up an Ubuntu server that we may use to exchange data. There is no one else that should know about this server, but every now and then I hear the hard drive ticking (it's a nine year-something old computer), and I go in and check netstat -n and see an something like this:
Code: | antimon@graysky:~$ netstat -n
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 192.168.3.161:22 203.66.151.27:36823 TIME_WAIT
tcp 0 0 192.168.3.161:22 203.66.151.27:36689 TIME_WAIT
tcp 0 0 192.168.3.161:22 203.66.151.27:38047 ESTABLISHED
tcp 0 0 192.168.3.161:22 203.66.151.27:36941 TIME_WAIT
tcp 0 0 192.168.3.161:22 192.168.3.150:49877 ESTABLISHED
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ] DGRAM 2754 @/com/ubuntu/upstart
unix 2 [ ] DGRAM 2944 @/org/kernel/udev/udevd |
a ps -ax a coupld of seconds later gave me this:
Code: | antimon@graysky:~$ ps -ax
Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
PID TTY STAT TIME COMMAND
1 ? Ss 0:03 /sbin/init
2 ? S< 0:00 [kthreadd]
3 ? S< 0:00 [migration/0]
4 ? S< 0:01 [ksoftirqd/0]
5 ? S< 0:00 [watchdog/0]
6 ? S< 0:00 [events/0]
7 ? S< 0:00 [khelper]
8 ? S< 0:00 [kstop/0]
9 ? S< 0:00 [kintegrityd/0]
10 ? S< 0:00 [kblockd/0]
11 ? S< 0:00 [kacpid]
12 ? S< 0:00 [kacpi_notify]
13 ? S< 0:00 [cqueue]
14 ? S< 0:12 [ata/0]
15 ? S< 0:00 [ata_aux]
16 ? S< 0:00 [ksuspend_usbd]
17 ? S< 0:00 [khubd]
18 ? S< 0:00 [kseriod]
19 ? S< 0:00 [kmmcd]
20 ? S< 0:00 [btaddconn]
21 ? S< 0:00 [btdelconn]
22 ? S 0:00 [pdflush]
23 ? S 0:00 [pdflush]
24 ? S< 0:00 [kswapd0]
25 ? S< 0:00 [aio/0]
26 ? S< 0:00 [ecryptfs-kthrea]
29 ? S< 0:00 [scsi_eh_0]
30 ? S< 0:45 [scsi_eh_1]
31 ? S< 0:00 [kstriped]
32 ? S< 0:00 [kmpathd/0]
33 ? S< 0:00 [kmpath_handlerd]
34 ? S< 0:00 [ksnapd]
35 ? S< 0:00 [kondemand/0]
36 ? S< 0:00 [krfcommd]
393 ? Ss 0:00 sshd: unknown [priv]
394 ? S 0:00 sshd: unknown [net]
395 pts/0 R+ 0:00 ps -ax
626 ? S< 0:04 [kjournald]
760 ? S<s 0:00 /sbin/udevd --daemon
908 ? S< 0:00 [w1_bus_master1]
1831 tty4 Ss+ 0:00 /sbin/getty 38400 tty4
1832 tty5 Ss+ 0:00 /sbin/getty 38400 tty5
1840 tty2 Ss+ 0:00 /sbin/getty 38400 tty2
1841 tty3 Ss+ 0:00 /sbin/getty 38400 tty3
1842 tty6 Ss+ 0:00 /sbin/getty 38400 tty6
1911 ? Ss 0:00 /usr/sbin/acpid -c /etc/acpi/events -s /var/run/acpid.socke
1956 ? Ss 0:04 /sbin/syslogd -u syslog
1979 ? S 0:00 /bin/dd bs 1 if /proc/kmsg of /var/run/klogd/kmsg
1981 ? Ss 0:00 /sbin/klogd -P /var/run/klogd/kmsg
2004 ? Ss 0:07 /bin/dbus-daemon --system
2028 ? Ss 0:02 /usr/sbin/sshd
2064 ? Ss 0:17 /usr/sbin/hald
2067 ? Ssl 0:10 /usr/sbin/console-kit-daemon
2130 ? S 0:01 hald-runner
2160 ? S 0:00 hald-addon-input: Listening on /dev/input/event0
2184 ? S 0:03 hald-addon-storage: polling /dev/sr0 (every 2 sec)
2185 ? S 0:03 hald-addon-storage: polling /dev/sr1 (every 2 sec)
2186 ? S 0:01 hald-addon-storage: no polling on /dev/fd0 because it is ex
2192 ? S 0:00 hald-addon-acpi: listening on acpid socket /var/run/acpid.s
2206 ? Ss 0:00 /usr/sbin/bluetoothd
2266 ? Ss 0:00 /usr/sbin/NetworkManager --pid-file /var/run/NetworkManager
2273 ? S 0:00 /sbin/wpa_supplicant -u -f /var/log/wpa_supplicant.log
2277 ? S 0:00 /usr/sbin/nm-system-settings --config /etc/NetworkManager/n
2314 ? Ss 0:00 /usr/sbin/cupsd
2342 ? Ss 0:00 /usr/bin/system-tools-backends
2415 ? Ss 0:00 /usr/sbin/atd
2448 ? Ss 0:00 /usr/sbin/cron
2546 tty1 Ss+ 0:00 /sbin/getty 38400 tty1
32504 ? Ss 0:00 sshd: antimon [priv]
32516 ? S 0:00 sshd: antimon@pts/0
32518 pts/0 Ss 0:00 -bash
antimon@graysky:~$ |
Gogling for the IP in netstat gives hits on "we banned this address for repeatedly trying to do stuff" and similar things. So - is someone doing bad stuff to my server?
/Stefan _________________ Antimon's Window
@soundcloud @Flattr home - you can't explain music |
|
Back to top
|
|
|
EdisonRex
Site Admin
Joined: Mar 07, 2007 Posts: 4579 Location: London, UK
Audio files: 172
|
Posted: Sat May 30, 2009 4:13 am Post subject:
|
|
|
Do you have ssh enabled to the world? Have you updated your ssh keys (this is an imperative)? pids 393 and 394 would concern me. _________________ Garret: It's so retro.
EGM: What does retro mean to you?
Parker: Like, old and outdated.
Home,My Studio,and another view |
|
Back to top
|
|
|
Antimon
Joined: Jan 18, 2005 Posts: 4145 Location: Sweden
Audio files: 371
G2 patch files: 100
|
Posted: Sat May 30, 2009 4:32 am Post subject:
|
|
|
I am running on the default config for ssh, so I guess it's open for everyone who knows a password.
I haven't updated the keys - does this make any difference even if the host is open to everyone?
/Stefan _________________ Antimon's Window
@soundcloud @Flattr home - you can't explain music |
|
Back to top
|
|
|
EdisonRex
Site Admin
Joined: Mar 07, 2007 Posts: 4579 Location: London, UK
Audio files: 172
|
Posted: Sat May 30, 2009 4:46 am Post subject:
|
|
|
Start here.
Then go get updates. Today. _________________ Garret: It's so retro.
EGM: What does retro mean to you?
Parker: Like, old and outdated.
Home,My Studio,and another view |
|
Back to top
|
|
|
Antimon
Joined: Jan 18, 2005 Posts: 4145 Location: Sweden
Audio files: 371
G2 patch files: 100
|
Posted: Sat May 30, 2009 5:07 am Post subject:
|
|
|
OK, getting updates. However, I downloaded and installed Ubuntu just about a month ago, so I think that the ssl package ought to be up to date enough.
/Stefan _________________ Antimon's Window
@soundcloud @Flattr home - you can't explain music |
|
Back to top
|
|
|
EdisonRex
Site Admin
Joined: Mar 07, 2007 Posts: 4579 Location: London, UK
Audio files: 172
|
Posted: Sat May 30, 2009 10:32 am Post subject:
|
|
|
How is your password strength? What specific flavour of Ubuntu did you download?
Looks like, if they haven't actually got in, they're trying, so it might be more effective to block either that single IP address, or all of APNIC 203 from your server. You might be able to do that on your internet router.
Looking at that ps ax output again, sshd (unknown) is simply an unknown username, so it looks like they are trying but have not actually succeeded.
Your keys are probably up to date, or you'd have been hacked already. _________________ Garret: It's so retro.
EGM: What does retro mean to you?
Parker: Like, old and outdated.
Home,My Studio,and another view |
|
Back to top
|
|
|
Kassen
Janitor
Joined: Jul 06, 2004 Posts: 7678 Location: The Hague, NL
G2 patch files: 3
|
Posted: Sat May 30, 2009 12:15 pm Post subject:
|
|
|
I wouldn't lose too much sleep over this; a Ubuntu exploit out in the wild would have hit the relevant news sites (slashdot, etc) quite quickly. Ubuntu out of the box is really quite secure; it lasted the whole p0wn2own competition while Vista and OSX fell.
Just keep using the automatic updates and pick a strong password (20 characters worth of keyboard bashing should do the trick, just copy-paste it and it'll last a *long* time against brute-force).
A much more likely cause for random disc access is automatic disk indexing. I turn that off and instead sort my files logically; disk indexing and realtime audio don't really like each other.
That said; those pids do look a bit odd. I think I might try killing those and seeing what happened. I might also try monitoring my network usage while not browsing and when I was sure my collaborator wasn't accessing anything, it'd be interesting to see whether those processes actually send or receive any data beyond what would be generated by port scans. I do think the default in Ubuntu is to close everything but banning that IP can't hurt. _________________ Kassen |
|
Back to top
|
|
|
BobTheDog
Joined: Feb 28, 2005 Posts: 4044 Location: England
Audio files: 32
G2 patch files: 15
|
Posted: Sat May 30, 2009 1:24 pm Post subject:
|
|
|
That IP is in Taiwan, know anyone there?
I would stick SSH on another port and not use 22. Open /etc/ssh/sshd_config file and look for line Port 22 and change it to another free port of your choice. Restart sshd server.
Look at psad http://packages.ubuntu.com/dapper/admin/psad and use very strong passwords. |
|
Back to top
|
|
|
Antimon
Joined: Jan 18, 2005 Posts: 4145 Location: Sweden
Audio files: 371
G2 patch files: 100
|
Posted: Sat May 30, 2009 4:44 pm Post subject:
|
|
|
Thanks for all your tips, much appreciated. I think the password is ok, even though it's not 20 characters. I'm checking top, netstat and ps every now and then, and there never seems to be anything other then what I showed in previous posts. This server is just for storing files and won't do any realtime audio, so the odd extra disk access does no harm. Jan said on the chat that might not be much use changing ports, since the hackers try them all, but I guess it'll not do much harm, since it's easy to do.
Googling for the IPs that appear in netstat is pretty informative, usually I get several hits on pages where sysadmins have put up the config files for blocked IPs. Just copying one of those might be an idea, though I guess the hackers change their IPs pretty often.
/Stefan _________________ Antimon's Window
@soundcloud @Flattr home - you can't explain music |
|
Back to top
|
|
|
blue hell
Site Admin
Joined: Apr 03, 2004 Posts: 24079 Location: The Netherlands, Enschede
Audio files: 278
G2 patch files: 320
|
Posted: Sat May 30, 2009 5:08 pm Post subject:
|
|
|
About changing IPs regularly ... I'm seeing more and more small scale use of IPs for spamming here. Not too long ago I would have lists of like 20 or 30 signups from the same IP, more recently I see they come in groups of four or five signups from one IP.
And no it doesn't hurt to change the ssh port of course, but don't expect security from it. BTW, why not use FTP for file sharing? Or SFTP whwn being paranoid _________________ Jan
also .. could someone please turn down the thermostat a bit.
|
|
Back to top
|
|
|
BobTheDog
Joined: Feb 28, 2005 Posts: 4044 Location: England
Audio files: 32
G2 patch files: 15
|
Posted: Sat May 30, 2009 11:46 pm Post subject:
|
|
|
I must admit I thought if you where to change the SSH port and use PSAD to stop port scans you would be a little more secure.
Leaving ports at default values will allow attacks that do not need to scan ports.
Maybe I'm paranoid though |
|
Back to top
|
|
|
Antimon
Joined: Jan 18, 2005 Posts: 4145 Location: Sweden
Audio files: 371
G2 patch files: 100
|
Posted: Sun May 31, 2009 1:17 am Post subject:
|
|
|
Isn't ftp insecure? Can I set up sshd to only accept sftp connections?
/Stefan _________________ Antimon's Window
@soundcloud @Flattr home - you can't explain music |
|
Back to top
|
|
|
Kassen
Janitor
Joined: Jul 06, 2004 Posts: 7678 Location: The Hague, NL
G2 patch files: 3
|
Posted: Sun May 31, 2009 3:55 am Post subject:
|
|
|
I do think that SSH should be more secure, yes.
Strong passwords matter and after that it also matters what encryption protocol is used, though by the time you have to worry about that you likely have larger issues as well (by that I mean a government or well funded organisation chasing you).
One of the obvious way to secure FTP (or SSH for that matter) is to only accept connections from certain IP's. that's the way to go when you only want to share files with one friend or admin your server from home exclusively. I'd look into that if I were you as that would suit your needs perfectly. I never set that up myself but that's what the pro's do and it can't be very hard. _________________ Kassen |
|
Back to top
|
|
|
Kassen
Janitor
Joined: Jul 06, 2004 Posts: 7678 Location: The Hague, NL
G2 patch files: 3
|
Posted: Sun May 31, 2009 4:05 am Post subject:
|
|
|
BobTheDog wrote: |
Leaving ports at default values will allow attacks that do not need to scan ports.
Maybe I'm paranoid though |
I don't see any harm in it but I wouldn't rely on it for security as port scans are easy. I'd rather have a strong password, strong crypto and a suitable protocol, potentially only open to certain IP's than play "hide&seek".
The serious options are all there and are free so when we are going to get paranoid let's do it for real. Don't forget to put a container of solidox near the computer; nothing wipes HD's when the doorbel expectantly rings like solidox. It goes without saying that you already covered your computer room in a faraday cage and are wearing your tinfoil hat. You do have a tinfoil hat, right? _________________ Kassen |
|
Back to top
|
|
|
BobTheDog
Joined: Feb 28, 2005 Posts: 4044 Location: England
Audio files: 32
G2 patch files: 15
|
Posted: Sun May 31, 2009 6:04 am Post subject:
|
|
|
Tinfoil full body suit, I found the hat alone did not offer adequate protection. |
|
Back to top
|
|
|
|