Frackin script kiddies!!

[David Kramer]

Matthew Gillen wrote:
> On 08/02/2010 10:20 PM, Dan Ritter wrote:
>> On Mon, Aug 02, 2010 at 08:49:43PM -0400, David Kramer wrote:
>>> Long story short, the MythTV mailing list folks pointed out that
>>> AutoExpire could not have done this, and it was more likely my MythWeb
>>> interface was left unprotected, and some script kiddie had some fun
>>> deleting it all.  And they were right.  After some update my .htaccess
>>> file disappeared, and I never noticed I didn't need a password anymore.
>> I don't have an .htaccess file.
>>
>> That's because my MythTV isn't listening to any ports from the
>> outside world. If I want to jigger it remotely, I have to SSH in
>> to my main machine, then tunnel over to the MythTV.

My needs are different.  I can't do that, in part because I use
passphrases so my keys have to be installed on the ssh client machine
(so I can't use any random machine), and because most ports out are
blocked at work, the most likely place for me to need it.

>> If you can afford to have a gateway machine on all the time --
>> and a $99 SheevaPlug only sips about 12W -- I do recommend this
>> approach.

... which would not have done ANYTHING in this scenario.  I keep hearing
people talking about a separate firewall in front of my server that
already has a firewall, but it only adds another layer to slow them
down.  And any port that has to be opened to the outside world (which is
a lot on my server since it does so much) is just being passed through
anyway.

The previous attack on my machine was against several of the dozens of
ridiculously blatant security holes in TWiki.  This one was against a
web page that the security was accidentally removed from.  A firewall in
front of my server wouldn't have done squat other than raise my cooling
and power bills.

> More and more, I believe hiding behind ssh tunnels is the only way to stay
> sane.  Precisely because David is probably a much better sys-admin than me
> (daily snapshots!), and problems like he described are so hard to predict:
> unless you know to look for it, why would you set up cron jobs to watch for
> disappearing .htaccess files?.

Well, I actually did some academic research into this area when I was
working at Aptima, but more importantly, as an Agile Software Engineer I
am into continuous improvement.  Every new thing I learn I can check
for, every time I find an avenue of attack, I adapt to it.

I have a watchdog script that I wrote that runs on another server and
every 20 minutes it would ping my server to see if it was up.  And then
there was a DNS problem and it couldn't handle that, so I added a check
for the domain name lookup first.  Then my database died, so I added a
check to make sure when I go to http://www/thekramers.net I see the
content I expect to see.  You get the idea.

I also run a script I wrote from root's cron called checkhealth.sh.  It
started out very simple (hard drive space, % idle, etc), now it does all
sorts of things, like making sure my TV capture card's device exists and
has the right permissions, because that was a problem for a while.  And
it makes sure my IR transceiver device exists.  And it makes sure that
there aren't zero-byte video files recorded by MythTV.  And it makes
sure the database is running.  Because I learned from each error.

I have another script that does the system snapshots.  But I've learned
what information I need to restore a system over the years, so it does
things like copying content from /proc, a "ls -al" of the entire system,
 database backups, tar/gzip of any mail files that have changed
recently, etc.  I even have it delete snapshots over 14 days old.

I have another script that backs up my server to an external hard drive
as a tar/gzip.  Not everything, because I have learned that backing up
the entire system is futile.  90% of the time I'm going to want to do a
fresh install and then restore my data and configurations on top of it,
and it will be almost as fast with cleaner results, and I can back up my
server that way in about 12GB,  NOTE: I do not advocate this technique
for everyone, but I suggest it as a highly effective compromise for home
users.

All of this has come from learning from experience and adapting to
handle each challenge I learn about.  And you can do that just as well
as I can.