grr..

Day 22. Naturally it would have to happen that Gateway would crash while I’m out of town. I’ve asked my friend Tory to reboot it, and hopefully he will.

Yesterday was a awful day on all fronts. Woke up at 10:00a, went down to the clients site, wrote a disaster recovery plan, and pulled a first-year-sysadmin stupidity stunt.

I was using text mode on a linux console and switching back and forth between about four consoles. I forgot that one of the consoles had been sshed to another host, and when a process hung because of some incorrect settings, I did a killall – on the wrong machine. Unfortunately, the process was a dependancy for the production services run on that machine, and so they went down, leaving 300 people without their database. All of whom were in the same building I was.

We restarted the database (took about 10 minutes) and ostensibly no data was lost. But I still feel awful. THe boss didn’t even yell at me – just said ‘you know what you did..’

Then more awful things happened..

2 Responses to “grr..”

  1. anonymous Says:

    *hug*

    killall considered dangerous

    kill (-9) `ps auxw | grep a_unique_identifier | awk ‘{print $2}’ much safer

    didn’t you teach me that one anyway? Like a decade ago 😉

    I wrote a tone dialer in matlab this week. I think maybe I’m going to go back to programming after all. It was too much fun.

  2. rarkrarkrark Says:

    duh. not logging in considered dangers. The previous comment was from me.

Leave a Reply