The previous post was written at about 9:30 last night, and ends on an optimistic note. Of course, any hint of optimism demands retribution from the Lords of Karma, so I got a phone call at 9:45 telling me the power had gone out on campus. The power was apaprently off for something like an hour and a half, which is distressingly typical.
Happily, the person who called me was my research student, calling from the lab to tell me that he had shut down the valves on the roughing pumps. This is an important step, because the turbopumps don't automatically re-start, but the roughing pumps do, and if you don't close the valves, they get oil all over the system.
The lasers all went down, but back when I was starting up, I bought fancy power supplies with various safety circuits for $1000 apiece. That's probably saved me replacing the lasers four or five times-- suddenly cutting the current to a diode laser can kill it, as can a power spike when the supply comes back on. The fancy supplies have protection circuits to prevent that, and are worth the money. There have probably been at least fifteen power outages in my time here, and the lasers have come through every one thus far (knock on imitation wood-like desk).
I don't think anything has been permanently damaged, though one of the mechanical roughing pumps was kind of cranky when I re-started it (it stalled out and tripped a breaker, but the second try got it running). The vacuum systems all pumped back down, and the lasers appear to be lasing. None of them will be at the right frequency, of course, but that's relatively easy to fix.
Still, it's incredibly frustrating when this kind of stuff happens. It screwed up my whole plan for the day, as I needed to come in early and get everything turned back on. Not to mention that these repeated outages will shorten the life of the vacuum pumps.
That's life in experimental physics, though...
- Log in to post comments
Yeah, we put all our experiments on various battery backups. Helpful as hell.
Of course, our semiconductor processing equipment would require some EPIC backups, so those we just rely on their built-in interlocks... which still makes my teeth itch.
What is it with universities and the local power? This weekend begins our monthly "maintenance". We could lose power every Sunday this week. It seems that every month there is another potential outage, so one may wonder what good the maintenance is doing....
Yeah, we put all our experiments on various battery backups. Helpful as hell.
The problem here is that when the pwoer goes out, it tends to stay out for an hour or so, and most cheap UPS's are designed for less than that. So you just draw the shut-down out a bit...
I did manage to get them to put the sump pump in the lab on the emergency backup generator, so at least the lab didn't flood this time.
I had just gotten hired at my job, eating lunch uptown (3 blocks away) at Wendys with 3 experimentalists. The lights flickered, they just got up and started walking back to the lab. I wondered what was going on, they were worrying about vacuum systems!
I remember at LBL, we used to have about one power outage a year. I'm not sure about labs -- but I do know that the people who ran the supercomputer center said that every power outage cost them some X number of dollars in equipment that was destroyed, plus the time and effort of getting everthing back up and running again after an unclean shutdown.
The computers I ran also wouldn't fully come up cleanly if they had been shutdown suddenly, until later when I managed to make things more robust. Typically, this sort of thing would happen between Christmas and New Year, so if I was away, it was a problem for everybody....
At Vanderbilt, a couple of years ago, I and some other professors raised a gigantic stink becuase we would receive notice a couple of days in advance that power was going to go down in a coming weekend. That's stopped now, becuase the construction that was driving it is done, but at the time it was maddening. I don't remember if there was ever a case where I was notified of an upcoming outage when I was going to be out of town (at an observatory) the whole time between notification and outage, but there were some close calls where it would have been a disaster for me if the power was off at the wrong time. That they only gave us such short notice made it all much harder. There seems to be this idea that nobody works on weekends.... they clearly don't understand scientists (and academics in general). Especially when we're talking things like telescope schedules.
You'd think that if an institution was really serious about supporting scientific research, they'd be aware that labs depned on continuous power. Yeah, sometimes it happens and can't be prevented, because there is Nature and there are accidents. But you'd think they'd be able to do something to mitigate the threat to the various endeavours that they're supposedly supporting.
I had a buddy who went to a small school, and the chilled water for his laser stopped being chilled around October. Well.......its because Physical Plant plumbed him into the air conditioning system.........He only lost 4-5 months and had to start ove.
Sounds like the Physical Plant folks (or whatever you call them) don't realize they're working for you! If their contracts included reliability standards, with penalties for each failure, I bet you'd see more attention.
"I did manage to get them to put the sump pump in the lab on the emergency backup generator, so at least the lab didn't flood this time."
Is your lab below sea level?