Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

What Bizarre IT Setups Have You Seen?

Cliff posted more than 7 years ago | from the truth-is-stranger-than-fiction dept.

Hardware Hacking 874

MicklePickle wonders: "I was talking to a co-worker the other day about the history of our company, (which shall remain nameless), and he started reminiscing about some of the IT hacks that our company did. Like running 10BaseT down a storm water drain to connect two buildings, using a dripping tap to keep the sewerage U-bend full of water in a computer room, (huh?). And some not so strange ones like running SCSI out to 100m, and running a major financial system on a long forgotten computer in a cupboard. I know that there must be a plethora of IT hacks around. What are some you've seen?"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered


Bizarre IT setup seen around the country... (1, Funny)

ShaunC (203807) | more than 7 years ago | (#17454010)

How about this one [shaunc.com]?

Re:Bizarre IT setup seen around the country... (5, Informative)

OECD (639690) | more than 7 years ago | (#17454264)

How about the 10 MB email limit? That seemed to show up in the last 5 years or so. Before that I've had success with almost every size attachment I've been sent (and I do printing, so I see some pretty fat files.) When was the meeting held where they decided that?

Email limits (1)

ShaunC (203807) | more than 7 years ago | (#17454478)

I don't think I've ever hit a limit on single SMTP attachment size unless sending to freebie providers (Hotmail, Yahoo, etc). Although I'll admit that if I want to send a large file to more than one person, I'll usually put it up on a server somewhere and send a link instead of forcing the file upon the recipients; and most of my dealings with emailing insanely large files (sometimes into the gigabyte range) have been in enterprise environments run by competent admins.

I do run into "quota exceeded" from time to time. This happens less and less frequently, though, particularly with even the free services constantly trying to one-up each other. My Gmail account says I've got 2801 megs of storage, which is far more than I'll ever use, but I imagine they'd give it to me if someone sent me that much.

You've got me trying to tease a long-dead part of my brain that wants to remember what the upper boundaries were on AOL back in the day, 1994 or 1995. They implemented quotas in a unique manner. I want to say that your mailbox could hold a maximum of 550 messages, 1100 if you worked for them, regardless of the message size. Their attachment limit was 16 megs, I think; this would have been nearly unheard of at the time with the exception of all the warez traffic.

Anyone out there remember the details more clearly than I do? We're talking 12 years ago, and sometimes I'm fortunate to remember 12 days ago...

the U-Bend (5, Informative)

Helix150 (177049) | more than 7 years ago | (#17454028)

Just to clarify- the U-Bend is what prevents bathrooms and drains from smelling horrible. Inside the drain, shower water, sink water and toilet waste all mix together. As you can imagine this smells horrible. So, where every toilet, sink, shower, etc connects to the drain system there is a 'u-bend'- a downward dip in the pipe which stays full of water. This prevents air from flowing out of the empty drain.
Most sinks have their u-bend visible under the sink and look like this:
http://twenteenthcentury.com/uologos/ubend_shaded. png [twenteenthcentury.com]
Water flows in the top, and out the back. Because the back is higher than the bottom of the bend, the bottom stays full of water at all times, preventing air from passing.

Problem is, if you leave a drain long enough without water passing through it, the water in the u-bend can evaporate, leaving an empty pipe and allowign the nasty sewer smell to escape. Thus, leave a faucet dripping to keep the U-Bend full!

Re:the U-Bend (3, Insightful)

Ninjaesque One (902204) | more than 7 years ago | (#17454056)

I would think that running the water for 5 minutes while using it(exempli grata:to wash greasy face after long day at work) would cost less than leaving it dripping.

But I guess you guys aren't responsible for utility bills and stuff.

Re:the U-Bend (3, Informative)

Helix150 (177049) | more than 7 years ago | (#17454066)

running the water at any sort of regular interval will keep the u-bend full. For the U-bend to evaporate would take weeks or months probably. Even the slightest drip should more than counter the evaporation. And it probably seemed like a better solution than a server room which stank up every month for no apparent reason :)

Re:the U-Bend (1)

Pig Hogger (10379) | more than 7 years ago | (#17454484)

running the water at any sort of regular interval will keep the u-bend full. For the U-bend to evaporate would take weeks or months probably.
Some decades ago, I shared an appartment in Paris with a total technical ignoramus (a philosophy professor). When I arrived, I found that the bathroom reeked. Upon investigation, the smell came from a washing machine drain which was not used (no washing machine), and hastily plugged with balls of crumpled paper and a totally ineffective piece of plastic held by an elastic.

Removing the blockage and pouring several glass of water in the drain did the trick...

Re:the U-Bend AKA trap (1)

darkonc (47285) | more than 7 years ago | (#17454112)

I've generally known that contraption as the 'trap' -- although I recognized the descriptive name 'u-bend',
I'm just wondering if this wasn't the same drain that they were using to run their ethernet connection, That would explain why that sink was 'never' used.

Re:the U-Bend (5, Interesting)

Animats (122034) | more than 7 years ago | (#17454294)

Yeah, and it's a huge hassle in vacant commercial buildings. Somebody needs to run every water tap and flush every toilet about once a month, or the whole place will stink up. Then the smell gets into the carpeting, which makes it hard to rent the building.

For special situations, there are calibrated drip valves. These are often found as part of fire sprinkler systems, which usually have a drain valve for when you need to drain the system for maintenance. The water from the drain valve has to go somewhere, which usually means a sewer connection. But you can't hook a water line to a sewer line; there are situations when you'd suck sewerage into the water system. So there has to be a vacuum break open to air. After the vacuum break, there's a U-trap with water to keep sewer gas inside. But since such drains are seldom used, the water will evaporate. So a tiny bit of water has to be dripped into the drain to keep up with evaporation. There are special "drip valves" for this.

One of the things you need to know about if you run large data centers.

Re:the U-Bend (4, Interesting)

Pig Hogger (10379) | more than 7 years ago | (#17454436)

So a tiny bit of water has to be dripped into the drain to keep up with evaporation. There are special "drip valves" for this.
I was surprised to see, in a large warehouse store, some automatic urinal flush reservoirs (they flush the toilets every so often) whose output was hooked to about 30 small half-inch pipes going into the floor. The reservoirs were installed about 20 feet high on columns.

Some amount of cogitation was needed to realize that each of those small pipes was headed to the traps of the floor drains installed throughout the store...

Now that's a (plumbing) hack in the true meaning of the term!

Re:the U-Bend (2, Informative)

TykeClone (668449) | more than 7 years ago | (#17454650)

Fill the trap with cooking oil - it will stop the smell and will not evaporate as quickly as water would.

the L-Bend (2, Funny)

Anonymous Coward | more than 7 years ago | (#17454846)

"Fill the trap with cooking oil - it will stop the smell and will not evaporate as quickly as water would."

A good example of lateral thinking [wikipedia.org]. Good thing they're cloning you. :)

Re:the U-Bend (0)

Anonymous Coward | more than 7 years ago | (#17454296)

How come when you remove that "U-bend" piece of pipe to find lost items you're not assaulted by stench?

Re:the U-Bend (1)

djh101010 (656795) | more than 7 years ago | (#17454354)

How come when you remove that "U-bend" piece of pipe to find lost items you're not assaulted by stench?

Oh, you are. But why post this as AC?

Re:the U-Bend (3, Interesting)

senaattori (730352) | more than 7 years ago | (#17454622)

I pour vegetable oil into the u-bend of our server room's sewerage. Vegetable oil itself doesn't evaporate very quickly. As it floats on top of water, it forms a layer which prevent's the water from evaporating.

Server room heating (2, Informative)

dtfinch (661405) | more than 7 years ago | (#17454034)

Company moved into a new, larger building. The server room had a heating vent leading into it, and no A/C. They solved it by clogging the vent with a bag full of shredded paper and cutting a hole in the wall to install a small consumer single-room air conditioner.

Re:Server room heating (1)

lthown (737539) | more than 7 years ago | (#17454194)

I remember reading a few years ago about someone using one of those but it condensed vapor out into a garbage can beneath the AC unit. One day someone moved the can and spilled water all over the servers underneath it.

Re:Server room heating & worker Safety (5, Interesting)

darkonc (47285) | more than 7 years ago | (#17454200)

At UBC we had a tiny (10'x10' computer room with a number of (un)pleasantly heat-generating computers (a couple of SUNs and a stack of SGI's). We managed to get the extra wiring put in place to handle the machines (a number of which required a 20AMP plug), but we never managed to get extra AC installed. This didn't bother us until summer came. ... and the build-up of heat would occasionaly trip the thermal breakers in some of the machines.

After begging facilities since the previous year to upgrade the AC (and having one last big machine installed), we 'solved' the problem by buying a small, window-type AC, and poking it out the door. With this setup, we could generally get the room to stablize at around 30C (about 86F).

This worked until facilities showed up and complained that we needed to go through them to get any sort of AC installed, and demanding that we stop using the offending unit. (but required us to continue with the un-responsive process of getting the room AC upgraded).

Peter resolved the impass by calling the health and safety group, and keeping the door closed until they arrived the next morning to inspect a worksite with a temperature of over 100F.
The AC was upgraded in well under a week.

Servers stacked (0)

Anonymous Coward | more than 7 years ago | (#17454074)

Rur datacenter has plenty of servers that are not on rails, which is bad enough in itself but what amazed me is seeing 4 or 5 2U servers resting upon the following.... 2 old SCSI drives, a Solaris User Guide book and some other manuals that were laying around. No one knows how this setup came about since it was before all our time at the company.

I've managed to use some industrial strength velcro as an additional set of hands.

Nothing THAT bad... (3, Funny)

thepropain (851312) | more than 7 years ago | (#17454092)

Two worst I've seen: 1) While I've chopped patch patch cables in half and turned 'em into crossovers, this one place I toured got a good deal on pre-made crossovers and chopped & spliced them into patch cables for over 50 PCs; 2) Where I work now, a former employee jacked a cable modem straight into a Win9x peer-to-peer network, despite my protests (scary part of that was that he said, and I quote, "Oh, I do this all the time and it's never been a problem before." I spent the next week reinstalling Win98 and software...)

Re:Nothing THAT bad... (2, Insightful)

b0s0z0ku (752509) | more than 7 years ago | (#17454824)

While I've chopped patch patch cables in half and turned 'em into crossovers, this one place I toured got a good deal on pre-made crossovers and chopped & spliced them into patch cables for over 50 PCs;

Why not just hack off the ends and crimp new ends onto one end? Once you've done a few, this should take less time than splicing wires together and insulating the connections? And ends are literally a dollar a dozen if you get them in bulk.


Re:Nothing THAT bad... (2, Interesting)

Blkdeath (530393) | more than 7 years ago | (#17454864)

When I started working at a local high school administering their network I found an amalgamation of two networks. One running primarily thinnet (10Base2) through the classrooms uplinked to 10BaseT hubs (three 16 port stackables) which were each connected to a 10BaseT switch ("the core"). The server ran Novell, the PCs ran a combination of DOS and Windows 3.11 which ... worked.

Now the new network installed right beside it consisted of a mighty IBM NetFinity 5500 server with a RAID 5 array of about 50GB and a plethora of IBM switching equipment. The core switch wasn't high-tech or anything, but atleast it had 100MBit fibre running to the IDFs which ran switched 10BaseTX to the workstations. Now, this was all fine and dandy and wired real pretty-like by people who had no comprehension that the labs running PII-400s would most likely place more demand on the network than the labs running 486DX4s, so we had to re-wire the thing to balance the loads somewhat. :)

Shortly thereafter we installed a Linux server to handle DNS and HTTP cacheing for the 128KBit ISDN connection to the Internet (real practical for a network of some 400+ workstations eh?), revamped the configs on the workstations, re-configured the network from the core on outward, re-wrote the network wiring diagrams manually (they were, apparently, somewhat classified "need to know" information and we as system administrators did not "need to know") and generally made the place hospitable.

The network where I'm currently working (along with the phone system) was apparently installed by a monkey. All 19" rack-mountable equipment with its rack mount hardware installed in such a way as to be able to bolt them flat alongside three inner walls of a closet. There's a nice Panduit patch panel there - with about 4' of patch cables tempting their connections by gravity; they just sortof hang there in a loop before connecting to the Cisco switch installed above it. Not so much as a zip tie in sight!

There's a 3Com 16-port switch in there that was powered and creating plenty of heat and noise; but the strange thing is it's not connected to anything but the AC outlet. (Yes, it's now unplugged, but still hanging there all useless-like). I also find myself at a loss to explain why, with a single ADSL connection to the building, we require three (yes, three (3!)) DSL modems. Or why, when there were 2 spare electrical outlets even before I unplugged the 3COm someone felt the need to connect one of the devices to an extension cord running out the closet and 5' along the wall.

The network drops consist of a motley combination of mis-labelled jacks and broken wall mounts compensated for by the random installation of cheap hubs and duct-taped CAT5 cables running helter skelter around the place.

The network is so shaky it's not possible to install a centralized high capacity network printer as of now because, well, too many print jobs and something could catch fire in that closet. I can't WAIT to write up a cost benefeit analysis for my boss to justify the disposal of the dozen or so laser printers installed on various desktops around the place. :)

Oh, and these aren't mine [snerk.org], but they make me feel better about my own situation whenever I look at them.

Oh, anywhere.. (-1, Offtopic)

Anonymous Coward | more than 7 years ago | (#17454114)

That uses Windows in any way..

Captcha: Redneck

300 wires with a conduit sawed off (4, Interesting)

plover (150551) | more than 7 years ago | (#17454120)

Back in the '80s we had a brand new computer room that had 300 shielded twisted pairs heading to 300 distant stations. The entire place was shiny, painted white, beautifully installed, all running through three large plastic conduits, one to each floor, hung professionally from the ceiling. A textbook illustration of beautiful wiring.

The fire marshall came in and said "you can't have those low-voltage wires run through that conduit, that conduit is designed for high voltage wiring." So the electricians came in and sawed off their beautiful conduits, leaving the wires draped between the four-foot-spaced supports. They tie-wrapped the bundles every foot or two, but it still looked like a dead python hanging between branches.

To this day I still can't fathom what the hell that inspector was thinking.

Re:300 wires with a conduit sawed off (1)

oneiros27 (46144) | more than 7 years ago | (#17454324)

That's okay ... at least in the fire marshall's case, he isn't expected to actually know anything about networking.

I worked in a large university datacenter, and we were waiting for a box to come in ... the techs who ran the cables were kind enough to pull me a service loop, and punch down a 1U panel for me, so that as soon as the rack finally showed up, I could just bolt it down, and run with it. The rack finally shows up ... I take off the little plate in the back, run the network inside so it's nice and neat, bolt everything down, and we're up and running ...

But the director is unhappy that we didn't use the hole with a grommet in it that Sun so kindly leaves for you ... of course, you can't thread the patch panel through it, so he made the techs cut all of the pairs, fish the cable through the damned hole, and re-punch everything so it looked pretty.

Of course, all of us knew that the rack was scheduled to move two rack spaces over with the consoldiation that were doing (further from where the cables were pulled from), so it was going to have to be cut in a month or two, so we could move the rack (which wouldn't have to have been done if we had just left it run the way we had originally planned).

300 wires with a conduit sawed off-cutups. (1)

Anonymous Coward | more than 7 years ago | (#17454600)

You do know that that grommet is to keep the sharp edges of the case from cutting into the cable?

Re:300 wires with a conduit sawed off-cutups. (1)

redcane (604255) | more than 7 years ago | (#17454732)

Only useful if the cable runs where it could be in contact with said sharp edges. I imagine the hole they put the patch panel in through is far enough from sharp edges.

Re:300 wires with a conduit sawed off (1)

mrchaotica (681592) | more than 7 years ago | (#17454652)

The fire marshall came in and said "you can't have those low-voltage wires run through that conduit, that conduit is designed for high voltage wiring."
That's okay ... at least in the fire marshall's case, he isn't expected to actually know anything about networking.

Okay, but can't he expected to know a tiny bit about voltage? Or even English vocabulary? Namely, that low voltage is less than high voltage?

Re:300 wires with a conduit sawed off (5, Insightful)

ars (79600) | more than 7 years ago | (#17454798)

You have it backwards.

The problem is not the low voltage cable - it's that since it's meant for high voltage cable someone could install some later not realizing that there is some low voltage stuff there.

Yes, there really is a code about that - not mixing high and low voltage in the same conduit.

You can, I guess, claim that the conduit is "low voltage". But if it looks like it's for high voltage you might not get away with that.

Re:300 wires with a conduit sawed off (4, Funny)

AJWM (19027) | more than 7 years ago | (#17454890)

The fire marshall was (giving the benefit of the doubt) probably thinking that if there was a high voltage conduit, sooner or later somebody would run a high voltage cable through it. Can't have high and low voltage wiring in the same conduit.

(Of course the reasons for all this are probably lost in the mists of time going back to fabric-insulated wires hung on insulators nailed to the studs. You'd think with modern wiring with obvious differences between 12 ga high voltage cable and cat-5e wires it wouldn't matter ... but then I've seen some pretty bizarre wiring setups that were "just temporary" or quick hacks, I can just see somebody provide a whole new meaning to "power over ethernet".)

Sodomized service (1)

b0s0z0ku (752509) | more than 7 years ago | (#17454792)

The fire marshall came in and said "you can't have those low-voltage wires run through that conduit, that conduit is designed for high voltage wiring."

Better than what a certain electrician did in an office that was being renovated. The Ethernet wiring was already installed. He decided, for some reason known only to himself, to pull BX cable (the heavy, metal-armored electrical cable) through the same conduits and cable grommets as the Ethernet cabling. Sans lube. Needless to say that a lot of new Ethernet cable got pulled to replace the sliced up cables! (He also had actually removed some cable grommets where the cabling went through studs to make more room for the BX cable since he was too freakin' lazy to drill extra holes!)


Re:300 wires with a conduit sawed off (1)

afidel (530433) | more than 7 years ago | (#17454928)

That's weird because UL rating and the national electric code always allow a higher class device or fixture the be substituted for a lower class one (with the obvious exception of breakers and such, I'm talking about things like plenum rated stuff can be used in normal wiring situations)

this is a classic! (-1, Troll)

Anonymous Coward | more than 7 years ago | (#17454128)

I used to intern at the geek compound. There was a lot of hardware...some of it sent to review, which we never did, some of it from VA Linux, some of it from UMich computer sales... a real manegarie of cutting edge and classic hardware. Since it was free/low cost and there was new stuff showing up weekly, we went crazy rigging up weird interfaces. I remember one day, I walked in and found (Cowboy)Neal and (Cmdr)Taco fucking Hemos up the butt. They just grinned and asked if I wanted to go next. I quit that very day, but I got to keep an Alpha AXP box. I still use it to run linux.

Dungeon radio (5, Interesting)

Centurix (249778) | more than 7 years ago | (#17454148)

I worked at one place where our room was a couple of floors underground (very depressing place) and we wanted to listen to the cricket on the radio (pre internet days). Armed with a crappy radio we found we could get perfect reception by connecting to the air conditioning vents with a set of crocodile clips purchased from Tandy's.

Another one I remember is something low-tec invented by some admin staff, we had a policy set in place that locked workstations after 5 minutes of activity, the PC's were severely locked down so you couldn't change this. Turned out the admin section of the company despised this as they would do something on their accounts package, talk to someone on the phone and by the time the phone call had ended the PC had locked itself requiring their password to unlock it. One lady actually took a small clock, took the plastic front off and attached a piece of paper to the second hand, when she wasn't doing anything, she placed the mouse in front of the clock so that when the second hand went past, it moved the mouse slightly stopping it from locking. When the guys in tech support found it, she was visited by practically every IT person just to see it in action.

Re:Dungeon radio (5, Funny)

ShaunC (203807) | more than 7 years ago | (#17454276)

we had a policy set in place that locked workstations after 5 minutes of activity
And the PHB's wondered why productivity was in the toilet... :)

Re:Dungeon radio (1)

Centurix (249778) | more than 7 years ago | (#17454868)

I need something to lock my brain up after 5 minutes of activity, it's obviously on a course of its own after that... :)

TV in disguise (5, Interesting)

smurfsurf (892933) | more than 7 years ago | (#17454510)

I and a few guys were doing customer phone support in a remote building (ten years ago or such some). Soccer euro cup was up, and a collegue was desperate to find a way to watch the games, as the company (ISP) has just started operation, and callers were few and knowledable (so it was actually fun). Opening the cable funnel, he saw a TV cable. He spliced it up and connected it to a RJ45 jack. He then installed a TV tuner card into his PC, build a network cable look-alike to connect the TV card to the fake network jack, and voila - you could not see he was tapping the TV signal (the cable funnel was very visible, the computer was under the desk).

As we left the building about a year later, the fake jack was left there. I wonder what kind of head scratching this caused for the future tenants :-)

Coat Hangers (5, Interesting)

tymbow (725036) | more than 7 years ago | (#17454150)

I've seen untwisted coat hangers covered in electrical tape and twisted together used to supply AC between two buildings in tropical weather in PNG. The wiring to the main building was bad enough but using coat hangers to supply power to the small hut that housed the computer equipment was priceless. I should also point out that they did not have power outlets for the computers either. They just cut the plugs off, stripped the wires, twisted them together and covered it in electrical tape.

Router at the end of a pier (3, Funny)

jgaynor (205453) | more than 7 years ago | (#17454216)

My previous job was in Network Operations at a University. Our Marine Science department had a large grant-funded sensor network running in a river somewhere in South Jersey that needed to talk to their machines on campus. They did this by getting a 56k leased line dropped out to the end of a long pier, to which they connected a cisco 2500 series router (state of the art at the time). It was housed in a box with just enough ventilation to keep it soaked in condensation, but not enough to allow for adequate cooling. Because of the heat it was on a permanent shutdown/reboot loop for most of spring, summer and early fall. They were lucky if they got more than a few hours of readings per day.

Re:Router at the end of a pier (3, Informative)

afidel (530433) | more than 7 years ago | (#17454960)

Wonder if that was before Cisco offered extended range products. These days you can get gear from Cisco that will survive in the Sahara in a NEMA3(sealed) outdoor enclosure.

OOOoooo (1)

Anonymous Coward | more than 7 years ago | (#17454242)

Probably can't guess why this is anon...

Worst running system: Exchange 4, 5, and 5.5 running in a single Exchange site for a former company. Needless to say, there were some issues with their email.

Most bizarre development: insistance that a newly designed and developed java system use CORBA to connect to ... a Java system. Last time I checked, Java was supposedly this network programming language and has these really poor supporting libraries for distributed computing....

Most? bizarre configurations:
Use of a second CORBA stack in an appserver that comes with one.
Using Access for a multi-user critical application
Using Excel as the budgeting and financial tracking tool for a multi-billion corportation
Using a third party POS layer built on a RDBMS as a RDBMS.
On an essentially synchronous SOA implementation, the request/response pattern included the request within the response. One of many inefficiencies in this design.

...it really is the answer (4, Funny)

techno-vampire (666512) | more than 7 years ago | (#17454248)

About ten years ago, I was working for what was then a small, startup ISP doing tech support. For about the first two years I was there, we often had to talk new customers through locking down their modems to 2400 baud in the registration/installation program, because that server often worked best at low speeds. (We also showed them how to reset it to the proper speed afterwards because our POPs were just fine.) I later found out that this was because whoever set up our one and only (at that time) registration server had multiplexed 42 modems through one COM port.

Seal it up (4, Funny)

crossmr (957846) | more than 7 years ago | (#17454254)

I had an instructor who used to work in industry. He'd told me about a company he was consulting for. They had a Novell box that they administered remotely. During some remodeling, the small closet/room it was in was sealed with drywall. It was 4 years before the box required maintenance and someone went about trying to find it and realized what had happened.

Re:Seal it up (4, Informative)

Scarletdown (886459) | more than 7 years ago | (#17454442)

During some remodeling, the small closet/room it was in was sealed with drywall. It was 4 years before the box required maintenance and someone went about trying to find it and realized what had happened.

http://www.techweb.com/wire/story/TWB20010409S0012 [techweb.com]

Re:Seal it up (2, Informative)

crossmr (957846) | more than 7 years ago | (#17454490)

Not the same place, so not the first time it happened. I don't think they had to call in Novell to find it.

Re:Seal it up (0)

Anonymous Coward | more than 7 years ago | (#17454904)

People hated Netware, but it did the job and it did it without needing any attention for years and years. Of course the problem was that when it did need something to be fixed you had already forgotten how to administer the damn thing...

http://bash.org/?5273 (0)

Anonymous Coward | more than 7 years ago | (#17455004)

Obligatory bash.org quote.

<erno> hm. I've lost a machine.. literally _lost_. it responds to ping, it works completely, I just can't figure out where in my apartment it is.

I hack, therefore I am (3, Interesting)

tverbeek (457094) | more than 7 years ago | (#17454260)

There was the time back in the late 1980s when the multiplexed underground cable that my college was using to provide terminal connectivity for their new-fangled computerized class registration stopped working, and they ran dozens of hastily-created RS-232 cables from the data center to the hall where the action was, half a block away... secured to the sidewalk by duct tape, of course. Which at least didn't remain in place as long as the 10base2 cable that connected two dorms, strung between their 2nd floors (until it was taken out by a lightning strike).

More recent ugly hacks that I can claim personal credit/blame for are mostly of the sort that involve pulling a rabbit out of my ass because a solution needs to be found By Tomorrow Morning... like for deploying 200 installations of Windows 95 in a week (in the days before Ghost, or even backup software that preserved Long FileNames) using DOS boot diskettes, Netware, a utility called lfnbk, and ncopy... or building an e-mail server out of RedHat Linux 6 and spare parts (no, I didn't even have a complete working computer at my disposal) when the company's glorified BBS mail software found itself unable to exchange mail with the standards-compliant system used by a major new business partner.

Re:I hack, therefore I am (1)

robpoe (578975) | more than 7 years ago | (#17454370)

I used to create Netware 3.x servers with a DOS boot disk with the netware netx drivers, fdisk and format ...

boot, fdisk make 20 meg part, reboot, format 20 meg part, login to the server, copy the c:\netware directory to the C: drive, reboot, load server.exe (the lan drivers were in the c:\netware directory, as were most of the (at the time) recent SCSI drivers I needed..

Then once the server comes up, create SYS: (which made the bindery), hit it from remote and copy in the SYS: stuff. Took longer for me to type it than it did to create it ;)

Analog e-mail. (5, Funny)

Anonymous Coward | more than 7 years ago | (#17454292)

When I got hired as an Information Specialist for one of the government sponsored agencies in Hellinois, the people there would write their e-mails on a piece of paper and give those to their previous IT guy. He would then type them up and send them out via a yahoo e-mail. No kidding.

long ping next door (4, Funny)

darkonc (47285) | more than 7 years ago | (#17454300)

We shared our internet with the small ISP who sublet a portion of the building from us. They were upgrading their connection to the backbone from a T1 to a microwave link (gives you an idea as to how long ago this was).

At one point, they had changed their routing so that they were using the new link but we hadn't, so we decided to see how a ping went.

A packet between the two machines would go through our router, over the ethernet that the two companies shared, out the (old) external router, and down the coast through Seattle, to California, then back up the coast to Vancouver, and then finally over the same shared ethernet cable that the packet had originally gone out before finally connecting to their router.

A cross-border round trip of a few thousand miles for a net distance of about 60 feet.

Oh, and did I mention that our server room was a converted bank vault?

Re:long ping next door (2, Funny)

techno-vampire (666512) | more than 7 years ago | (#17454464)

I remember when (at the ISP I did tech suppport for) a traceroute from our office in Pasadena to Caltech would take 11 hops, 4 of them in the midwest because our backbone supplier routed everything through their main datacenter. It didn't take long for us to find a different backbone suppliier!

some totally wicked "weird" security (1, Funny)

ILuvRamen (1026668) | more than 7 years ago | (#17454312)

I helped design a setup that has an insane security system that's unheard of. It's trap based, as it everything I do. For instance, there's a keyboard sitting on top the desk and a keyboard and if any key is pressed on it, an alarm goes off. The real keyboard is on a pullout tray beneath that and all employees allowed to use the system are told that. If anyone sits down and doesn't know that, they don't get to use the computer. Simple yet effective. There's also some other absolutely 100% hacker proof crazy things in the setup that haven't been used anywhere else before because I invented them but I obviously can't reveal exactly how they work because that's part of why they work. But trust me, it's probably the most secure setup in the world and any hacker or person that broke in intending to do any sort of digital harm would end up confused and arrested no matter how skilled they were.

Re:some totally wicked "weird" security (-1, Offtopic)

Anonymous Coward | more than 7 years ago | (#17454772)


Re:some totally wicked "weird" security (1, Insightful)

Trillian_1138 (221423) | more than 7 years ago | (#17455022)

As I'd hope you know, security through obscurity (which it *sounds* like you have, at least in part) is not in-and-of-itself bad. But I hope you also have security systems which don't rely on people simply not knowing how they work, as that is bad policy. A combination of the two is gonna be your best bet.


Honorable Mention (5, Informative)

ShaunC (203807) | more than 7 years ago | (#17454348)

I know the poster was looking for funny/interesting anecdotes directly from our community, but for those of you who haven't stumbled across The Daily WTF [thedailywtf.com], hop on over to that site and make it a part of your daily reading. While the focus used to be mostly on programming, it's abstracted itself to the generic IT level in recent months, and you'll see all sorts of bizarre stories there.

The Daily WTF is to IT workers what Jerry Springer is to everyone else. Just when you think you're having a bad day and your life is in the crapper, you can take a few minutes to soak in a situation where somebody else has it much, much worse... :)

ISP I swear (1)

zakeria (1031430) | more than 7 years ago | (#17454350)

I worked for an ISP that ran everything on one single SUN Solaris machine i kid you not! and it was so old when it rebooted it took over 30 mins to come back online thats if you where lucky sometimes it never came back. The poor thing just wanted to die and stay dead.. The funny thing is six years on and they still use the thing, but have over 20 servers now running linux.. not sure if thats saying something bad or good lol

Phone cables from modems (5, Funny)

lexarius (560925) | more than 7 years ago | (#17454364)

I wanted to try out the option to have the server page me in case of problems. Only problem was that the only phone jack in the server room was on the other side of the room, and I didn't have a phone cable nearly that long. But I did have a box of old ISA modems and short phone cables. My intuition told me that the "Line In" ports were wired directly to the "Phone" ports and didn't require power or actual computers to drive them. So I daisy chained modem cards and short cables together across the ceiling, wedging the actual cards behind cable housing and drop ceiling tiles, until finally I got dialtone. My supervisor commended me for my creativity but made me take it down, since the policy was that the modems were not to be connected to phone lines for fear of people being able to dial in to them or something. Never mind the dedicated internet connection.

Re:Phone cables from modems (4, Interesting)

Anml4ixoye (264762) | more than 7 years ago | (#17454856)

You reminded me of the time our network admin wanted to setup a failover for our main (high-traffic) website. He figured that he could just add the IP address of our off-site emergency server as a third entry in DNS, since (at least to him) DNS worked by always hitting the first IP, and only moving down the list if it couldn't hit the previous one.

Only it doesn't. It round-robined the requests, so 1/3rd of our traffic was immediately and swiftly rerouted to our emergency site, which some enterprising webmaster had setup to email the webmaster box if anyone hit it (to make sure, I guess, that no one was going to it).

We noticed it because 5 emails came in at once, and then 10 more, and then it didn't stop until Groupwise crashed. We lost all the email in the box, and emails were coming in at some insane rate. We figured it out maybe 3 minutes in, but by the time we logged in and made the change, it was way too late.

Personal favorite. (1)

GrueMaster (579195) | more than 7 years ago | (#17454372)

Back in the day of Atari ST computers, a friend of mine purchased several boxes of 5, 10, 20, and 40 meg hard drives from a computer store bankrupcy auction. He then build a wood shelving unit that mounted to the wall over his computer desk, and set all the drives on the shelves, using cereal boxes as separators. He then wired them up to scsi controllers, 6 drives per. On one end, he had a set of lighted low voltage toggle switches attached to low and high voltage regulators. It looked like an old computer from the 50's. Each drive had a label, and if he wanted something off a particular drive, he'd flip several toggle switches, throw the main power switch, and boot up. The entire wall would vibrate with the drives spinning. It was great to watch. A couple of drives would not spin up sometimes, so he'd pick them up and shake them until they started spinning.

Another one of my favorites was this database system running in the catalog sales office of an art gallery. I had taken them on as clients, fixing bugs in this database, and working on a migration path to Foxpro. One day I got a call that they had garbage data for the two years prior, when they went to review a customer's history. Turned out the hard drive was full, but the database system was happily writing new data over old files. Fortunately, I had all the data on tape as part of my development efforts.

Re:Personal favorite. (-1, Flamebait)

Anonymous Coward | more than 7 years ago | (#17454530)

What a shame we can't moderate +1 'Bullshit'.

Re:Personal favorite. (1)

GrueMaster (579195) | more than 7 years ago | (#17454988)

Why do you say that? It is true. Unfortunately, I didn't have a digital camera back in 1990, otherwise I would have taken a picture of his setup. I still have one of the drives he gave me, a Microscience 40M 5.25 RLL drive, hooked up to an adaptec RLLSCSI controller. And yes, I have to shake it to get it to spin up.

If you are refering to the other story, I still have the backup tapes, and the database program in question was marketed by Tandy Computers, called filePro 16. Look it up, you pathetic welp.

U.S. Navy's NMCI (1, Insightful)

Anonymous Coward | more than 7 years ago | (#17454374)

Try this:

  • Determine your network needs to be worked over
  • Spend millions before realizing its a big job
  • Ignore you in house staff and bring in a contract org whose primary focus is milking the money out of Uncle Sam
  • Accept decreasing deliverables from the contractor as they raise the price
  • Ignore your user complaints and jimmy the surveys to show 98% satisfaction
  • Deal with security issues created by the contract org and their sub-contractors
  • Accept more decreases in deliverables
  • Consider turning over more of your network to the contract org
  • Have your legacy staff cover all the local complaints generated by SNAFUs created by the contract org

My direct experience... (4, Funny)

rah1420 (234198) | more than 7 years ago | (#17454376)

I was the "computer guy" at a fabric processor in a town in Eastern PA that Shall Remain Nameless. Being "The computer guy" meant that they blamed me for the outages, but of course gave me no spending authority to do anything to fix the problems...

About 1 month into the gig, I was in the front office which was connected to the computer room by fiber optic cable (probably the smartest thing this company did.) However, once the fiber terminated at the switch in the office, the horizontal wiring to the workstations was, God help me, silver satin cable. Telephone wire. The shit was everywhere. There were about 100 workstations salted through the plant (which ran high voltage AC and heaters and whatnot) and everyone complained about the server performance. I wasn't even allowed (!) to put a network analyzer on the wire and was too naive/stupid at the time to realize what the problem was. The guy who had the spend authority, the "chief engineer," told me the problem was lack of RAM in the server and was always harping on me to upgrade the memory.

Another time I opened a closet to find a splice of this satin cable (they must have bought it surplus, they had hundreds of reels of the stuff) and the splice was made with, I kid you not, wire nuts.

I lasted 18 months there. I heard they brought an ex-Accenture conslutant in soon after to fix the "computer problems" and she ran the company into the ground.

Re:My direct experience... (1)

b0s0z0ku (752509) | more than 7 years ago | (#17454914)

, the horizontal wiring to the workstations was, God help me, silver satin cable. Telephone wire.

I remember some sort of proprietary network setup that used small boxes hooked up to the parallel ports to share printers and files over phone cable. This was in a NYC museum in 1998 or so. Slow as hell and difficult to find parts for. I was 18 at the time and really didn't know better, so I somehow obtained parts inexpensively to extend the network when what I really *should* have done is junked the setup and gone to Ethernet over Cat 5. Oh well, they still paid me quite a bit for a weekend's work fixing their other computer problems, and they were happy, so no one's really complaining...


Unsynchronized air conditioners (4, Informative)

Animats (122034) | more than 7 years ago | (#17454406)

Many computer rooms have packaged units which both heat and cool, and some also both humidify and dehumidify. That's fine if you only have one. If you have more than one, they need to be interlocked so you don't get one cooling while another is heating, or one humidifying while another is dehumidifying. If you get into that situation, everything will seem to be just fine, but your energy bills will be maybe 5x what they should be.

Saw that situation in a server room at Stanford a few years ago.

Stupid IT maneuvers (4, Interesting)

ximenes (10) | more than 7 years ago | (#17454426)

I have a bunch of stupid cobbled together setups to talk about. It all comes from a combination of poor IT staff at university wages, infintessimal budgets and the overbearing institutional and faculty pressures.

1. A "server room" that was essentially the most worthless room in the entire building, a long skinny room with four windows (perfect for keeping an uneven temperature!). Rather than buy 19" racks or even wire racks, they found a bunch of tables and put one server on each all the way around the edge of the room.
1.a. All of the servers were in fact desktop systems; an Ultra 1 was the mail server, a SPARCstation 5 the print server, a Gateway Pentium Pro 200 desktop the web server, etc.

2. A lab had to be moved one room over, because its current location was deemed too valuable. The original room was designed for a lab, it had 20+ fiber optic networking ports, twist-lock power connections in the ceiling, that sort of thing. The new room had two electrical outlets, no dropped ceiling, and one fiber optic networking port. It had previously been used as a copy room/storage closet. The cost to move the fiber optic wiring (just one room over mind you!) was over $25,000.

So instead, I had the great idea to cut a hole in the common wall (above the drop ceiling line), purchase additional ceiling tiles and cut up 2x4's into wooden supports. The original ceiling boxes containing the networking were put on top of the blocks above the new tiles, and extension cables run through the wall into the new room. In the original room, which was turned into a lounge, you couldn't tell that there was anything funny going on.

The best part is that the lab manager, who insisted they needed every single network port, never used a single one of them in the new room. All of those cables now reside in a box marked "Giant waste of money".

3. The main Windows file server was purchased in 2002 and has an internal RAID (bad idea in my opinion). What was huge then is worthless now; 5 disks that total 135GB. To get more space, the administration begged for a single external 250GB USB drive to host all user data. Nevermind that there is no redundancy, that an external drive is more suspectible to theft or failure, and that USB is unnecessarily slowing things down.

4. A system administrator got it into his head that rackmounting was the way to go (I agree). So he begged for a 19" rack to be ordered, and placed all of his servers into it. Except he doesn't have a single rack mountable server, and he didn't get the rails for any of the cases either. So now he has one $500 rack, and 8 $100 shelves to go in it. Same guy also switched the KVM monitor to a 15" LCD that doesn't support the resolutions of 9 out of 10 systems connected to it.

5. A consultant was brought in to tell us what needed to be done with the computing infrastructure (what DOESN'T need to be done is more the question). His main suggestion was to set up a central backup service just for this college, so as to avoid paying the central university IT group fees to use their central service. OK, thats an idea I guess... except that he wanted us to buy this: http://www.sun.com/storagetek/tape_storage/tape_li braries/sl8500/ [sun.com] (its $200,000). Luckily this one didn't actually come to pass.

Basically every day is a new adventure in ridiculous IT methodology.

Re:Stupid IT maneuvers (1)

ximenes (10) | more than 7 years ago | (#17454460)

I totally forgot one of the best things. We used to call this "Ozsoyogluing" a machine, based on the person responsible's name.

Take 5 workstations spread in various offices. Each one has a 5-10GB drive (this was 5-6 years ago) and runs Solaris. Each one has a few gigs of space left over after the OS install, and since you can't possibly let that go to waste, you have the other four NFS mount from it. Repeat for each workstation.

You cannot boot a Solaris system (of this vintage) with NFS mounts if those mounts are unavailable. If a Solaris system with NFS mounts is online and one of those mounts goes away, it becomes very unhappy. So you see, this method means that none of those five workstations can ever be rebooted without an incredible headache. Any networking problem on any one of them becomes a problem on all of them. A total disaster, and all to get at that sweet 1-2GB of extra space.

CWRU.edu (0)

Anonymous Coward | more than 7 years ago | (#17454968)

Your email address indicates that you are from Case Western. Might not want to let your boss see that...

I have a great idea for them to spend money on: Why not invest a ton of money into ATM networking? It's going to be the wave of the future!

Re:CWRU.edu (1)

ximenes (10) | more than 7 years ago | (#17455030)

I'm not too worried. When the day comes that I'm not able to speak my mind positively or negatively about work or anything else, then I guess I'll just get fucked over.

Yes, CWRU was one of the largest ATM deployments in the world. In fairness, this was in the days of 10Mbit Ethernet, so I guess it made sense at the time. I do know that I took great joy in throwing piles of (totally worthless) $1000+ FORE ATM cards into the garbage when the network was converted to gigabit Ethernet. That certainly was not a blunder. Its still problematic that we use fiber instead of copper, but who knew that you'd be able to run gigabit Ethernet over basically the same cabling. I remember back in 1997 or whenenver, http://en.wikipedia.org/wiki/Fast_Ethernet#100BASE -T4 [wikipedia.org] was the wave of the future.

hacks? (2, Interesting)

robpoe (578975) | more than 7 years ago | (#17454444)

Heck, feel glad that the "U Bend" (a.k.a. trap) had a water faucet. when maintenance boxed in our sink they took off the spigots off the mop sink, but left the drain functional. Then they boxed it in plywood. After the A/C was installed (and had dehumidified the room) - and the building's humidified air was shut off to the room .. no more humidity for us, and the drain dried out.

Hmmm .. ugly hacks?

How about a Netware 3.x server stuck in a closet between two 10base2 (coax) runs, connecting one segment with another (glorified IPX router).

How about a 395 foot run of BNC ... that someone stuck a 10base-T hub on at 200 feet (give or take) because the hub actually strengthened the signal? Then they filled the hub with 8 10baseT computers..

How about a 450 foot run of BNC with old, frayed screw on ends. That ohmed out at about 76 ohms. And they wondered why the network was slow.. (I re-crimped all the ends and cut the wire approximately in half, and used a second NIC in the seerver). In that same place, one of the workers figured out that if they took one of the BNC connectors off the T in the back of the PC, the network would go down and they could just sit there and do nothing...

The company that had a 1000 foot run, so instead of buying ARCNet wire .. they put in BNC .. then ran ARCNet over it (averaged about 300kbit/sec). Then complained it was too slow (well no kidding!!)

The BNC wire I saw that someone had repaired with a paper clip and electrical tape ... after they'd sliced through it moving office furniture ... (yeah, it kinda worked)..

And we won't even talk about how many networks I ran into that looked like

Server ----- hub ----- hub ----- hub ----- hub

and they wondered why the people on the 4th hub would lose server connections randomly..

2 SUNs == remote control (4, Interesting)

darkonc (47285) | more than 7 years ago | (#17454452)

With our company was based in Vancouver, we determined that we could get much better bandwidth charges in Seattle, so most of our live servers were there. Two of our larger machines were SUN 450 boxes (bought because, back then, Oracle didn't have full support for Linux). After I set them up, we pulled out the graphics cards that they came with and shipped the cards and monitors back the Vancouver (they were part of the bundle). Then I connected the two machines with null modem cables, Port A - Port B. and Port B to Port A.

Once the graphics cards were removed, the machines defaulted to booting with Serial consoles. This meant that if anything went seriously wrong, just about anything other than hardware maintenance could be done by SSHing to machine X and using a terminal program to connect to the console port of machine Y (or vice versa).
This included the ability to do a complete wipe and install, needing only to instruct the CoLo staff to insert the install CD (which were left on top of the machines) into the appropriate box.

One of the monitors ended up on my desk. I can't remember who got the other one.

AC Guys Using A Garden Hose... (0)

creimer (824291) | more than 7 years ago | (#17454516)

The AC guys were fixing the unit on top of the building. They ran a garden hose through the overhead space above the IT department to pump green coolant fluid from one unit to another. The IT director heard a loud rumbling above his work area and evacuated from the area. The garden hose exploded. Coolant fluid broke through the ceiling panel to soak the 19-inch CRT monitor. That was a mess.

Slow boat to the Internet Highway (1)

darkonc (47285) | more than 7 years ago | (#17454554)

When I worked for one ISP, the main servers were connected to the internet via a stack of OC-3 links, but the building where all of the admins, support people, etc worked (3 floors of a decent-sized office building) was served by a single T-1 line. The ADSL link that one of our technical support people had on a test bench (it was for testing, so we couldn't route general traffic over it) turned out to be much faster than the link that the rest of the building shared.

quick, not so dirty... but quick (1, Interesting)

Anonymous Coward | more than 7 years ago | (#17454582)

The PR firm I worked for had a 66mhz 486 box with 24mb of RAM... this computer was running NAT, mail server and web server on Linux. It ran adequately, 24x7 for probably four years until a hardware failure. Suddenly there was no network, mail or web server.

There was an identical PC chassis collecting dust... I took that, installed the impressive quantity of 72 mb of RAM in it (maxed!), a new drive; two identical Ethernet cards; installed NetMax "firewall in a box" on it; transferred e-mail and web service to a vendor we were working with (the other unsung heroes of the story) and in about 72 hours after propagation, all was well. Total expenditure on this "rescue" was about $250.

This day-and-night effort to put the network back on its feet earned me the undying skepticism of the CEO, who postulated that I had somehow caused the old setup to fail in order to justify new purchases. So it sometimes goes. YMMV...

collision detection? (3, Funny)

trb (8509) | more than 7 years ago | (#17454648)

In the early 80's, I was working for a company that did lots of its own kernel hacking on UNIX and VMS systems. They had a habit of implementing lots of their own software systems, rather than using standard ones. Some were not very clever. For instance, they had a communication "protocol" that ran over ethernet cable, but it didn't handle collisions. Yes, we had thick ethernet running to every office, and when anyone wanted to use it, they'd run out in the hall and yell to make sure it wasn't in use. If there was contention, data would be corrupted. Eventually, we punted on this stupidity and used TCP/IP.

bad ethernet habits die hard (2, Interesting)

Anonymous Coward | more than 7 years ago | (#17454662)

A small business asked me to help them with their reliability problems. The company had been there for 20 years and the cabling for about 6 workstations and 4 or 5 big production printers consisted of premade cables tacked on the walls and ceiling, and one or more ancient efforts by electricians. There were serveral 10/100 hubs to extend the distance or 'multiplex' a single cable back to the internet connection. I went in several times to plug power into hubs, or pull cable out of a doorframe where it was being pinched. It was apparent that the main issue was this messy, poorly designed cable infrastructure.

So I got a data cable guy in to quote a clean installation-- home runs back to a single switch, cable in conduit, all clean and reliable. It was a few thousand dollars.

The owner decided it was too much $$, so he got an electrician to do it and now he has 3 more switches and even fewer home runs than he had before. But it's now Cat6, so he somehow thinks it's better.

I've already been back to troubleshoot... bad cabling.

Redneck Network Alarms (4, Funny)

mkcmkc (197982) | more than 7 years ago | (#17454664)

A Very Large Telecom Corp(TM) had let a contract for a hardware subsystem that was to be connected to their very expensive network monitoring system (probably HP Openview). Anyway, the vendor couldn't quit make this work. So, to satisfy the contract, they had a tape monkey with a laptop in the NOC. Whenever an event happened on the subsystem, he'd manually copy the message into a dialog box on the master monitoring system, at which point it'd pop up on the regular NOC alarm system...

Sprinklers from hell (4, Funny)

plopez (54068) | more than 7 years ago | (#17454692)

This happened just this past year.

We had moved into larger building with a server room in the basemnent (cue ominous music).

We rapidly began to run out of space so decided to place the chief sysape in the basement near the servers, which made sense. We cleaned up some items in the basement, moved them into storage, carpeted, dry walled etc. Since it was in the basement it needed an egress window with a steel casing and ladder. This actually turned the office into a nice garden level. You could look out the window and watch the sprinklers, see trees and grass etc.

On day, the chief sysape comes in and notices water on the floor. He looks over at the egress window and there is about 2 feet of water collected in the base of the exit well.

Well, they shut down the water to the entire building. Luckily the server room actually had about an 18 inch raised floor, so no damage.

To make a long story short, upon investigation it turned out that when the sprinkler system was installed, instead of capping off the ends of the plastic piping, they folded it over and crimped it. They relied on the mass of the dirt to keep the ends crimped, and for years it worked. Until the egress well was installed and the dirt was disturbed. Once it was disturbed, the crimps began to fail under water pressure. Leading to a near IT disaster.

Re:Sprinklers from hell (1)

Lost Engineer (459920) | more than 7 years ago | (#17454854)

This happens occasionally. My plumber friend assures me it's not standard practice to crimp off pipes, but some folks will do however poor a job they can get away with. Since the problem usually shows up years later, as in your case, I doubt they're ever taken to task.

power outages (1)

Dr_Art (937436) | more than 7 years ago | (#17454696)

I once worked on a project where we were developing a new telecom product. Our development servers and source code repository were in a lab with no uninterruptable power supplies (UPS's). We kept having our servers rebooting at odd hours. Then we found out that the cleaning crew was using portable "backpack" vacuum cleaners. They would plug into a wall outlet that was on the same breaker as our lab. When the breaker popped, they would reset it, and continue cleaning. There was a tape backup to preserve our source code, but unfortunately the backup was bigger than our tape, so the backups kept failing silently. We requisitioned some UPS's, and they kept failing. We found out later that the IT guys had inventoried some bad UPS's and those were the ones we got. This was the same project where I had to go to the PC graveyard and scavenge parts to build our production servers. Serious hackage, but a pretty fun project to work on...

Crap "servers" overheating? Rig some crap cooling (2, Interesting)

Frater 219 (1455) | more than 7 years ago | (#17454698)

My first job was for a small -- very small -- college. The IT department didn't have money for things like proper servers. We had cheap-ass desktop PCs stuck on a shelf in the one air-conditioned room in the office. Most of them, we built ourselves from the cheapest parts we could find -- usually, the corpses of broken workstations. The really important servers even had a UPS.

The machine with the user accounts on it had a few more hot, high-speed disks than the case was really designed to keep cool. It got hot and beeped. My boss wouldn't consider replacing it or even getting a new case. So I was forced to improvise: I cut a hole in the front panel and fitted a spare case fan into it. Then I realized that the motherboard didn't have another power connector for the case fan ... but I had a spare 5V wall-wart. A little wire-cutting and electrical-taping later, I had an externally cooled disk bay.

That "machine room" sucked. It was in the corner of the basement of a college office building. In the winter, the (crappy, household-type) AC unit iced over and the servers overheated. One summer, the facilities staff decided to power-wash the wood siding of the building. High-pressure water ran up through the wall and rained down right onto the server shelf. The only thing that blew up was the fancy new monitor that had come with the expensive and utterly overpowered RS/6000 just purchased by the library.

A couple of years ago when I visited the campus, they were still using that wall-wart-powered fan to cool the disks ....

Re:Crap "servers" overheating? Rig some crap cooli (1)

b0s0z0ku (752509) | more than 7 years ago | (#17454974)

The only thing that blew up was the fancy new monitor that had come with the expensive and utterly overpowered RS/6000 just purchased by the library.

Speaking of library computers, where I went to school (small school in SE PA) was using dumb terminals in the library connected to a VAX for their catalog system well into the 2000s - they still did as of my 5th reunion in May 06 actually! The wierd thing was that if you wanted to check e-mail and all of the public lab computers were taken, if you power cycled the terminal quickly, you'd get a command prompt. If you then typed C , you'd be able to telnet into the mail system. This worked fine until they made SSH mandatory in 2000 or so.


Network closet A/C suction (3, Interesting)

b0s0z0ku (752509) | more than 7 years ago | (#17454708)

Network closet shared with an A/C unit. First time I went in there, I couldn't open the door - I had to force it and it opened with a hiss. Turns out that the A/C system was installed without return ductwork and was sucking all of its intake air through a window that was open approximately two inches.


Newb Haxx (3, Funny)

NotoriousHood (970422) | more than 7 years ago | (#17454790)

When I started working at the school where I still work we were in two separate office buildings separated by more than 100m. I eventually ran coax through the wall via a light fixture, along a fence for about 100 feet into a tree to the roof (where it was held down by a sack of river rocks attached to some plywood) over the roof down a rain gutter and under the door. The building landlord was actually ok with this setup. It was mostly hidden except for the hop from the building to the fence and the whole tree to roof span.

The most messed up LAN, evar! (2, Funny)

SoupIsGood Food (1179) | more than 7 years ago | (#17454800)

OK. This was in the late nineties for a small computer hardware firm that had been in business since the '70s... but still, there was no excuse for this. It was a rambling wreck, a crazy collection of every ethernet standard implementation, and a few that were decidedly non-standard, just sort of tossed into place as time went on.

The backbone was a five port AUI concentrator... it was too primitive to be called a hub. (AUI was Sun's insane proprietary ethernet connector.) Hanging off that was a Sun server that was shiny and new when the Soviet Union was still in the news, which was the router to the DMZ, and a media adapter for thicknet. That thumb-thick yellow cable snaked over to the engineering cubes and hardware labs, with "vampire taps" hanging off it everywhere - vampire taps have a screw that drills into the cable, which is how you hook stuff up to thicknet. No lie. These were connected to 10Base-5 thin-net adapters, which hooked up to co-ax concentrators, which hooked up to AUI media adapters which hooked up to the various Sun workstations. I had never seen before, nor have I seen since, a BNC co-ax hub used just to hook up workstations in a star topology... for whatever reason, they decided that ring topology wasn't good enough to string five lightly used workstations together. I have no idea why any of this worked. It usually didn't, and needed various pieces of arcane equipment power-cycled and jiggled and cursed at to get any data to make it from the file servers to the workstations.

It gets worse. Another port on the AUI concentrator went to the Cabletron TPT-2 setup, which took care of accounting, sales, support and executive row. This was like 10Base-T ethernet, with a patch-panel that was wired to RJ-45 jacks in the offices and the cubes, except it was completely incompatible with 10Base-T equipment. Media adapters for all! And when one of the adapters goes down, the whole TPT-2 system locks up, a hundred or so systems. Let's play the hunt-the-locked-adapter! So much fun when the CIO is screaming at you.

I went on vacation, and the engineers were left to figure out how to bring the network back on when one of these adapters froze. You'd think they would unplug the patch cords one at a time in the computer room until the network came back up, but no. They just remembered that I told them power cycling an adapter would usually bring it back online. So they powered down the building. Serious. They needed to reboot the building... by this time all the critical systems were on UPS, so nothing was fuxxored, but still.

I eventually got the penny-pinchers in charge of the business to invest in nice 100B-T and 10B-T switches and AUI adapters and a few nice new Sun servers. Worked much better thereafter.

reboot monkey (2, Funny)

Dr_Art (937436) | more than 7 years ago | (#17454802)

I was once working in a datacenter on a machine close to the company email servers. There's a message on the console for the email server saying that the email database was corrupted and that some utility had to be run to fix it. An IT guy walks up to the console and looks at the message. He then proceeds to reboot the server. After the reboot, the same message appears. Reboot again. Same message. Reboot again. Same message. The IT guy repeated this 10 or 15 times. Then he walked away perplexed at why the email server wasn't working.

Laptop mail server (1)

b0s0z0ku (752509) | more than 7 years ago | (#17454808)

Not a huge hack, but migration from one mail system to another in a firm I was working for took longer than expected, and we were using the same hardware for old and new. So our interim mail server was an HP laptop running Redhat and a POP3 server. It ended up serving for around a week, after which it wasn't quite the same. It had run very hot (due to lack of power management?) and possibly had gotten a bit drain-bramaged in the process.


College internet (0)

Anonymous Coward | more than 7 years ago | (#17454842)

College internet service that is 45 Mbit in to over 4000 students living on campus, over 8000 total, switched to 100 Mbit to go between academic buildings, 10 Mbit to each dorm, each dorm then has a 10 Mbit switch in it and each floor has a 10 Mbit HUB (some dorms got lucky and had theirs replaced with switches). That's right, 10 Mbit hubs! Oh, and this isn't a flashback story, this is how it is now. Plus side: They are upgrading to 90 Mbit so that students will stop complaining about the slow internet speeds. Larger plus side: That's still nothing, students will still be complaining, some of the students get 30 / 4 to themselves at home.

Re:College internet (0)

Anonymous Coward | more than 7 years ago | (#17454918)

Sorry, I forgot to mention the CAT-3 lines from those hubs to the rooms. (that's right, the original phone lines from about 40 or more years ago, depending on the dorm... used to be two lines per room, now its 1 line 1 LAN)

Poland, 1996. (1)

b0s0z0ku (752509) | more than 7 years ago | (#17454874)

I travelled around Poland at the tender age of 17 in the mid-90s. From what I heard from my uncle who worked at Warsaw Polytechnic, the entire country had a link composed of 10 64kbps lines to the rest of Europe. And boy did it show - I had a UNIX account on NJ Superlink at the time. Whenever I telnetted to check my e-mail (t.g. for cybercafe), a keystroke would echo back to me with a lag time of 2-3 sec.

Furthermore, a lot of the rural phone system was still very old-fashioned. A different uncle's phone # was "42" in a certain small town in the Beskidy Mountains. The exchange itself wasn't automatic - you'd pick up the phone and give the number you wanted to the operator, who'd connect you (or call you back after 5 min. if you wanted long distance).


Space Station Video Hack (5, Interesting)

cyclone96 (129449) | more than 7 years ago | (#17454922)

If you've ever seen TV coverage of a Progress or Soyuz docking to the International Space Station, you've probably seen the ubiquitous black and white docking camera video with data overlayed on it as the vehicle approached the docking target.

Unfortunately, this television signal was only within the Russian Segment, and could only be downlinked through Russian communication assets over Russian ground sites. That limited the video to around 10 minutes each orbit, and required the docking to physically occur over Russia.

The US segment downlinks television via the Tracking and Data Relay Satellites (TDRS), which have more or less worldwide coverage. But the US segment and Russian Segment systems used incompatible video standards and weren't physically connected.

Yup, two video systems that cost tens of millions to develop, and they can't talk to each other. Classic "square peg, round hole" problem.

So we devised a setup where the crew ran a cable from the Russian Segment TV system into an IBM A31p laptop which converts the Russian SECAM signal to US NTSC video. The output from the laptop is connected to another cable strung down the stack into the US video system and downlinked via TDRS. Voila, greatly increased video coverage thanks to a lowly Thinkpad.

Details of this being tested can be found here: http://www.spaceref.com/news/viewsr.html?pid=18791 [spaceref.com]

Bizarre setups (1)

suv4x4 (956391) | more than 7 years ago | (#17454952)

Some of the bizarre setups I've witnessed on previous and current work locations:

* Intranet Application server also running a TV tuner for the workers to enjoy their favorite TV shows without a hassle.

* The server room is actually the living apartment of the boss. He has a cat. The cat is occasionally found napping on top of the servers, despite attempts to keep it outside that specific room inside the apartment.

* A guy running a bunch of servers decided that using electric socket splitters is too messy, so he instead cut a bunch of PC power cable and soldered them directly to the bare wires in the wall.

* Mission critical databases backed up daily to a collection of attached USB (mp3 player) flash sticks.

Peace and quiet and a good cup of coffee... (3, Funny)

ScrappyLaptop (733753) | more than 7 years ago | (#17455000)

The server room was a fairly large closet with an a/c outlet and a combination of wire racks and IKEA shelves. Nothing too bad, there; it all worked and everything was strapped down in case of a quake. However, to get to the server room you had to go through the breakroom and pass by the kitchen. Which had maybe two outlets and a hardwired coffee maker. Which shared a breaker panel with the server room in the hallway behind both rooms. Can't tell you how many times the admin assistant killed the server room trying to shut off the coffee maker. On a Friday, at 5 p.m... Funny thing is, the Head Cheese only worried about his coffee, not the servers that housed our precious COBOL and account information and wouldn't authorize either a separate breaker panel or a good UPS. Then again, that same admin assistant had to print out the boss' email and Excel spreadsheets and then re-type in his modifications...

GravityNet (0)

Anonymous Coward | more than 7 years ago | (#17455072)

I ran a cat5 cable down the side of a 5 story building to a conference room, because we didn't own the floors in between. The PHBs needed a connection to demo some things. Later I learned that they sold the company, and then they downsized me because the other company already had an IT department, which ran on M$ Windoze. I was a happy Linux+Sendmail+Netscape admin. Great job.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account