Bulk Data Storage For The Common Man? 483
Vigyaan writes "Lately, I have been looking into different bulk data storage options available to a common man. My work
depends on generating, storing and analyzing a
large amount of data -- averaging about 1 TB per
month. I would like to have a storage system which is automated, fast, reliable
and most importantly does not cost the price of an
eye. Right now, I have a 4 node Linux cluster with
10 large hard disks (total capacity 1.6 TB); data storage roughly costs
about $0.60/GB (excluding the cost of PC
hardware). But long term storage is painful -- DVDs
cost about $0.10-$0.15/GB but takes too much human time
and leaving data on hard disks makes me nervous
because of possible failures. RAID is a possibility, but it increases the cost significantly. I was wondering, if
Slashdot readers have any recommendations for a
cheap automated way to store and retrieve data."
Finally a use for my 1GB Gmail invites... (Score:4, Funny)
Re:Finally a use for my 1GB Gmail invites... (Score:5, Funny)
Re:Finally a use for my 1GB Gmail invites... (Score:3, Informative)
Yee. Sent someone else who replied a invite too.
Mod this up and I'll send you one too.
Re:Finally a use for my 1GB Gmail invites... (Score:3, Informative)
Knowing how long hotmail and M$ has been around, and still failed to backup hotmail with their infinite windows license. What makes you think your 1 Gig will be backed up by Google.
Re:Finally a use for my 1GB Gmail invites... (Score:3, Informative)
Inside source? Just call them up and ask! It's not hidden knowledge [com.com].
Re:Finally a use for my 1GB Gmail invites... (Score:3, Interesting)
You always need at least 3 generations of backup. The Current backup, the "father", and the "Grandfather." These
Hard disks (Score:5, Informative)
If you buy them in bulk you can save.
Burning DVDs is going to take you forever and drive you nuts.
Find a hotswappable set of drives and use that for your offline backups. Use a raid for your current backups.
Re:Hard disks (Score:2)
A friend tells me that 20GB tapes are cheap. With a simple tape changing robot that might be manageable, just put in a new tape container every day...
Re:Hard disks (Score:4, Informative)
If you want one of those nifty things with robotic arms and whatnot, plan on spending upwards of $3500. The AIT Automated Tape Library goes for that much and holds only 15 tapes. Plan on spending tens of thousands for something like Ampex's DIS 914 for 30 Terabytes.
Your friend is right: tapes or cheap. The equipment needed to support them is expensive, slow and error prone. It gets cost effective once you have enough money for a new Porsche though...
Re:Hard disks (Score:3, Funny)
Tapes or cheap.
Re:Hard disks (Score:3, Interesting)
Re:Hard disks (Score:3, Informative)
Re:Hard disks (Score:4, Informative)
If I had the money, at a minimum I'd get a tape drive that could handle the 200 GB (uncompressed) tapes. Something like IBM's LTO Gen-2 Tape Library. That should run a bit less than $6,000.
For that matter, if I won the lottery, my first purchase would probably be a top of the line tape backup system instead of a the usual new car.
Since I can't afford it, I use DVDs and CDs for backups. They are a pain in the neck and are not that dependable, but I keep backups up to a year on DVD+RW so if one fails, hopefully the others will have the data.
Instead of writing directly to the DVD writer, I write the backups to disk and then copy the backup sets to the DVDs.
I also keep a complete current backup of nearly everything important on a seperate computer.
4*400Go Sata on Raid 5 (Score:5, Informative)
Not the cheapest, but fast, simple and saves you the unholy pleasure of having 2-3 DLT boxes to archive/cycle each month...
You already have a linux cluster, so implementing a distributed file system, or even simply a nightly incremental mirror to the target server if you can afford losing one day work/computation...
It would help if you told us what sort of data you work with... from databases and to automated telescope tracking system, both need large amount of storage, but you won't need the same system array for each...
I seem to remember a
You don't need to go to the Petabyte capacuty but you will find some interesting comments on filesystems, disk virtualisation, 1U rack providers and so on....so a 1 Terabyte rack server is definetly possible...
Good luck...
Re:Hard disks (Score:5, Interesting)
Re:Hard disks (Score:3, Informative)
RAID does not a backup system make. You still need backups.
For increased on-line availability, how about a good distribued file system with several servers? And, of course, back everything up anyway.
Waiting for ... (Score:3, Interesting)
Those will be sweet =)
good luck (Score:5, Funny)
Re:good luck (Score:2, Funny)
Poor luser.
Re:good luck (Score:3, Funny)
Please don't joke about that. Not too long ago, I received an email asking for help opening a Word document. Attached was a bitmap image which I naturally assumed was an error message. Instead, it was a screen-capture of the document's icon! The user was double-clicking on the image!
So I shot her.
Wirewire drives? (Score:5, Interesting)
Again, I'm not sure if that's as cheap as you'd want, but that's a solution I came up with for a similar problem. My company's going to be 3D rendering some stuff that could end up eating 50 megabytes a frame. (Extra data is stored for future refinement... I can go into detail if I've piqued anybody's curiosity.) We can't afford to lose this data, so the Firewire drive approach is what we're considering right now.
Re:Wirewire drives? (Score:4, Informative)
Oh, but they are cheap. Just buy a large IDE disk and a $30 firewire/fast-usb enclosure.
I'm just not sure about the "long term", though. I have no idea what the shelf life of a hard disk is.
Re:Wirewire drives? (Score:5, Informative)
Lacie makes their 1 terabyte firewire (943 gigabyte formatted) drive. I we get them for $1,080 a drive (Macmall matched Provantage's price). This is more then the article author spends now per gig, but these drives have done quite well in the studio. You can find cheaper firewire though.
We are at the point where hard drives give the best bang for the buck. The only fault of firewire is that my bosses have burned several bridges. ground yourself before unplugging the drives. The bridges were cheap though. In any case, hard drives are probably the most failsafe and cost effective solution, with firewire being the easiest interface to use those drives with.
Firewire hub with hardware RAID (Score:3, Interesting)
How expensive could something like this really be? $300-400 at most, I'd have to guess considering what most plac
Re:Firewire hub with hardware RAID (Score:3, Informative)
Firewire drives can be daisychained, and in fact OS X allows you to set up software RAID on multiple firewire drives attached to the system. You can't move them to another system and get access, but that's about the only limitation that I've found and it's more than decent for local high-density storage..
Re:Wirewire drives? (Score:2)
As opposed to a bunch of recordable DVDs also bought from Target?
So where's your solution, smart ass?
Re:Wirewire drives? (Score:5, Informative)
Re:Wirewire drives? (Score:3, Funny)
People take standard ATA/IDE drives and use an ATA/Firewire bridge to connect them up externally and bypass the extremely limited cable length of ATA."
Well that totally blows my point out of the water!
Personally I prefer something in a blonde (Score:4, Insightful)
Although the good ones don't come cheap. I guess this another case of "pick any two."
KFG
Re:Personally I prefer something in a blonde (Score:3, Informative)
KFG meant to say "You can have fast, good, or cheap. Pick two."
It's an old software design maxim that applies suprisingly well to this subject.
Re:Personally I prefer something in a blonde (Score:3, Interesting)
KFG meant to say "You can have fast, good, or cheap. Pick two."
It's an old software design maxim that applies suprisingly well to this subject.
...and to many things, particularly if you replace "fast" with "convenient". Just for kicks, think about it.
Food? Check. Clothing? Check. Beer? Check. Housing construction? Check.
Pretty much anything that involves the exchange of money for goods and services follows this maxim.
1TB a month?!? (Score:5, Funny)
SD
Re:1TB a month?!? (Score:5, Funny)
I simply make a tar.bz2 file with all my important files, filter it through gpg, then post it on edonkey, usually titled, "Olsen twins getting it on", and then usually the date.
Viola, instant backup that is available to me whereever I may go.
Re:1TB a month?!? (Score:5, Funny)
Re:1TB a month?!? (Score:4, Funny)
And you really need to fire your accountant. Your Caymon Island bank account was overdrawn twice in a month.
Re:1TB a month?!? (Score:5, Funny)
oh shit! I totally missed the part of the history where FMRI scanners came commonplace for men.
oh wait the whole ask slashdot blurb is twisted, the headline implies asking for datastorage possibilities for the common man - yet one of the first things mentioned that he needs it for his special job that generates tb's of data per month. by that definition he is not a common man, except that he hopes to have a miracle solution - that is quite common.
still, a common man would choose whatever possibility gave the cheapest price per gb(probably harddrive). with dvd-r's he would end up burning multiple dvd-r's per day and it's kind of implied that the data would need to be retrievable so he would have to burn the same disc multiple times, even then it wouldn't be a sure thing.
his needs are quite bigh though still, big enough to warrant for professional help since his likely going to be spending quite a bit of money on the thing.
Cheap solution (Score:4, Insightful)
Re:Cheap solution (Score:4, Insightful)
RAID is not backup !
RAID is not backup !
RAID is not backup !
[..]
Re:Cheap solution (Score:3, Interesting)
Indeed, I don't believe in any backup that doesn't have multiple copies that can be stored offsite. Fire really doesn't care what was on your hard drive, nor do thieves, or axe-wielding maniacs.
And anyone who has been in IT long enough can tell you one of the above stories first hand.
Bulk storage? (Score:3, Funny)
The Sony "lifetime" warranty may still be good on them too!
age old problem... (Score:5, Insightful)
If your data is worth $20,000.00 then a $2000.00 solution is dirt cheap.
what is your data worth? that is where you need to start and then look at the 10-30% of the data's value to start looking at how must to spend on it's storage.
If 1 month's data was lost forever, how much money would it cost the company? that is your actual $ amount that you should be shopping at.
and that is how I got the company to buy a $20,000.00 1000 tape DLT jukebox.
my data is worth over $100,000 a month and is much lower than yours is size.
That is where you need to start. Justify your storage costs by figureing out what it is worth to begin with.
Re:age old problem... (Score:5, Funny)
This 'data' doesnt happen to be a large collection of email addresses does it?
Re:age old problem... (Score:4, Informative)
Well... (Score:4, Interesting)
Tape? (Score:2, Interesting)
Re:Tape? (Score:2)
I think everyone imagines a room full of those spinning reel-to-reel things when you mention tape. It's archaic, but viable today!
Buy an older tape drive (Score:5, Informative)
Capacities are (for the cost of a sub $50 tape):
- LTO1: 100 GB uncompressed
- LTO2: 200 GB uncompressed
- SDLT220: 110 GB uncompressed
- SDLT320: 160 GB uncompressed
If your data is particularly ammenable to compression (i.e. database data) you could easily get 3 or 4 to 1 compression with these drives without sinking your CPU utilization.
Oh, I see... (Score:4, Funny)
1 TB/Month (Score:2, Insightful)
Sounds like you need a WORM (Score:2)
If you're not looking for permanent backups, the per-media cost may be prohibitive though.
Cheap and Big (Score:3, Insightful)
eh (Score:2, Interesting)
If you don't have the patience for DVD backups (neither do I), then you're pretty much stuck with RAID. So buck up, spend the extra cash, and setup a storage box or two on the network with one or two terabytes in each. I have a branch of my network setup on gigabit, one box has 250 GB of storage on RAID 1 across two 250 GB (this one's for video projects), the other has 160 GB in RAID 0 (my learning system). Works fine and easy as hell to setup. If I need to add storage I can either add some drives or just a
Re:eh (Score:2)
compression (Score:3, Informative)
Then backup using tapes just like every other place that has to do backups. Generally do full backups once a week and incremental ones nightly or whatever is necessary based on the data you are working with.
spongedrive is best (Score:5, Funny)
i have found that a Teutonium cluster of 6.5 TB Spongedrives (either Cray or SecreTech are fine) fits the bill nicely. housed in a 15-unit rack server, the amoeba-shaped drives utilize BioLas technology to store data on 6-dimensional Moebius Cilia for a slick seek time of 0.00 ms.
a cluster costs about $45,000 USD but the price should come down in 2004 Q4 when SecreTech launches their new 40-platter blackholium SCSI's.
Drawbacks, what are you willing to put up with? (Score:5, Informative)
Tape: Tapes break, they wear, they have dropouts, take a while to back everything up, can't always access files if you just want to restore something (Different methods vary, folks)... but ultimately, it's cheap when you use DAT because they're a common media. Swap the tapes twice as often (and throw old ones out) if you're paranoid about tape related failures.
Hard Drive: Most common form of backup I see now, mainly for the 1:1 size factor. Yeah, drives fail, too. Sometimes you have a pretty good warning when this is going to happen, sometimes you don't. (My 13GB Maxtor and 40GB IBM Deathstar drives both went *pfft* on reboot.) Get enough of them at once, you could swap out the logic boards if one does fry out. Ultimately, RAID or just simple 1:1 mirroring is probably the most efficient and easy method. Accessing bits and pieces is also easiest under this method. I personally just use an external USB2 case with a 120GB drive in it. Everything I want to back up goes on that drive, and then eventually... DVDRs. I turn off the drive when I don't need it... hopefully prolonging the life of it when I need it most.
DVDR: Not anymore. If we had these new-fangled DVDR discs (+ or -) say... when 2 to 6GB drives were common.... sure... But in addition to hard drives, recovering selective files is easy under this method too... Unless you use a backup program that crunches everything together on the disc in some spanning format. Burn times can be tedious... but it's not bad if you consider the overall amount of data you're putting on the disc. Cheaper than quality-brand name CDRs, though, in terms of price per mega/gigabyte. Only an idiot would trust $0.01-per-disc spindles for long-term backups. Even the longevity of DVDR has yet to be seen...
CDR: I'm not going to bother.
Network: Well, still relies on hard drives and other components... but good if you don't want to saddle one room with a ton of boxes. Simply for space and efficiency... external drive is probably better anyway.
Old fashioned method: Print everything out and keep it in a filing cabinet somewhere. You could always OCRA the stuff later.
Re:Drawbacks, what are you willing to put up with? (Score:3, Insightful)
LTO Ultrium 2 Tape Drive (Score:4, Informative)
The "Common Man" doesn't need bulk data storage... (Score:2, Insightful)
The reason you won't find such things on the cheap is because the average person with a PC doesn't even know what a GB is. He simply goes into the store, the sly salesman says "oh, what do you need it for," and then says "well 60-80 gb should be all you ever need."
Now, contrast that to me - my friends shit when they hear I have a 250 gb drive and a 12
I'm not a "Common Man" then (Score:3, Informative)
I have to disagree with the sister system though. For most geeks like you and I a sister system would be fairly adequate. It would be better with an occasiona
Why not tape drives? (Score:2, Insightful)
I don't know how much an eye goes for at the moment, but if you can spring for a Super DLT drive you'll get up 320GB (Compressed) for each tape.
It all comes down to the Quality:Cost:Time triangle.
Easy (Score:5, Funny)
What you want, for price. (Score:2)
There are many plausible suggestions though that won't break the bank totally. One of course is raid as has been mentioned and will be a few times I imagine. But you may also wish to look into hot swappable solutions.
USB 1.1/2.0, Firewire and SATA are all relatively cheap storage solutions if you shop around (Pricewatch [pricewatch.com] is a good place if you are willing.). You can con
Tape (Score:2)
If you have money burning a hole in your pocket, but a tape changer, so you don't have to change the tapes.
Hijack Cassini (Score:5, Funny)
It's about 90 minutes away, so at 250 Kbps that's over one terabit in storage on the way out there, and another terabit on the way back.
Worst-case access latency is about three hours, though. Maybe the hard disks are a better idea.
If you send your probe^H^H^H^H^H repeater to Alpha Centauri, you'll get more than 20,000 times the storage capacity.
Use those HDDs! (Score:2, Interesting)
The stockmarket is backed up to three (or more?) seperate locations. Look into NVRAM (e.g. flash media) or a cluster with all those hard drives linked together, with a constant backup. With the builtin IDE controlle
Re:Use those HDDs! (Score:5, Funny)
Let's say for the sake of argument that all 256 bytes can be printed as a visibly distinguishable character, or that he's got 1TB of plaintext. Also assume you can fit 10,000 characters on a 8 1/2 by 11 page.
You can fit 10^4 bytes per page, and you need to print 10^12 bytes (I know, it's actually 2^120, but that needlessly complicates the math, so shush)
That means you will need 10^12 bytes / 10^4 bytes/page = 10^8 pages.
One hundred million pages. Assuming he has a good laser printer with infinite toner, let's say he can print 60 ppm or one page per second. It would take one hundred million seconds to print the data, which is 1157 days, or a little over 3 years.
Given that he generates 1TB per month, I think this backup plan would probably become the top agenda item of most of the anti-deforestation groups out there.
Re:Use those HDDs! (Score:4, Interesting)
That said, this method would still be more than twice as expensive as storing data on hard drives, would still require a million pages, but would take a little under 2 weeks to print.
It still doesn't seem like a feasible option.
The up-side is that, if stored properly, the data would likely be safe potentially for many hundreds of years.
Do what Google does (Score:5, Insightful)
Bad idea (Score:5, Informative)
Re:Bad idea (Score:4, Informative)
google does not want ANY data to be lost. The have many mirrors of all data.
Re:Do what Google does (Score:4, Insightful)
You can solve this by ensuring some kind of in-process backup (like a SQL maintenance schedule, where it replicates itself), but then you're loading your replication process with a bunch of data that doesn't really need to be online, it needs to be in a vault someplace.
Besides, Sarbannes-Oxley and the IRS want you to keep backups 5+ years anyway, so this replication-only model is only good for data whose internal integrity isn't meaningful to anyone but the owner.
Vital information left out (Score:4, Insightful)
options options, what is your time and data worth? (Score:5, Insightful)
So, you can put your data on 4-5 HD's, 10 tapes or 232 DVD's per month. The Cost of doing so will be about $500 per month for the tapes or HD's and $50 for the DVD's (assuming your time cost $0)
At work, we had a need to keep a few TB of data online permanently, so we purchased a few NexSAN [nexsan.com] ATABeast's. At $50,000 for 10TB of usable storage ($5/GB), they may be a bit out of your price range. The advantage is that you can hold almost a years worth of data and it is protected by RAID5. It also makes management a lot easier, since it is very difficult to mount 42 300G drives in a single chassis (and it takes only 4U of rack space).
On the low end, NexSAN has the ATABoy2 or ATABaby (2TB or 1TB) for the $8-$15K range. This will let you hold a months worth of data
On the high end, You have EMC disk arrays (Think upwards or $20+/GB for the 'cheap' stuff from them.
Overall, if you have 1TB per month, you need to either a) get a grant to fund your work, b) hire somebody to swap DVD's for you or b) seriously rethink your data generation.
Any of the "cheap" storage methods have serious drawbacks, and the low cost ones are, well, not so low cost if $15,000 sounds like a lot of money to you.
otherwise, good luck
Re:options options, what is your time and data wor (Score:3, Informative)
A disadvantage is that the data cannot change while you write all N+1 DVDs and restoring would require lots of DVD swaps (regardless of whether you've lost a DVD or not) and the ability to incrementally write files with gaps in them (not an issue with most filesystems).
If its volume you want (Score:5, Informative)
Run the disks RAID 5 and you will get about 800GB of storage for $600 . Now get two cheap ata100 cards so you have a total of 6 channels, and mount each drive as a master on each channel. Build a 2gb root partition on the first disk (mirror it if you want) and then set the rest of the space up as a huge raid 5 array.
Et Voila cheap, big server. To archive data, turn off pc, and throw into attic
CD Changer (Score:2, Interesting)
Now the problem
DLT is the way to go (Score:3, Informative)
Ultrium (Score:3, Informative)
Consider Online Backup (Score:2, Interesting)
Only on slashdot (Score:5, Funny)
carousel for everyman (Score:4, Informative)
Um, 1TB a month in IDE drives is cheap . . . (Score:3, Informative)
Promise SX6000 = $255.95. (6) 200GB IDE drives in a Raid 5 = $624.95
If you had a separate boot drive from the SX6000, you could just bring the system down for a couple hour maintenance once a month and slam all the drives out and put fresh ones in.
Just keep buying new 200GB drives anymore and shelf the old ones (or if its *really* valuable and your home firesafe isn't enough, pay Iron mountain or someone to keep it).
There aren't hidden labor costs outside of those two hours it takes to setup a new array every month (DVD's are about 60 bucks a month for a TB, with a hundred or so for a drive (which *will* need to be replaced occasionally if you are burning that much) but you'll spend hours and hours just dealing with the swap outs and breaking up your data .
If you don't have to keep the TB of data after a month or three, then your price gets even cheaper after you invest in your initial hard drive media sets . . . and you can put all the drives in hot swap chassis to further minimize your time dealing with the issue.
Of course this is all moot if your 1TB of data isn't valuable enough to invest 600 a month in . . .
DVD Autoloader (Score:3, Insightful)
Heck, I remember a slashdot article about a guy who built one out of WOOD!
This would be a great solution for short term recovery storage. Just keep a stack of CD's or DVD's ready, and it will load them in and burn them all automatically.
On a site note, it would be great for converting a 400 disc cd collection into MP3's.
The simple economics of your "work" (Score:4, Insightful)
If that's too expensive (and it usually is), you can kludge your own system using low-end stuff from Hpaq/IBM/Dell's x86-server-oriented product lines. LTO1 drives are pretty cheap and we've found them to be very reliable over the past 3+ years, as well as offering 100 gig native per tape.
If even that's too expensive, then I seriously think you need to re-think the economics of your work situation. If your work doesn't cover your capital costs, you're not charging enough. If the work and data are business valuable enough, cutting your storage bill to the bone by building Linux clusters crammed with IDE HDDs is just a bad business decision.
If this is just your hobby-type work, then you need a cheaper hobby, like heroin addiction or something affordable. Physical space and electricity aren't cheap enough in a metropolitan area to burn through 1TB of storage per month, let along reliable data storage.
You're NOT using RAID??!!!! (Score:3, Insightful)
This is utter insanity! Without RAID, your only hope of safety is in your backups--which you're only asking about now!
RAID your data ASAP, and then start looking for backup systems. Take a look at some of the DLT4000 replacements.
Another perspective (Score:3, Interesting)
Just tossing out another point of view - similar but different than some of the others previously discussed. First off, examine the data you are keeping - do you really need that much? Nowadays it's common to be able to acquire data faster than it can be processed, and if you never stop gathering data, well, you never will catch up, only fall farther and farther behind.
If you DO need this data, and you are going to need it for awhile, (year or more) I'd recommend cheap HDs. They also have an advantage of being easily catalogged, and are untouchable when compared with access time of tapes. Don't go raid5 though, this is not "catastrophy-proof". (flood, fire, tornado, etc) For catastrophy protection, mirror your drives. When you have them loaded up with data, pull the FW cables and swap drives in the enclosures with fresh empty drives. Label them well, and then take each half of the mirror to DIFFERENT LOCATIONS. It's OK to keep one set on-site, but the other set must be somewhere else, preferably in another zip code. This will allow you near instant access to your data (since it's onsite), will protect your data from mechanical failure (through mirroring) and will protect you against catastrophy. (you WILL need to acquire new firewire boxes etc if your office gets leveled... don't forget this detail in general - the data is of no value if you lack the equipment (tape drives etc) to read it back in with) I know you can get compression and fit more on a tape etc by using archiving software, but it may be worth the extra cost to obey the KISS rule and just simply drag and drop the data to the formatted HDs. This will make data recovery MUCH SIMPLER, and if there are errors on the HD when you need to recover, this will insure you can actually recover most/all of the information. Archive streams and tapes are notorious for losing 100% of the data that follows a corruption point in the stream.
Once you know you no longer need a specific set, drop it back into the pool of usable drives. Buy them by the case, it's much cheaper this way. It also is advisable to buy the same make/model every time you have to get more drives, even if there are newer, larger, cheaper models out, because having all the same drives means one less complication to worry about in times of crisis.
Xraid (Score:3, Insightful)
If that doesn't suit ya, and it's bulk storage without necessarily speed you're looking for, check into the ATABoy line from Nexsan.
Re:Xraid (Score:3, Funny)
What are your near- and long-term requirements (Score:5, Informative)
I looked through some of the answers here, and as near as I can tell, you've got a bunch of home hobbyists telling you how to back up your home computers. Perhaps all your needs entail is a computer with an external IDE drive array and 4-10 200G SATA drives in it. But from your initial post, it's not clear what you need your offline storage _for_.
First of all, you mention that you generate and use 1G of data a month. What happens at the end of that month? Does all of the data become useless? Is some of it carried through? Is it useful for historical processing for some time after it's not "live" any more? The disposition of that offline data is important; you can't determine how you can most effectively back up your data until you know what you need to do with that data once it's backed up.
Since no one cares about backing up old data that they never use any more, I'm going to assume you need this data in some form in the future. I'm also assuming that your data ages out completely every month.
Realistically, you have two options: Large redundant disk arrays, or tape. Various factors give credence to one or the other.
First of all, get off of the SATA hacks, and realize you're going to need to go to SCSI, whether you end up with disk or tape. You're backing up data, you're going ot want it to be reliably written out, and SCSI is the de facto standard for backup architecture. Yes, you pay more for it, but there's a reason for it: the SCSI equipment I manage at work fails a fraction of the percentage of time that the various IDE/ATA systems fail. While SATA is marketed as a consumer technology, it will never meet the rigors of being a reliable backup methodology.
How about abusing physics law's? (Score:3, Interesting)
Work Demands should be realistic. (Score:3)
It sounds like you need a good cost benifit analysis and an idea of a budged.
First RAID your existing data.
Second Replicate any working solution you have now identically for next month and backup hardware.
Have a serious talk with work as to what is expensive and what you can afford. What happens if a data set is lost? How much damage\cost would that incur? I would look int AIT drives from Sony.
It sounds like you are in a frame of mind where you see everything as expensive. This will heavily influence your decision. Walk through a data disaster scenario with your backers and examine your costs in that light.
ls
Re:!RAID (Score:5, Insightful)
eric
Re:Give Up Now (Score:5, Informative)
Of course higher quality media might be better, but then you can no longer quote the $0.10/GB figure.
Re:Give Up Now (Score:3, Informative)
It will get scratched and damaged.
Which means that unless you're adding recovery data (using QuickPar) or burning 2 copies, you will lose at least some data on the media within a few years. (Cheap media sometimes only lasts a few months if not stored in dark and climate controller conditions.)
QuickPar is nice because you can pick how much redundancy you want on the disc. I find that 5-10% is plenty for most uses and guards against all but catestrophic da
Re:Give Up Now (Score:5, Insightful)
Re:Give Up Now (Score:5, Informative)
Re:Give Up Now (Score:3, Interesting)
Sounds trite, but EBay to the rescue.
I started with a single Exabyte 8mm backup drive and picked up 2 more on EBay for around $150. (This was a few years ago even and the original drive had been given to me with 50-60 used/new tapes.) Now that I have a DVD-burner, those drives don't do me
This is why I hate Ask Slashdot (Score:3, Insightful)
OK, 1TB/month that doesn't say much.
Always look at different levels of case scenarios and work from there. I usually start with loss of building by fire and work down through limited hardware failure or data corruption.
There are several factors that determine how often you should backup. Here's just a couple of questions to answer.
How much is the data worth?
How much is your time worth? If you lost a day or week of processing time.
Is your work time dependent? (deadlines)
Re:I have one (Score:3, Funny)