wow, great that they are sharing their design. one thing I wonder about is
PSU failure. from the way he outlined the connections, the raid is spread
across PSU's so if one PSU goes down, more than 2 drives will fail which
would potentially corrupt the raid, no?
with google sharing some of their server specs and blackblaze sharing their
storage, i wonder how long it'll be before we start seeing some opensource
hardware configurations tightly tuned with linux distros.
You could try one of these ...
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/
Klaus
On 8/31/09 12:43 PM, "Greg Whynott" Greg.Whynott@oicr.on.ca etched on
stone tablets:
Another free non rsync approach is a ?copy on write? file system, such as
ZFS. I took a look at a snapshot solution on linux using XFS at a
previous job which worked as expected. I?m sure there are other
filesystems out there that support it. I don?t think you have to spend a
lot of money to get file level protection, its more a matter of available
capacity.
-g
On 8/28/09 12:20 PM, "Jorg-Ulrich Mohnen, M.Sc., MBA"
jorg.mohnen@wavgen.com wrote:
Just followed this thread and wanted to give my two cents on this. Perhaps
someone already mentioned it.
Its really a 0 or a 1 here. If your company has no money, then setup a
cheap RAIDed solution as Jesse C suggests, and do an rsync every night from
12 midnight to 6am on the folders you need rsynced. Its really quite easy
under linux /etc
I have setup cheap Linux RAIDs for this type of nightly back up on Linux
for under $2000 and with a more stable and secure MAC OSX server and XRAID
for under $5000. The RAIDed linux software solution would yield about 4TB of
usable space, and the MAC Server XRAID solution would yield around 6TB of
usable space. Keep in mind that Apple only delivered IDE XRAIDs with maximum
500GB IDE disks. I bought the 750GB disks from CDW (OEM) and they work just
fine under non-warranty conditions.
This solution guarantees a 24 hour ~quasi~ snap shot at times of low to
minimal production. And make this KNOWN to the artists and folks that there
are only 24 hour back ups as there is no money for a more expensive
solution.
The more expensive hardware like Isolons and NetApps and especially
BlueArcs have more elegant solutions and they will actually give you
scripts. In Blue Arcs case, they actually have prewritten stuff as part of
the deal. SUN MIcroSystems is a class in itself and are excellent in this
type of support. If you pay $70,000 for a small 9TB usable volume (16TB
raw), you might expect them to bend over backwards to help you out. And in
the three companies I have recently worked for since SONY, they did (Jim
Henson Co, YuCo, and StudioGPU). I rewrote some stuff elegantly and the snap
shots approached runtime and were not ~quasi~ snapshots or nightly backups,
but actual snapshots.
J.
On Aug 28, 2009, at 9:02 AM, Jesse C wrote:
We're currently snapshotting between 1.8TB and 2.3TB of production files
with 2.7TB of snapshot disk. We snapshot every 6 hours and keep snapshots
for a month by default, but we've got cron scripts that go through and
remove older snapshots if we start running out disk space (which can happen
if there has been ridiculous amounts of churn in the file system).
We could easily increase the size of our snapshot if we needed to. We've
got 5x750GB drives right now in the RAID. If we had to, we could expand out
to 6 drives in the RAID and move up to 2TB drives, which would give us close
to 10TB. Not quite what you guys need, but there are motherboards out there
with 10xSATA ports on them, which when combined with 2TB drives, would get
you more into your ballpark.
On Fri, Aug 28, 2009 at 9:44 AM, Curtis Linstead curtis@rodeofx.com
wrote:
Jesse,
Thanks for the info!
I will definitely look further into this as a potential option for us.
How large was the pool you were doing the snapshot on and how large was the
RAID you built?
From what I?m reading I would need the size of one full backup plus a
little more and my current data pool can hit upwards of 16TB.
Not sure how easily I could configure that on a montherboard based raid
heh, but it definitely interests me.
Thanks again!
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
Jorg-Ulrich Mohnen, M.Sc. MBA
WavGen / Terratracer Incorporated
901 North Curson Avenue
West Hollywood, CA 90046
Tel 888-654-5615
Email jorg.mohnen@wavgen.com
------------------------------
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
