I did a fair bit of work with Lustre over 2007-2008. It?s pretty sweet
technology, but whether or not something like this backblaze would work with
it would depend a great deal on how the host OS presents storage. Lustre is
just another Linux file system, so it looks for LUNs upon which to construct
Lustre file system objects (metadata storage, object storage, etc.), so if
these things act as NAS devices, it would be a no-go. The one thing I really
like about Lustre is that it supports a variety of network interconnects, so
you can build a clustered SAN using GigE, IB, 10 GigE, etc. and it?s all
transparent to the client. It?s fast as hell, too. I don?t get too jazzed
about file systems and storage, but Lustre is definitely pretty interesting
technology.
Hm.
I?m wondering if a collection of backblaze systems could, if organized into
a Lustre file system and front-ended by an adequate Linux Samba/NFS file
server, provide a useable and relatively high-density, low-cost solution for
multi-hundred TB online archives.
Has anyone on the list been working with Lustre?
- Sean
From: studiosysadmins-discuss-bounces@studiosysadmins.com
[mailto:studiosysadmins-discuss-bounces@studiosysadmins.com] On Behalf Of
Daniel Roizman
Sent: Tuesday, September 01, 2009 8:07 PM
To: discuss@studiosysadmins.com
Subject: Re: Question about network deleting.
wow, great that they are sharing their design. one thing I wonder about is
PSU failure. from the way he outlined the connections, the raid is spread
across PSU's so if one PSU goes down, more than 2 drives will fail which would
potentially corrupt the raid, no?
with google sharing some of their server specs and blackblaze sharing their
storage, i wonder how long it'll be before we start seeing some opensource
hardware configurations tightly tuned with linux distros.
On Tue, Sep 1, 2009 at 5:06 PM, Klaus Steden klaus-s@moving-picture.com
wrote:
You could try one of these ...
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-
cloud-storage/
Klaus
On 8/31/09 12:43 PM, "Greg Whynott" Greg.Whynott@oicr.on.ca etched on stone
tablets:
Another free non rsync approach is a ?copy on write? file system, such as
ZFS. I took a look at a snapshot solution on linux using XFS at a
previous job which worked as expected. I?m sure there are other filesystems
out there that support it. I don?t think you have to spend a lot of money to
get file level protection, its more a matter of available capacity.
-g
On 8/28/09 12:20 PM, "Jorg-Ulrich Mohnen, M.Sc., MBA" jorg.mohnen@wavgen.com
wrote:
Just followed this thread and wanted to give my two cents on this. Perhaps
someone already mentioned it.
Its really a 0 or a 1 here. If your company has no money, then setup a cheap
RAIDed solution as Jesse C suggests, and do an rsync every night from 12
midnight to 6am on the folders you need rsynced. Its really quite easy under
linux /etc
I have setup cheap Linux RAIDs for this type of nightly back up on Linux for
under $2000 and with a more stable and secure MAC OSX server and XRAID for
under $5000. The RAIDed linux software solution would yield about 4TB of
usable space, and the MAC Server XRAID solution would yield around 6TB of
usable space. Keep in mind that Apple only delivered IDE XRAIDs with maximum
500GB IDE disks. I bought the 750GB disks from CDW (OEM) and they work just
fine under non-warranty conditions.
This solution guarantees a 24 hour ~quasi~ snap shot at times of low to
minimal production. And make this KNOWN to the artists and folks that there
are only 24 hour back ups as there is no money for a more expensive solution.
The more expensive hardware like Isolons and NetApps and especially BlueArcs
have more elegant solutions and they will actually give you scripts. In Blue
Arcs case, they actually have prewritten stuff as part of the deal. SUN
MIcroSystems is a class in itself and are excellent in this type of support.
If you pay $70,000 for a small 9TB usable volume (16TB raw), you might expect
them to bend over backwards to help you out. And in the three companies I have
recently worked for since SONY, they did (Jim Henson Co, YuCo, and StudioGPU).
I rewrote some stuff elegantly and the snap shots approached runtime and were
not ~quasi~ snapshots or nightly backups, but actual snapshots.
J.
On Aug 28, 2009, at 9:02 AM, Jesse C wrote:
We're currently snapshotting between 1.8TB and 2.3TB of production files with
2.7TB of snapshot disk. We snapshot every 6 hours and keep snapshots for a
month by default, but we've got cron scripts that go through and remove older
snapshots if we start running out disk space (which can happen if there has
been ridiculous amounts of churn in the file system).
We could easily increase the size of our snapshot if we needed to. We've got
5x750GB drives right now in the RAID. If we had to, we could expand out to 6
drives in the RAID and move up to 2TB drives, which would give us close to
10TB. Not quite what you guys need, but there are motherboards out there with
10xSATA ports on them, which when combined with 2TB drives, would get you more
into your ballpark.
On Fri, Aug 28, 2009 at 9:44 AM, Curtis Linstead curtis@rodeofx.com wrote:
Jesse,
Thanks for the info!
I will definitely look further into this as a potential option for us.
How large was the pool you were doing the snapshot on and how large was the
RAID you built?
From what I?m reading I would need the size of one full backup plus a little
more and my current data pool can hit upwards of 16TB.
Not sure how easily I could configure that on a montherboard based raid heh,
but it definitely interests me.
Thanks again!
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
Jorg-Ulrich Mohnen, M.Sc. MBA
WavGen / Terratracer Incorporated
901 North Curson Avenue
West Hollywood, CA 90046
Tel 888-654-5615
Email jorg.mohnen@wavgen.com
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
