Heterogeneous file system with SSD disks would solve the issue.
http://www.ramsan.com/?gclid=CPzbvZjlrKACFYEwpAod-naN3g
Set up multiple tiers of disk:
Tier1 - SSD
Tier2 - HDD
Tier3 - LTO
Use data migration to truncate mature data to slower disks.
-----Original Message-----
From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com
[mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On
Behalf Of Andrew Klaassen
Sent: Wednesday, 10 March 2010 4:49 AM
To: discuss@studiosysadmins.com
Subject: RE: Researching: Isilon and alternatives
I think any of the vendors we're talking about here would struggle
with a workload like that. It's not because they sell bad servers; it's
the nature of platter-based hard drives. Let's do some math with the
Fastest Hard Drive In The World (according to the press release), a 15K
SCSI drive with a 3.2ms seek time:
1 second / 3.2 milliseconds = 312.5 seeks per second
312.5 seeks per second * 2KB per seek = 625KB/second per hard drive
Damn, that's low. Let's try with 4KB files:
312.5 seeks per second * 4KB per seek = 1250KB/second per hard drive
If you're constantly seeking for 4KB files, that's as fast as a single
hard drive (a damn fast hard drive) is going to be able to pump out the
data. Now, let's say you want to fill up a 1 GigE network pipe coming
out of that server:
1Gb/s = 128MB/s = 131072KB/s
131072 / 1250 = 104.9 hard drives needed
With files that small, you'd need over one hundred 15K hard drives just
to fill a single GigE network connection. I've never used Isilon, so I
don't know if it's great or if it's shit, but in this case Isilon
definitely can't be blamed for poor performance. Any vendor would've
struggled with a workload like that.
Andrew
--- On Tue, 3/9/10, Dave Algar dalgar@rainmaker.com wrote:
The entire
project used a little over 11TB
and had 53 million files for an average of just over 200KB
for file size.
I donât have specifics on how small the files
actually got but I do
recall seeing a lot of files in the 2 to 4k range. At
the time the
decision was made by the pipeline developer to use a lot of
small files and
bake out a lot of things for quicker load times.
Maybe it worked for him
but it didnât for me.
Dave
From:
studiosysadmins-discuss-bounces@mailman.studiosysadmins.com
[mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com]
On Behalf Of Greg
Whynott
Sent: Monday,
March 08, 2010 2:50
PM
To: '
discuss@studiosysadmins.com '
Subject: Re:
Researching: Isilon
and alternatives
What size is small? Just
wondering as from a fs level a meg
is considered large usually.
G
From:
studiosysadmins-discuss-bounces@mailman.studiosysadmins.com
studiosysadmins-discuss-bounces@mailman.studiosysadmins.com
To:
discuss@studiosysadmins.com
< discuss@studiosysadmins.com >
Sent: Mon Mar
08 17:46:32 2010
Subject: RE:
Researching: Isilon
and alternatives
Hi
Ian,
We have
both Isilon and Bluearc. I
have heard that a lot of people are quite happy with their
Isilon system but
our experience didnât go that well. I think a
lot has to do with
your file size. At the time we had an enormous amount
of small files and
the Isilon didnât fare too well. We had 24
nodes at the time with
about 250 to 300 artists. At the time they did not
have a backup node so
all we had were 1Gb connections available. There were
no 10Gb nodes at
the time so balancing userâs across nodes was another
issue.
Troubleshooting problems with a lot of nodes wasnât a
lot of fun.
They do have some nice features like quotaâs on a
directory or snapshots
but it just didnât handle our load. We had
IQ3000 nodes.
Dave
From:
studiosysadmins-discuss-bounces@mailman.studiosysadmins.com
[mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com]
On Behalf Of Ian
Haskin
Sent: Monday,
March 08, 2010 2:01
PM
To:
discuss@studiosysadmins.com
Subject:
Researching: Isilon and
alternatives
Hey everyone,
I've been looking into Isilon
storage and I'm quite impressed with
their sales-pitch. Can anyone share their experiences
with Isilon products
or any other products that compete/compare with
them?
Thanks,
Ian Haskin
TOPIX
This
e-mail and any attachments are
intended only for use by the addressee(s) named herein and
may contain
confidential information. If you are not the intended
recipient of this e-mail,
you are hereby notified any dissemination, distribution or
copying of this
email and any attachments is strictly prohibited. If you
receive this email in
error, please immediately notify the sender by return email
and permanently
delete the original, any copy and any printout thereof. The
integrity and
security of e-mail cannot be guaranteed.
This e-mail and any attachments are intended only for use
by the addressee(s) named herein and may contain
confidential information. If you are not the intended
recipient of this e-mail, you are hereby notified any
dissemination, distribution or copying of this email and any
attachments is strictly prohibited. If you receive this
email in error, please immediately notify the sender by
return email and permanently delete the original, any copy
and any printout thereof. The integrity and security of
e-mail cannot be guaranteed.
-----Inline Attachment Follows-----
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@mailman.studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-
discuss
>
Ask a question on any topic and get answers from real people. Go to
Yahoo! Answers and share what you know at http://ca.answers.yahoo.com
_
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@mailman.studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-
discuss
StudioSysAdmins-Discuss mailing list
StudioSysAdmins-Discuss@mailman.studiosysadmins.com
http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss
