If you truly need small file multi-threaded performance talk with someone about the HP X9000 series.

-----Original Message----- From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Andrew Klaassen Sent: Tuesday, March 09, 2010 12:49 PM To: discuss@studiosysadmins.com Subject: RE: Researching: Isilon and alternatives

I think any of the vendors we're talking about here would struggle with a workload like that. It's not because they sell bad servers; it's the nature of platter-based hard drives. Let's do some math with the Fastest Hard Drive In The World (according to the press release), a 15K SCSI drive with a 3.2ms seek time:

1 second / 3.2 milliseconds = 312.5 seeks per second

312.5 seeks per second * 2KB per seek = 625KB/second per hard drive

Damn, that's low. Let's try with 4KB files:

312.5 seeks per second * 4KB per seek = 1250KB/second per hard drive

If you're constantly seeking for 4KB files, that's as fast as a single hard drive (a damn fast hard drive) is going to be able to pump out the data. Now, let's say you want to fill up a 1 GigE network pipe coming out of that server:

1Gb/s = 128MB/s = 131072KB/s

131072 / 1250 = 104.9 hard drives needed

With files that small, you'd need over one hundred 15K hard drives just to fill a single GigE network connection. I've never used Isilon, so I don't know if it's great or if it's shit, but in this case Isilon definitely can't be blamed for poor performance. Any vendor would've struggled with a workload like that.

Andrew

--- On Tue, 3/9/10, Dave Algar dalgar@rainmaker.com wrote:

The entire project used a little over 11TB and had 53 million files for an average of just over 200KB for file size.  I don’t have specifics on how small the files actually got but I do recall seeing a lot of files in the 2 to 4k range.  At the time the decision was made by the pipeline developer to use a lot of small files and bake out a lot of things for quicker load times.  Maybe it worked for him but it didn’t for me.

 

Dave

 

From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Greg Whynott

Sent: Monday, March 08, 2010 2:50 PM

To: ' discuss@studiosysadmins.com '

Subject: Re: Researching: Isilon and alternatives

 

What size is small? Just wondering as from a fs level a meg is considered large usually.

G

From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com studiosysadmins-discuss-bounces@mailman.studiosysadmins.com

To: discuss@studiosysadmins.com < discuss@studiosysadmins.com >

Sent: Mon Mar 08 17:46:32 2010

Subject: RE: Researching: Isilon and alternatives

Hi Ian,

 

We have both Isilon and Bluearc.  I have heard that a lot of people are quite happy with their Isilon system but our experience didn’t go that well.  I think a lot has to do with your file size.  At the time we had an enormous amount of small files and the Isilon didn’t fare too well.  We had 24 nodes at the time with about 250 to 300 artists.  At the time they did not have a backup node so all we had were 1Gb connections available.  There were no 10Gb nodes at the time so balancing user’s across nodes was another issue.  Troubleshooting problems with a lot of nodes wasn’t a lot of fun.  They do have some nice features like quota’s on a directory or snapshots but it just didn’t handle our load.  We had IQ3000 nodes.

 

Dave

 

From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Ian Haskin

Sent: Monday, March 08, 2010 2:01 PM

To: discuss@studiosysadmins.com

Subject: Researching: Isilon and alternatives

 

Hey everyone,

 

I've been looking into Isilon storage and I'm quite impressed with their sales-pitch.  Can anyone share their experiences with Isilon products or any other products that compete/compare with them?

 

 

Thanks,

 

Ian Haskin

TOPIX

 

This e-mail and any attachments are intended only for use by the addressee(s) named herein and may contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any dissemination, distribution or copying of this email and any attachments is strictly prohibited. If you receive this email in error, please immediately notify the sender by return email and permanently delete the original, any copy and any printout thereof. The integrity and security of e-mail cannot be guaranteed.

 

This e-mail and any attachments are intended only for use by the addressee(s) named herein and may contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any dissemination, distribution or copying of this email and any attachments is strictly prohibited. If you receive this email in error, please immediately notify the sender by return email and permanently delete the original, any copy and any printout thereof. The integrity and security of e-mail cannot be guaranteed.

-----Inline Attachment Follows-----

_ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

Ask a question on any topic and get answers from real people. Go to Yahoo! Answers and share what you know at http://ca.answers.yahoo.com

---

StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

---

StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss