holly smokes thats a lot of errors... ________________________________________ From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Tom Taylor [taylor@the-mill.com] Sent: Thursday, March 11, 2010 6:23 PM To: discuss@studiosysadmins.com Subject: RE: Researching: Isilon and alternatives Everything is in an HP 2900 - 2 x 1Gb trunks into each cluster. (Soon to be Brocade or Extreme 10Gb) Seems like the interfaces are not happy. Port Total Bytes Total Frames Errors Rx Drops Tx Ctrl Limit ------- -------------- -------------- ------------ ------------ ----- ------ 1-Trk1 2,512,407,710 2,727,621,561 86,495 11,466,568 off 0 2-Trk1 3,123,561,267 1,198,521,278 18,014 8,707,104 off 0 3-Trk2 798,322,066 982,377,621 49,155 11,695,269 off 0 4-Trk2 53,405,790 2,926,838,905 163,569 14,461,588 off 0 5-Trk3 553,600,861 3,618,989,149 82,960 7,622,094 off 0 6-Trk3 1,887,102,210 4,292,551,184 60,317 6,679,635 off 0 7-Trk4 1,137,517,211 3,081,662,500 43,361 8,464,187 off 0 8-Trk4 3,503,697,323 3,465,088,585 85,276 9,135,323 off 0 9-Trk5 2,296,075,992 1,128,920,646 72,032 7,489,051 off 0 10-Trk5 2,879,550,610 2,600,747,860 54,721 8,322,167 off 0 11-Trk6 2,748,755,020 3,165,756,973 24,034 183,152,905 off 0 12-Trk6 1,611,690,357 2,354,340,713 9430 64,319,051 off 0 Thanks everyone for the test results! Cheers, Tom -----Original Message----- From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of bill@yuco.com Sent: Thursday, March 11, 2010 6:19 PM To: discuss@studiosysadmins.com Subject: Re: Researching: Isilon and alternatives Out of curiosity, what type of switches are you using? >-----Original Message----- >From: Tom Taylor [mailto:taylor@the-mill.com] >Sent: Thursday, March 11, 2010 01:10 PM >To: discuss@studiosysadmins.com >Subject: RE: Researching: Isilon and alternatives > >I've got a quick one, I'd like to test out our Isilon cluster. > >It seems as if I'm getting low performance at the moment and I was wondering if anyone else out there would be able to run a simple DD to compare. > >Here's my output from 2 different machines. > > >[root@flame-09-ny-courtney Tom]# dd if=/dev/zero of=test3.2gb bs=1M count=2096 >2197815296 bytes (2.2 GB) copied, 47.8699 seconds, 45.9 MB/s > >[root@ny3d-24 Tom]# dd if=/dev/zero of=ny3d.4gb bs=1M count=20096 >21072183296 bytes (21 GB) copied, 432.756 s, 48.7 MB/s > >I'm currently running a 6 node 6000x cluster on version 5.5.4.21. > >If anyone else could send me their output I'd appreciate it. > > >In related news, they've just announced a product called InsightIQ. It's a VM that shows incredible statistics about the cluster. I suggest chatting to your Isilon rep and getting a demo. > >Cheers, >Tom > > > > >-----Original Message----- >From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Sven Nielsen >Sent: Tuesday, March 09, 2010 6:25 PM >To: discuss@studiosysadmins.com >Subject: Re: Researching: Isilon and alternatives > >This is one thing I've been asked for a few times; the problem is that >NAS systems have their own filesystem on the NAS head, then present >the files of that filesystem out via NFS or CIFS (or whatever file- >based protocol). > >A fiber channel interface only knows how to speak blocks (via SCSI >commands); so there has to be a driver on top of that to implement a >filesystem (ext3, NTFS, CVFS, etc). Additionally, while the NAS head >manages competing requests for the same blocks, there's nothing like >that when you talk to a disk via Fiber Channel -- the disk will >happily take a write from one system that overwrites a change just >made by a different system. If one system decides to put a file where >the previous system just extended a directory entry, then your >filesystem is corrupt. > >Even NDMP over fiber channel is similar; the NDMP server (which >understands the filesystem's layout) is telling the NDMP tape device >to read blocks in a particular order from the disk. > >The only way for BlueArc, NetApp, etc. to present the same view of a >volume via fiber channel or iSCSI that they do via NFS or CIFS would >be if they: >a) released a windows/linux/mac/whatever driver that implements WFS or >WAFL, their on-disk filesystem type, and >b) set up these drivers and the NAS head to know how to communicate to >make sure that only one system is modifying a disk block at any one >time (which CVFS/StorNext handles for you). > >It gets to be a can of worms, and (at least for now) if you need fiber >channel access to disk for multiple clients, StorNext seems to be the >best combination of FCP-and-network access, since you can connect an >FCP client to the filesystem, and have it re-serve the filesystem via >NFS, CIFS, or StorNext LAN client. There's also filesystem software >like (I think) GPFS, Lustre, and others, that I think do similar >things to StorNext/CVFS, but I haven't explored those too much. > >If there are other and better options, I'd love to hear about them -- >anything to make the bits move faster. }:> > >-Sven > >On Mar 8, 2010, at 3:18 PM, J. J. Franzen wrote: > >> Actually, do any of the big guys (Netapp, Isilon, etc.) allow direct >> FC access to their volumes? I know Blue Arc does not and am curious >> if anyone else does. I would so love to do away with NDMP for my >> backups and be able to do true incremental backups like in the good >> ol days... >> >> J^2 >> >> On Mar 8, 2010, at 3:18 PM, Ian Haskin wrote: >> >>> We're a small TV-commercial shop (3D/compositing, 20 people, with >>> 50 or so >>> machines, incl. 4 flames) that is outgrowing our home-built Linux >>> NAS. I'm >>> looking for something scalable with high-bandwidth links for our >>> compositing >>> nodes, with a tie-in to the rest of our 3D 1Gb-ethernet network. >>> >>> >>> -----Original Message----- >>> From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com >>> [mailto:studiosysadmins-discuss- >>> bounces@mailman.studiosysadmins.com] On >>> Behalf Of Barry Robison >>> Sent: March-08-10 5:18 PM >>> To: discuss@studiosysadmins.com >>> Subject: Re: Researching: Isilon and alternatives >>> >>> On Tue, Mar 9, 2010 at 9:00 AM, Ian Haskin wrote: >>>> I've been looking into Isilon storage and I'm quite impressed with >>>> their >>>> sales-pitch. Can anyone share their experiences with Isilon >>>> products or >>> any >>>> other products that compete/compare with them? >>>> >>> >>> Hi Ian, >>> >>> What is your budget and expected workload? >>> >>> We evaluated both Isilon and BlueArc, with BlueArc winning our >>> business. For our expected artist count (200) and farm size ( ~6k >>> cores ), we liked BA better. We were sold on the data tiering, >>> disaster recovery, and throughput the hardware architecture allows. >>> From other shops we heard that BA was the only thing saturating >>> 10GbE. >>> We had issues getting Isilon working on 10GbE at all... >>> >>> At a previous facility I worked at we used a small Isilon cluster ( 4 >>> nodes ) quite happily. I think when you get up to the number of nodes >>> ( and accelerator nodes ) you need for a large cluster, some problems >>> can be expected. >>> >>> >>> Good luck, >>> -Barry >>> _______________________________________________ >>> StudioSysAdmins-Discuss mailing list >>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >>> >>> _______________________________________________ >>> StudioSysAdmins-Discuss mailing list >>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >> >> _______________________________________________ >> StudioSysAdmins-Discuss mailing list >> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >> > >_______________________________________________ >StudioSysAdmins-Discuss mailing list >StudioSysAdmins-Discuss@mailman.studiosysadmins.com >http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss > >v1 >http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=318704758&mt=8 >If you experience problems downloading, please use the alternative link below: >http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=318704758&mt=8 > >_______________________________________________ >StudioSysAdmins-Discuss mailing list >StudioSysAdmins-Discuss@mailman.studiosysadmins.com >http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss > _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss