I forgot to mention Arista, apparently they have some good kit and are gaining some ground. I've not used or know of anyone who has, but you may want to check them out too. www.aristanetworks.com g ________________________________________ From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Greg Whynott [Greg.Whynott@oicr.on.ca] Sent: Thursday, March 11, 2010 7:07 PM To: discuss@studiosysadmins.com Subject: RE: Researching: Isilon and alternatives I think he is hitting a resource limit within the switch. you shouldn't see many if any TX errors. when you do it sometimes is an indication of a buffer depletion issue. HP 8212's are great devices for HPC when you consider the cost per port and chassis capacity. Extreme and Force10 are both having a sale on 24 port 10Gbit switches this month. Personally I prefer Force10 over Extreme. I' am in contact wiht many of North America's largest HPC admins, non of them that I am aware of have Fourndry deployed. nor could I recommend them. Especially now since they were bought by the kings of gouging. Soon you'll have to license your ethernet ports individualy, and pay a prenium of you want to use any feature, such as QoS, or trunking. -g ________________________________________ From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Klaus Steden [klaus-s@moving-picture.com] Sent: Thursday, March 11, 2010 6:36 PM To: discuss@studiosysadmins.com Subject: Re: Researching: Isilon and alternatives When I was doing HPC storage research (Lustre, specifically), I found that Extreme switches (Summit series) consistently outperformed the competition (Cisco, Juniper, and others) by sometimes as much as 5-10%, and the Extreme was able to reliably get me about 118 MBps every time, and was giving me as much as 95% of theoretical max without any tweaking. As always, YMMV, but I've found Extreme to be extremely reliable in terms of switch throughput, and it comes without the "Cisco" sticker premium. There are other great performers out there as well (Foundry is one of them), but I didn't get a chance to punish one the same way so unfortunately I don't have numbers handy. Klaus On 3/11/10 3:23 PM, "Tom Taylor" etched on stone tablets: > Everything is in an HP 2900 - 2 x 1Gb trunks into each cluster. (Soon to be > Brocade or Extreme 10Gb) > > Seems like the interfaces are not happy. > > > Port Total Bytes Total Frames Errors Rx Drops Tx Ctrl Limit > ------- -------------- -------------- ------------ ------------ ----- ------ > 1-Trk1 2,512,407,710 2,727,621,561 86,495 11,466,568 off 0 > 2-Trk1 3,123,561,267 1,198,521,278 18,014 8,707,104 off 0 > 3-Trk2 798,322,066 982,377,621 49,155 11,695,269 off 0 > 4-Trk2 53,405,790 2,926,838,905 163,569 14,461,588 off 0 > 5-Trk3 553,600,861 3,618,989,149 82,960 7,622,094 off 0 > 6-Trk3 1,887,102,210 4,292,551,184 60,317 6,679,635 off 0 > 7-Trk4 1,137,517,211 3,081,662,500 43,361 8,464,187 off 0 > 8-Trk4 3,503,697,323 3,465,088,585 85,276 9,135,323 off 0 > 9-Trk5 2,296,075,992 1,128,920,646 72,032 7,489,051 off 0 > 10-Trk5 2,879,550,610 2,600,747,860 54,721 8,322,167 off 0 > 11-Trk6 2,748,755,020 3,165,756,973 24,034 183,152,905 off 0 > 12-Trk6 1,611,690,357 2,354,340,713 9430 64,319,051 off 0 > > Thanks everyone for the test results! > > Cheers, > Tom > > > -----Original Message----- > From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com > [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf > Of bill@yuco.com > Sent: Thursday, March 11, 2010 6:19 PM > To: discuss@studiosysadmins.com > Subject: Re: Researching: Isilon and alternatives > > > Out of curiosity, what type of switches are you using? > >> -----Original Message----- >> From: Tom Taylor [mailto:taylor@the-mill.com] >> Sent: Thursday, March 11, 2010 01:10 PM >> To: discuss@studiosysadmins.com >> Subject: RE: Researching: Isilon and alternatives >> >> I've got a quick one, I'd like to test out our Isilon cluster. >> >> It seems as if I'm getting low performance at the moment and I was wondering >> if anyone else out there would be able to run a simple DD to compare. >> >> Here's my output from 2 different machines. >> >> >> [root@flame-09-ny-courtney Tom]# dd if=/dev/zero of=test3.2gb bs=1M >> count=2096 >> 2197815296 bytes (2.2 GB) copied, 47.8699 seconds, 45.9 MB/s >> >> [root@ny3d-24 Tom]# dd if=/dev/zero of=ny3d.4gb bs=1M count=20096 >> 21072183296 bytes (21 GB) copied, 432.756 s, 48.7 MB/s >> >> I'm currently running a 6 node 6000x cluster on version 5.5.4.21. >> >> If anyone else could send me their output I'd appreciate it. >> >> >> In related news, they've just announced a product called InsightIQ. It's a VM >> that shows incredible statistics about the cluster. I suggest chatting to >> your Isilon rep and getting a demo. >> >> Cheers, >> Tom >> >> >> >> >> -----Original Message----- >> From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com >> [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On >> Behalf Of Sven Nielsen >> Sent: Tuesday, March 09, 2010 6:25 PM >> To: discuss@studiosysadmins.com >> Subject: Re: Researching: Isilon and alternatives >> >> This is one thing I've been asked for a few times; the problem is that >> NAS systems have their own filesystem on the NAS head, then present >> the files of that filesystem out via NFS or CIFS (or whatever file- >> based protocol). >> >> A fiber channel interface only knows how to speak blocks (via SCSI >> commands); so there has to be a driver on top of that to implement a >> filesystem (ext3, NTFS, CVFS, etc). Additionally, while the NAS head >> manages competing requests for the same blocks, there's nothing like >> that when you talk to a disk via Fiber Channel -- the disk will >> happily take a write from one system that overwrites a change just >> made by a different system. If one system decides to put a file where >> the previous system just extended a directory entry, then your >> filesystem is corrupt. >> >> Even NDMP over fiber channel is similar; the NDMP server (which >> understands the filesystem's layout) is telling the NDMP tape device >> to read blocks in a particular order from the disk. >> >> The only way for BlueArc, NetApp, etc. to present the same view of a >> volume via fiber channel or iSCSI that they do via NFS or CIFS would >> be if they: >> a) released a windows/linux/mac/whatever driver that implements WFS or >> WAFL, their on-disk filesystem type, and >> b) set up these drivers and the NAS head to know how to communicate to >> make sure that only one system is modifying a disk block at any one >> time (which CVFS/StorNext handles for you). >> >> It gets to be a can of worms, and (at least for now) if you need fiber >> channel access to disk for multiple clients, StorNext seems to be the >> best combination of FCP-and-network access, since you can connect an >> FCP client to the filesystem, and have it re-serve the filesystem via >> NFS, CIFS, or StorNext LAN client. There's also filesystem software >> like (I think) GPFS, Lustre, and others, that I think do similar >> things to StorNext/CVFS, but I haven't explored those too much. >> >> If there are other and better options, I'd love to hear about them -- >> anything to make the bits move faster. }:> >> >> -Sven >> >> On Mar 8, 2010, at 3:18 PM, J. J. Franzen wrote: >> >>> Actually, do any of the big guys (Netapp, Isilon, etc.) allow direct >>> FC access to their volumes? I know Blue Arc does not and am curious >>> if anyone else does. I would so love to do away with NDMP for my >>> backups and be able to do true incremental backups like in the good >>> ol days... >>> >>> J^2 >>> >>> On Mar 8, 2010, at 3:18 PM, Ian Haskin wrote: >>> >>>> We're a small TV-commercial shop (3D/compositing, 20 people, with >>>> 50 or so >>>> machines, incl. 4 flames) that is outgrowing our home-built Linux >>>> NAS. I'm >>>> looking for something scalable with high-bandwidth links for our >>>> compositing >>>> nodes, with a tie-in to the rest of our 3D 1Gb-ethernet network. >>>> >>>> >>>> -----Original Message----- >>>> From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com >>>> [mailto:studiosysadmins-discuss- >>>> bounces@mailman.studiosysadmins.com] On >>>> Behalf Of Barry Robison >>>> Sent: March-08-10 5:18 PM >>>> To: discuss@studiosysadmins.com >>>> Subject: Re: Researching: Isilon and alternatives >>>> >>>> On Tue, Mar 9, 2010 at 9:00 AM, Ian Haskin wrote: >>>>> I've been looking into Isilon storage and I'm quite impressed with >>>>> their >>>>> sales-pitch. Can anyone share their experiences with Isilon >>>>> products or >>>> any >>>>> other products that compete/compare with them? >>>>> >>>> >>>> Hi Ian, >>>> >>>> What is your budget and expected workload? >>>> >>>> We evaluated both Isilon and BlueArc, with BlueArc winning our >>>> business. For our expected artist count (200) and farm size ( ~6k >>>> cores ), we liked BA better. We were sold on the data tiering, >>>> disaster recovery, and throughput the hardware architecture allows. >>>> From other shops we heard that BA was the only thing saturating >>>> 10GbE. >>>> We had issues getting Isilon working on 10GbE at all... >>>> >>>> At a previous facility I worked at we used a small Isilon cluster ( 4 >>>> nodes ) quite happily. I think when you get up to the number of nodes >>>> ( and accelerator nodes ) you need for a large cluster, some problems >>>> can be expected. >>>> >>>> >>>> Good luck, >>>> -Barry >>>> _______________________________________________ >>>> StudioSysAdmins-Discuss mailing list >>>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >>>> >>>> _______________________________________________ >>>> StudioSysAdmins-Discuss mailing list >>>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >>> >>> _______________________________________________ >>> StudioSysAdmins-Discuss mailing list >>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >>> >> >> _______________________________________________ >> StudioSysAdmins-Discuss mailing list >> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >> >> v1 >> http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=318704758&a >> mp;mt=8 >> If you experience problems downloading, please use the alternative link >> below: >> http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=318704758&a >> mp;mt=8 >> >> _______________________________________________ >> StudioSysAdmins-Discuss mailing list >> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >> > > > _______________________________________________ > StudioSysAdmins-Discuss mailing list > StudioSysAdmins-Discuss@mailman.studiosysadmins.com > http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss > _______________________________________________ > StudioSysAdmins-Discuss mailing list > StudioSysAdmins-Discuss@mailman.studiosysadmins.com > http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

The chipset they have been using was recently bought by Broadcom. Not sure what the roadmap is. Strong backers and interesting product. ---------------------------------------------------- Douglas C. Atkinson 101-3738 North Fraser Way Burnaby, B.C. V5J 5G7 604.419.8585 Direct 604.434.2035 Fax 604.619.9430 Cell Seven Group - Calgary 106-919 Centre Street N.W Calgary, Alberta T2A 2P6 1-877-462-1777 x 102 "Data Management with Integrity" ----- Original Message ----- From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com To: discuss@studiosysadmins.com Sent: Thu Mar 11 16:27:03 2010 Subject: OFFTOPIC: switches. RE: Researching: Isilon and alternatives I forgot to mention Arista, apparently they have some good kit and are gaining some ground. I've not used or know of anyone who has, but you may want to check them out too. www.aristanetworks.com g ________________________________________ From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Greg Whynott [Greg.Whynott@oicr.on.ca] Sent: Thursday, March 11, 2010 7:07 PM To: discuss@studiosysadmins.com Subject: RE: Researching: Isilon and alternatives I think he is hitting a resource limit within the switch. you shouldn't see many if any TX errors. when you do it sometimes is an indication of a buffer depletion issue. HP 8212's are great devices for HPC when you consider the cost per port and chassis capacity. Extreme and Force10 are both having a sale on 24 port 10Gbit switches this month. Personally I prefer Force10 over Extreme. I' am in contact wiht many of North America's largest HPC admins, non of them that I am aware of have Fourndry deployed. nor could I recommend them. Especially now since they were bought by the kings of gouging. Soon you'll have to license your ethernet ports individualy, and pay a prenium of you want to use any feature, such as QoS, or trunking. -g ________________________________________ From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Klaus Steden [klaus-s@moving-picture.com] Sent: Thursday, March 11, 2010 6:36 PM To: discuss@studiosysadmins.com Subject: Re: Researching: Isilon and alternatives When I was doing HPC storage research (Lustre, specifically), I found that Extreme switches (Summit series) consistently outperformed the competition (Cisco, Juniper, and others) by sometimes as much as 5-10%, and the Extreme was able to reliably get me about 118 MBps every time, and was giving me as much as 95% of theoretical max without any tweaking. As always, YMMV, but I've found Extreme to be extremely reliable in terms of switch throughput, and it comes without the "Cisco" sticker premium. There are other great performers out there as well (Foundry is one of them), but I didn't get a chance to punish one the same way so unfortunately I don't have numbers handy. Klaus On 3/11/10 3:23 PM, "Tom Taylor" etched on stone tablets: > Everything is in an HP 2900 - 2 x 1Gb trunks into each cluster. (Soon to be > Brocade or Extreme 10Gb) > > Seems like the interfaces are not happy. > > > Port Total Bytes Total Frames Errors Rx Drops Tx Ctrl Limit > ------- -------------- -------------- ------------ ------------ ----- ------ > 1-Trk1 2,512,407,710 2,727,621,561 86,495 11,466,568 off 0 > 2-Trk1 3,123,561,267 1,198,521,278 18,014 8,707,104 off 0 > 3-Trk2 798,322,066 982,377,621 49,155 11,695,269 off 0 > 4-Trk2 53,405,790 2,926,838,905 163,569 14,461,588 off 0 > 5-Trk3 553,600,861 3,618,989,149 82,960 7,622,094 off 0 > 6-Trk3 1,887,102,210 4,292,551,184 60,317 6,679,635 off 0 > 7-Trk4 1,137,517,211 3,081,662,500 43,361 8,464,187 off 0 > 8-Trk4 3,503,697,323 3,465,088,585 85,276 9,135,323 off 0 > 9-Trk5 2,296,075,992 1,128,920,646 72,032 7,489,051 off 0 > 10-Trk5 2,879,550,610 2,600,747,860 54,721 8,322,167 off 0 > 11-Trk6 2,748,755,020 3,165,756,973 24,034 183,152,905 off 0 > 12-Trk6 1,611,690,357 2,354,340,713 9430 64,319,051 off 0 > > Thanks everyone for the test results! > > Cheers, > Tom > > > -----Original Message----- > From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com > [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf > Of bill@yuco.com > Sent: Thursday, March 11, 2010 6:19 PM > To: discuss@studiosysadmins.com > Subject: Re: Researching: Isilon and alternatives > > > Out of curiosity, what type of switches are you using? > >> -----Original Message----- >> From: Tom Taylor [mailto:taylor@the-mill.com] >> Sent: Thursday, March 11, 2010 01:10 PM >> To: discuss@studiosysadmins.com >> Subject: RE: Researching: Isilon and alternatives >> >> I've got a quick one, I'd like to test out our Isilon cluster. >> >> It seems as if I'm getting low performance at the moment and I was wondering >> if anyone else out there would be able to run a simple DD to compare. >> >> Here's my output from 2 different machines. >> >> >> [root@flame-09-ny-courtney Tom]# dd if=/dev/zero of=test3.2gb bs=1M >> count=2096 >> 2197815296 bytes (2.2 GB) copied, 47.8699 seconds, 45.9 MB/s >> >> [root@ny3d-24 Tom]# dd if=/dev/zero of=ny3d.4gb bs=1M count=20096 >> 21072183296 bytes (21 GB) copied, 432.756 s, 48.7 MB/s >> >> I'm currently running a 6 node 6000x cluster on version 5.5.4.21. >> >> If anyone else could send me their output I'd appreciate it. >> >> >> In related news, they've just announced a product called InsightIQ. It's a VM >> that shows incredible statistics about the cluster. I suggest chatting to >> your Isilon rep and getting a demo. >> >> Cheers, >> Tom >> >> >> >> >> -----Original Message----- >> From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com >> [mailto:studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On >> Behalf Of Sven Nielsen >> Sent: Tuesday, March 09, 2010 6:25 PM >> To: discuss@studiosysadmins.com >> Subject: Re: Researching: Isilon and alternatives >> >> This is one thing I've been asked for a few times; the problem is that >> NAS systems have their own filesystem on the NAS head, then present >> the files of that filesystem out via NFS or CIFS (or whatever file- >> based protocol). >> >> A fiber channel interface only knows how to speak blocks (via SCSI >> commands); so there has to be a driver on top of that to implement a >> filesystem (ext3, NTFS, CVFS, etc). Additionally, while the NAS head >> manages competing requests for the same blocks, there's nothing like >> that when you talk to a disk via Fiber Channel -- the disk will >> happily take a write from one system that overwrites a change just >> made by a different system. If one system decides to put a file where >> the previous system just extended a directory entry, then your >> filesystem is corrupt. >> >> Even NDMP over fiber channel is similar; the NDMP server (which >> understands the filesystem's layout) is telling the NDMP tape device >> to read blocks in a particular order from the disk. >> >> The only way for BlueArc, NetApp, etc. to present the same view of a >> volume via fiber channel or iSCSI that they do via NFS or CIFS would >> be if they: >> a) released a windows/linux/mac/whatever driver that implements WFS or >> WAFL, their on-disk filesystem type, and >> b) set up these drivers and the NAS head to know how to communicate to >> make sure that only one system is modifying a disk block at any one >> time (which CVFS/StorNext handles for you). >> >> It gets to be a can of worms, and (at least for now) if you need fiber >> channel access to disk for multiple clients, StorNext seems to be the >> best combination of FCP-and-network access, since you can connect an >> FCP client to the filesystem, and have it re-serve the filesystem via >> NFS, CIFS, or StorNext LAN client. There's also filesystem software >> like (I think) GPFS, Lustre, and others, that I think do similar >> things to StorNext/CVFS, but I haven't explored those too much. >> >> If there are other and better options, I'd love to hear about them -- >> anything to make the bits move faster. }:> >> >> -Sven >> >> On Mar 8, 2010, at 3:18 PM, J. J. Franzen wrote: >> >>> Actually, do any of the big guys (Netapp, Isilon, etc.) allow direct >>> FC access to their volumes? I know Blue Arc does not and am curious >>> if anyone else does. I would so love to do away with NDMP for my >>> backups and be able to do true incremental backups like in the good >>> ol days... >>> >>> J^2 >>> >>> On Mar 8, 2010, at 3:18 PM, Ian Haskin wrote: >>> >>>> We're a small TV-commercial shop (3D/compositing, 20 people, with >>>> 50 or so >>>> machines, incl. 4 flames) that is outgrowing our home-built Linux >>>> NAS. I'm >>>> looking for something scalable with high-bandwidth links for our >>>> compositing >>>> nodes, with a tie-in to the rest of our 3D 1Gb-ethernet network. >>>> >>>> >>>> -----Original Message----- >>>> From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com >>>> [mailto:studiosysadmins-discuss- >>>> bounces@mailman.studiosysadmins.com] On >>>> Behalf Of Barry Robison >>>> Sent: March-08-10 5:18 PM >>>> To: discuss@studiosysadmins.com >>>> Subject: Re: Researching: Isilon and alternatives >>>> >>>> On Tue, Mar 9, 2010 at 9:00 AM, Ian Haskin wrote: >>>>> I've been looking into Isilon storage and I'm quite impressed with >>>>> their >>>>> sales-pitch. Can anyone share their experiences with Isilon >>>>> products or >>>> any >>>>> other products that compete/compare with them? >>>>> >>>> >>>> Hi Ian, >>>> >>>> What is your budget and expected workload? >>>> >>>> We evaluated both Isilon and BlueArc, with BlueArc winning our >>>> business. For our expected artist count (200) and farm size ( ~6k >>>> cores ), we liked BA better. We were sold on the data tiering, >>>> disaster recovery, and throughput the hardware architecture allows. >>>> From other shops we heard that BA was the only thing saturating >>>> 10GbE. >>>> We had issues getting Isilon working on 10GbE at all... >>>> >>>> At a previous facility I worked at we used a small Isilon cluster ( 4 >>>> nodes ) quite happily. I think when you get up to the number of nodes >>>> ( and accelerator nodes ) you need for a large cluster, some problems >>>> can be expected. >>>> >>>> >>>> Good luck, >>>> -Barry >>>> _______________________________________________ >>>> StudioSysAdmins-Discuss mailing list >>>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >>>> >>>> _______________________________________________ >>>> StudioSysAdmins-Discuss mailing list >>>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >>> >>> _______________________________________________ >>> StudioSysAdmins-Discuss mailing list >>> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >>> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >>> >> >> _______________________________________________ >> StudioSysAdmins-Discuss mailing list >> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >> >> v1 >> http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=318704758&a >> mp;mt=8 >> If you experience problems downloading, please use the alternative link >> below: >> http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=318704758&a >> mp;mt=8 >> >> _______________________________________________ >> StudioSysAdmins-Discuss mailing list >> StudioSysAdmins-Discuss@mailman.studiosysadmins.com >> http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss >> > > > _______________________________________________ > StudioSysAdmins-Discuss mailing list > StudioSysAdmins-Discuss@mailman.studiosysadmins.com > http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss > _______________________________________________ > StudioSysAdmins-Discuss mailing list > StudioSysAdmins-Discuss@mailman.studiosysadmins.com > http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

On 12/03/2010, at 1:27 PM, Greg Whynott wrote: > I forgot to mention Arista, apparently they have some good kit and > are gaining some ground. I've not used or know of anyone who has, > but you may want to check them out too. > > www.aristanetworks.com > We've got a demo of a 10G Arista and so far it's really nice. Arista use the same Fulcrum chipset as just about all the other 10G switches out there but apparently they do make their own mainboards. Have a look at the Fulcrum Monaco switch and then the Juniper EX2500 or the Blade Networks Rackswitch... The Brocade Turboiron is a Woven which is now sold by Fortinet... So it really comes down to which software feature set you need. And price. Arista is doing some really interesting stuff with their software at the moment. One thing to keep in mind is that most of the new 10G stackable switches are cut-through which is how they get low latency. But that means that they have tiny buffers so they don't deal well with with oversubscribed links. They'll just drop everything they can't handle. For the most part you still need to go to a chassis switch for good buffering. Thanks, Matt _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

That Juniper EX2500 does not run JUNOS, FWIW. I think they licensed someone else's, for the sake of having a top-of-rack 10G option. I hope to see a proper Juniper 10G ToR switch that runs JUNOS some day. Also take a look at Fujitsu's line of ToR 10G switches. They're decent, fast, and mature (not Johnny-come-lately). For full disclosure, my employer reps both Juniper and Fujitsu, but I'm an engineer, not a sales dude... ---Ken On Mar 11, 2010, at 7:42 PM, Matt Provost wrote: > On 12/03/2010, at 1:27 PM, Greg Whynott wrote: >> I forgot to mention Arista, apparently they have some good kit and >> are gaining some ground. I've not used or know of anyone who has, >> but you may want to check them out too. >> >> www.aristanetworks.com >> > > > We've got a demo of a 10G Arista and so far it's really nice. > > Arista use the same Fulcrum chipset as just about all the other 10G > switches out there but apparently they do make their own mainboards. > Have a look at the Fulcrum Monaco switch and then the Juniper EX2500 > or the Blade Networks Rackswitch... > > The Brocade Turboiron is a Woven which is now sold by Fortinet... > > So it really comes down to which software feature set you need. And > price. Arista is doing some really interesting stuff with their > software at the moment. > > One thing to keep in mind is that most of the new 10G stackable > switches are cut-through which is how they get low latency. But that > means that they have tiny buffers so they don't deal well with with > oversubscribed links. They'll just drop everything they can't > handle. For the most part you still need to go to a chassis switch > for good buffering. > > Thanks, > Matt > _______________________________________________ > StudioSysAdmins-Discuss mailing list > StudioSysAdmins-Discuss@mailman.studiosysadmins.com > http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

Junipers EX2500 are oem blade networks. ---------------------------------------------------- Douglas C. Atkinson 101-3738 North Fraser Way Burnaby, B.C. V5J 5G7 604.419.8585 Direct 604.434.2035 Fax 604.619.9430 Cell Seven Group - Calgary 106-919 Centre Street N.W Calgary, Alberta T2A 2P6 1-877-462-1777 x 102 "Data Management with Integrity" ----- Original Message ----- From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com To: discuss@studiosysadmins.com Sent: Thu Mar 11 20:41:43 2010 Subject: Re: OFFTOPIC: switches. RE: Researching: Isilon and alternatives That Juniper EX2500 does not run JUNOS, FWIW. I think they licensed someone else's, for the sake of having a top-of-rack 10G option. I hope to see a proper Juniper 10G ToR switch that runs JUNOS some day. Also take a look at Fujitsu's line of ToR 10G switches. They're decent, fast, and mature (not Johnny-come-lately). For full disclosure, my employer reps both Juniper and Fujitsu, but I'm an engineer, not a sales dude... ---Ken On Mar 11, 2010, at 7:42 PM, Matt Provost wrote: > On 12/03/2010, at 1:27 PM, Greg Whynott wrote: >> I forgot to mention Arista, apparently they have some good kit and >> are gaining some ground. I've not used or know of anyone who has, >> but you may want to check them out too. >> >> www.aristanetworks.com >> > > > We've got a demo of a 10G Arista and so far it's really nice. > > Arista use the same Fulcrum chipset as just about all the other 10G > switches out there but apparently they do make their own mainboards. > Have a look at the Fulcrum Monaco switch and then the Juniper EX2500 > or the Blade Networks Rackswitch... > > The Brocade Turboiron is a Woven which is now sold by Fortinet... > > So it really comes down to which software feature set you need. And > price. Arista is doing some really interesting stuff with their > software at the moment. > > One thing to keep in mind is that most of the new 10G stackable > switches are cut-through which is how they get low latency. But that > means that they have tiny buffers so they don't deal well with with > oversubscribed links. They'll just drop everything they can't > handle. For the most part you still need to go to a chassis switch > for good buffering. > > Thanks, > Matt > _______________________________________________ > StudioSysAdmins-Discuss mailing list > StudioSysAdmins-Discuss@mailman.studiosysadmins.com > http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

On 12/03/2010, at 5:46 PM, Douglas C. Atkinson wrote: > Junipers EX2500 are oem blade networks. Yes that's the point I was trying to make sorry I wasn't clearer. There are a bunch of companies out there (Juniper, Blade Networks, etc) reselling the Fulcrum reference design. And most of the rest (Arista, Force10) are using the Fulcrum chipset and maybe doing something more clever with it. So the hardware is generally about the same and it really comes down to features and price. Thanks, Matt _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

I was doing reading on these switches, one of the 'geeky' interesting things with the Arista line is they run linux and apparently you can drop to a bash shell. The reversable fans are a neat idea too, front to back or back to front cooling. -g ________________________________________ From: studiosysadmins-discuss-bounces@mailman.studiosysadmins.com [studiosysadmins-discuss-bounces@mailman.studiosysadmins.com] On Behalf Of Matt Provost [mprovost@wetafx.co.nz] Sent: Friday, March 12, 2010 12:02 AM To: discuss@studiosysadmins.com Subject: Re: OFFTOPIC: switches. RE: Researching: Isilon and alternatives On 12/03/2010, at 5:46 PM, Douglas C. Atkinson wrote: > Junipers EX2500 are oem blade networks. Yes that's the point I was trying to make sorry I wasn't clearer. There are a bunch of companies out there (Juniper, Blade Networks, etc) reselling the Fulcrum reference design. And most of the rest (Arista, Force10) are using the Fulcrum chipset and maybe doing something more clever with it. So the hardware is generally about the same and it really comes down to features and price. Thanks, Matt _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss

On 12/03/2010, at 7:08 PM, Greg Whynott wrote: > I was doing reading on these switches, one of the 'geeky' > interesting things with the Arista line is they run linux and > apparently you can drop to a bash shell. The reversable fans are a > neat idea too, front to back or back to front cooling. Indeed: #show interfaces status | tail -n +2 | awk '{print $2}' | grep -c connected 5 #show environment cooling System cooling status is: Ok Ambient temperature: 26C Airflow: front-to-back Fan Tray Status Speed --------- --------------- ------ 1 Ok 42% 2 Ok 42% 3 Ok 42% 4 Ok 42% 5 Ok 42% _______________________________________________ StudioSysAdmins-Discuss mailing list StudioSysAdmins-Discuss@mailman.studiosysadmins.com http://mailman.studiosysadmins.com/mailman/listinfo/studiosysadmins-discuss