Sponsors
Sponsor Products
Nexenta - Anyone still using or considering a SDS solution from them?
posted by Greg Whynott  on July 17, 2017, 3:25 p.m. (9 days ago)
12 Responses     0 Plus One's     0 Comments  
Brian said:

However with your cluster arrangement and other needs, I can see why you chose a seemingly promising fee based option.

Let me be clear on this, while I was very happy to see someone pushing commercial solutions based on open source products (in this case ZFS which seemed like a god send of a file system/ vol management solution at the time) it was far from my first choice. Being in the M&E sector, deadlines are a way of life as is high performance backends. Not many sectors push the hardware as hard as we can/do daily.

At the time we had our main file systems on BlueArc. When it came time to expand I requested a quote from HDS for some additional disk shelves, one more head, and licenses to use. They came back with a quote for 1.1 million. That was so far out of the park I wondered if they thought it was a joke or they were smoking some good crack. ( funny, afterwards when they knew we were going else where that quote went from 1.1 million down to 700k and below quickly).

This prompted us to look elsewhere. To make another long story shorter, the ownership insisted we went with the cheapest, ignoring my resistance and advice for weeks about avoiding going with an 'unknown' or 'unproven' vendor or technology for this particular role in the company.

So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology. That said, it was easy for me to get behind as it was based on an opensource solution. The thinking at the time was "yeah, stick it to the over priced vendors, start a revolution in our sector, F HDS, or 110,000 dollar nodes anytime you want to expand!!". And the truth is I *REALLY* wanted to see it and them succeed. I kept the horror stories to myself for a very long time in an effort to see this revolution gain traction.


That said, I still think ZFS is one kick ass solution and I think everyone should be considering/evaluating it. This disk/data stuff isn't black magic anymore. Its crazy how vendors (Isilon, NetApp, HDS and so on) want to charge you a license for everything, protocals, reports, moving data from one solution to another.. Its such gouging. They work on the "well you paid this much already, this little hump won't be so bad, and besides, which options do you have?".



If I were to buy commercial storage again, I'd be looking at Sun/Oracle and Isilon, unless the vendor could point me to about 10 people on this list that are using it already without major show stoppers.



-g




Thread Tags:
  discuss-at-studiosysadmins 

Response from Michael Oliver @ July 19, 2017, 2:45 a.m.
Heads up sdedge is capped at 36 TB



Michael Oliver
mcoliver@gmail.com
858.336.1438
On Jul 18, 2017 1:18 AM, "Peter Smith" <peter.smith@framestore.com> wrote:
*checks out the "free and frictionless download"*

On 18 July 2017 at 09:06, julian firminger <justdigitalfilm@gmail.com> wrote:
ScaleIO maybe an option, but also checkout IsilonSD-Edge. It's basically software Isilon you can run on other hardware. And it comes pre-packed with all the licensing you need. So if you already have a SyncIQ license, you're good to go. We're about to look at it for NL for our NL410... somewhere between that and NL tape. Who said storage tiering was dead!

Julian Firminger

Senior Systems AdministratorUnited Broadcast FacilitiesAmsterdam, The Netherlands

On Tue, Jul 18, 2017 at 12:32 AM, Todd Smith <todd@sohovfx.com> wrote:
ScaleIO is an Dell/EMC product, I think it became available for purchase only recently, although it has been around for a while in the R&D. I was looking at it for a side project last year, never touched down but it has promise.

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 6:04:55 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

done some reading...

Where was ScaleIO when we were looking at Nexenta? :) Is it out there in the wild and being used in our sector?

greg



On Mon, Jul 17, 2017 at 5:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
We are very much considering re-using the hardware if we can't sort something out soon. The intention is to use this for nearline and VTL storage predominately.

I'd very much like to have HA between hosts, but I think for this role we can go without. I'll take 2 of the 4 heads and add them to our Proxmox VM cluster, they are each 390 gigs 24 physical core machines. The other idea is to turn all the JBODS into VTL targets directly connected to the backup server via SAS. Or maybe a Ceph file server?

ScaleIO is a dell product correct? I've not read up on it yet but will. Is that re-branded complelent?

-greg




On Mon, Jul 17, 2017 at 5:22 PM, Brian Krusic <brian@krusic.com> wrote:
Wow, seems to have done all the work for us.
All one has to do is read and follow directions. Nice link!

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:12 PM, William Sandler <william.sandler@allthingsmedia.com> wrote:

If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore. Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct, they did learn. there is an untold amount of money loss, both tangible and intangible. When shots took 40 minutes to open instead of under a minute, that causes the artists to become frustrated, and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'... How can you be creative when your pissed off and stressed out you may not make the deadline? This has a trickle down effect that we could talk for an hour about... reduced confidence, artists start saving work outside of the pipeline to avoid the slowness, and so on and so on. could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective, and the reps are very professional. Its expensive, but it works as advertised. We wouldn't of had to buy the Isilon if the Nexenta was able to deliver. So in my eyes those costs are on Nexenta as well. The company would of gone under if we tried to stick it out in my opinion. Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution. The Isilon just works.


Make cheap, pay twice.

greg






On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg. I know you had an uphill battle to fight on new storage infrastructure. One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.
Cheers,

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDSsolution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.


- Brian

Every day is a gift
Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--
Framestore Peter Smith Senior Systems Engineer
London New York Los Angeles Chicago Montral
T+44 (0)20 7344 8000 M+44 (0)7816 123009
19-23 Wells Street, London W1T 3PQ
Twitter Facebook framestore.com
https://www.framestore.com/

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

0 Plus One's     0 Comments  
   

Response from Anonymous @ July 18, 2017, 4:20 a.m.
*checks out the "free and frictionless download"*

On 18 July 2017 at 09:06, julian firminger <justdigitalfilm@gmail.com> wrote:
ScaleIO maybe an option, but also checkout IsilonSD-Edge. It's basically software Isilon you can run on other hardware. And it comes pre-packed with all the licensing you need. So if you already have a SyncIQ license, you're good to go. We're about to look at it for NL for our NL410... somewhere between that and NL tape. Who said storage tiering was dead!

Julian Firminger

Senior Systems AdministratorUnited Broadcast FacilitiesAmsterdam, The Netherlands

On Tue, Jul 18, 2017 at 12:32 AM, Todd Smith <todd@sohovfx.com> wrote:
ScaleIO is an Dell/EMC product, I think it became available for purchase only recently, although it has been around for a while in the R&D. I was looking at it for a side project last year, never touched down but it has promise.

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 6:04:55 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

done some reading...

Where was ScaleIO when we were looking at Nexenta? :) Is it out there in the wild and being used in our sector?

greg



On Mon, Jul 17, 2017 at 5:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
We are very much considering re-using the hardware if we can't sort something out soon. The intention is to use this for nearline and VTL storage predominately.

I'd very much like to have HA between hosts, but I think for this role we can go without. I'll take 2 of the 4 heads and add them to our Proxmox VM cluster, they are each 390 gigs 24 physical core machines. The other idea is to turn all the JBODS into VTL targets directly connected to the backup server via SAS. Or maybe a Ceph file server?

ScaleIO is a dell product correct? I've not read up on it yet but will. Is that re-branded complelent?

-greg




On Mon, Jul 17, 2017 at 5:22 PM, Brian Krusic <brian@krusic.com> wrote:
Wow, seems to have done all the work for us.
All one has to do is read and follow directions. Nice link!

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:12 PM, William Sandler <william.sandler@allthingsmedia.com> wrote:

If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore. Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct, they did learn. there is an untold amount of money loss, both tangible and intangible. When shots took 40 minutes to open instead of under a minute, that causes the artists to become frustrated, and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'... How can you be creative when your pissed off and stressed out you may not make the deadline? This has a trickle down effect that we could talk for an hour about... reduced confidence, artists start saving work outside of the pipeline to avoid the slowness, and so on and so on. could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective, and the reps are very professional. Its expensive, but it works as advertised. We wouldn't of had to buy the Isilon if the Nexenta was able to deliver. So in my eyes those costs are on Nexenta as well. The company would of gone under if we tried to stick it out in my opinion. Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution. The Isilon just works.


Make cheap, pay twice.

greg






On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg. I know you had an uphill battle to fight on new storage infrastructure. One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.
Cheers,

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDSsolution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.


- Brian

Every day is a gift
Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--
Framestore Peter Smith Senior Systems Engineer
London New York Los Angeles Chicago Montral
T+44 (0)20 7344 8000 M+44 (0)7816 123009
19-23 Wells Street, London W1T 3PQ
Twitter Facebook framestore.com
https://www.framestore.com/

0 Plus One's     0 Comments  
   

Response from Julian Firminger @ July 18, 2017, 4:10 a.m.
ScaleIO maybe an option, but also checkout IsilonSD-Edge. It's basically software Isilon you can run on other hardware. And it comes pre-packed with all the licensing you need. So if you already have a SyncIQ license, you're good to go. We're about to look at it for NL for our NL410... somewhere between that and NL tape. Who said storage tiering was dead!

Julian Firminger

Senior Systems AdministratorUnited Broadcast FacilitiesAmsterdam, The Netherlands

On Tue, Jul 18, 2017 at 12:32 AM, Todd Smith <todd@sohovfx.com> wrote:
ScaleIO is an Dell/EMC product, I think it became available for purchase only recently, although it has been around for a while in the R&D. I was looking at it for a side project last year, never touched down but it has promise.

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 6:04:55 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

done some reading...

Where was ScaleIO when we were looking at Nexenta? :) Is it out there in the wild and being used in our sector?

greg



On Mon, Jul 17, 2017 at 5:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
We are very much considering re-using the hardware if we can't sort something out soon. The intention is to use this for nearline and VTL storage predominately.

I'd very much like to have HA between hosts, but I think for this role we can go without. I'll take 2 of the 4 heads and add them to our Proxmox VM cluster, they are each 390 gigs 24 physical core machines. The other idea is to turn all the JBODS into VTL targets directly connected to the backup server via SAS. Or maybe a Ceph file server?

ScaleIO is a dell product correct? I've not read up on it yet but will. Is that re-branded complelent?

-greg




On Mon, Jul 17, 2017 at 5:22 PM, Brian Krusic <brian@krusic.com> wrote:
Wow, seems to have done all the work for us.
All one has to do is read and follow directions. Nice link!

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:12 PM, William Sandler <william.sandler@allthingsmedia.com> wrote:

If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore. Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct, they did learn. there is an untold amount of money loss, both tangible and intangible. When shots took 40 minutes to open instead of under a minute, that causes the artists to become frustrated, and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'... How can you be creative when your pissed off and stressed out you may not make the deadline? This has a trickle down effect that we could talk for an hour about... reduced confidence, artists start saving work outside of the pipeline to avoid the slowness, and so on and so on. could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective, and the reps are very professional. Its expensive, but it works as advertised. We wouldn't of had to buy the Isilon if the Nexenta was able to deliver. So in my eyes those costs are on Nexenta as well. The company would of gone under if we tried to stick it out in my opinion. Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution. The Isilon just works.


Make cheap, pay twice.

greg






On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg. I know you had an uphill battle to fight on new storage infrastructure. One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.
Cheers,

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDSsolution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.


- Brian

Every day is a gift
Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from Todd Smith @ July 17, 2017, 6:35 p.m.
ScaleIO is an Dell/EMC product, I think it became available for purchase only recently, although it has been around for a while in the R&D.  I was looking at it for a side project last year, never touched down but it has promise. 

Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 6:04:55 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

done some reading...

Where was ScaleIO when we were looking at Nexenta?  :)    Is it out there in the wild and being used in our sector? 

greg



On Mon, Jul 17, 2017 at 5:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
We are very much considering re-using the hardware if we can't sort something out soon.  The intention is to use this for nearline and VTL storage predominately.  

I'd very much like to have HA between hosts,  but I think for this role we can go without.   I'll take 2 of the 4 heads and add them to our Proxmox VM cluster,  they are each 390 gigs 24 physical core machines.  The other idea is to turn all the JBODS into VTL targets directly connected to the backup server via SAS.    Or maybe a Ceph file server? 

ScaleIO is a dell product correct?   I've not read up on it yet but will.   Is that re-branded complelent? 

-greg




On Mon, Jul 17, 2017 at 5:22 PM, Brian Krusic <brian@krusic.com> wrote:
Wow, seems to have done all the work for us.
All one has to do is read and follow directions.  Nice link!

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:12 PM, William Sandler <william.sandler@allthingsmedia.com> wrote:

If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .  


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore.  Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct,  they did learn.   there is an untold amount of money loss,  both tangible and intangible.   When shots took 40 minutes to open instead of under a minute,  that causes the artists to become frustrated,  and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'...   How can you be creative when your pissed off and stressed out you may not make the deadline?     This has a trickle down effect that we could talk for an hour about...  reduced confidence,  artists start saving work outside of the pipeline to avoid the slowness,  and so on and so on.  could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective,  and the reps are very professional.  Its expensive, but it works as advertised.   We wouldn't of had to buy the Isilon if the Nexenta was able to deliver.   So in my eyes those costs are on Nexenta as well.    The company would of gone under if we tried to stick it out in my opinion.   Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution.   The Isilon just works.


Make cheap,  pay twice.

greg




  

On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg.  I know you had an uphill battle to fight on new storage infrastructure.  One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.  
Cheers,

Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS        solution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made,  it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.

 
- Brian

Every day is a gift
        Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

0 Plus One's     0 Comments  
   

Response from Todd Smith @ July 17, 2017, 6:35 p.m.
ScaleIO is an Dell/EMC product, I think it became available only recently, although it has been around for a while in the R&D.  I was looking at i 
Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 6:04:55 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

done some reading...

Where was ScaleIO when we were looking at Nexenta?  :)    Is it out there in the wild and being used in our sector? 

greg



On Mon, Jul 17, 2017 at 5:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
We are very much considering re-using the hardware if we can't sort something out soon.  The intention is to use this for nearline and VTL storage predominately.  

I'd very much like to have HA between hosts,  but I think for this role we can go without.   I'll take 2 of the 4 heads and add them to our Proxmox VM cluster,  they are each 390 gigs 24 physical core machines.  The other idea is to turn all the JBODS into VTL targets directly connected to the backup server via SAS.    Or maybe a Ceph file server? 

ScaleIO is a dell product correct?   I've not read up on it yet but will.   Is that re-branded complelent? 

-greg




On Mon, Jul 17, 2017 at 5:22 PM, Brian Krusic <brian@krusic.com> wrote:
Wow, seems to have done all the work for us.
All one has to do is read and follow directions.  Nice link!

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:12 PM, William Sandler <william.sandler@allthingsmedia.com> wrote:

If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .  


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore.  Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct,  they did learn.   there is an untold amount of money loss,  both tangible and intangible.   When shots took 40 minutes to open instead of under a minute,  that causes the artists to become frustrated,  and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'...   How can you be creative when your pissed off and stressed out you may not make the deadline?     This has a trickle down effect that we could talk for an hour about...  reduced confidence,  artists start saving work outside of the pipeline to avoid the slowness,  and so on and so on.  could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective,  and the reps are very professional.  Its expensive, but it works as advertised.   We wouldn't of had to buy the Isilon if the Nexenta was able to deliver.   So in my eyes those costs are on Nexenta as well.    The company would of gone under if we tried to stick it out in my opinion.   Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution.   The Isilon just works.


Make cheap,  pay twice.

greg




  

On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg.  I know you had an uphill battle to fight on new storage infrastructure.  One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.  
Cheers,

Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS        solution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made,  it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.

 
- Brian

Every day is a gift
        Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

0 Plus One's     0 Comments  
   

Response from Greg Whynott @ July 17, 2017, 6:10 p.m.
done some reading...

Where was ScaleIO when we were looking at Nexenta? :) Is it out there in the wild and being used in our sector?

greg



On Mon, Jul 17, 2017 at 5:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
We are very much considering re-using the hardware if we can't sort something out soon. The intention is to use this for nearline and VTL storage predominately.

I'd very much like to have HA between hosts, but I think for this role we can go without. I'll take 2 of the 4 heads and add them to our Proxmox VM cluster, they are each 390 gigs 24 physical core machines. The other idea is to turn all the JBODS into VTL targets directly connected to the backup server via SAS. Or maybe a Ceph file server?

ScaleIO is a dell product correct? I've not read up on it yet but will. Is that re-branded complelent?

-greg




On Mon, Jul 17, 2017 at 5:22 PM, Brian Krusic <brian@krusic.com> wrote:
Wow, seems to have done all the work for us.
All one has to do is read and follow directions. Nice link!

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:12 PM, William Sandler <william.sandler@allthingsmedia.com> wrote:

If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore. Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct, they did learn. there is an untold amount of money loss, both tangible and intangible. When shots took 40 minutes to open instead of under a minute, that causes the artists to become frustrated, and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'... How can you be creative when your pissed off and stressed out you may not make the deadline? This has a trickle down effect that we could talk for an hour about... reduced confidence, artists start saving work outside of the pipeline to avoid the slowness, and so on and so on. could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective, and the reps are very professional. Its expensive, but it works as advertised. We wouldn't of had to buy the Isilon if the Nexenta was able to deliver. So in my eyes those costs are on Nexenta as well. The company would of gone under if we tried to stick it out in my opinion. Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution. The Isilon just works.


Make cheap, pay twice.

greg






On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg. I know you had an uphill battle to fight on new storage infrastructure. One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.
Cheers,

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDSsolution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.


- Brian

Every day is a gift
Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



0 Plus One's     0 Comments  
   

Response from Anonymous @ July 17, 2017, 6:10 p.m.
Hi Greg,
Very sorry to hear what you've been through - stuff of nightmares..
**Really** appreciate you sharing this - and so pleased you are out the other side!
George
On Mon, Jul 17, 2017 at 7:02 PM, Greg Whynott <greg.whynott@gmail.com> wrote:
None the less I didn't intend it for public consumption at this time. I'm so embarrassed and waiting for Robs words of wisdom. :/

I'm not the most pleasant person to deal with when I don't think I'm getting g the service we are paying for, but they should suck it up. Especially after the message they sent us. I tried working with them for a long time but it's very difficult. I'm done with them. Can't believe I endorsed them at the onset.
G

On Jul 17, 2017, at 13:50, Brian Krusic <brian@krusic.com> wrote:

Hi,
Eh no worries Greg.
To me it was straight forward, no profanities of unprofessionalism.
In fact, a rather valuable warning to the rest of us in terms of choosing vendors.
- Brian
Every day is a gift Thats why they call it the present"
On Jul 17, 2017, at 10:44 AM, greg whynott <greg.whynott@gmail.com> wrote:
well now. another in a long list of idiot moves on my end. Thought that reply was going direct to Ben.. *sigh*... unprofessional. I am going to go find a job as a cook now...

greg


On Mon, Jul 17, 2017 at 1:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
long story Ben. The coles notes version:

1. inital deployment (1.5 PB, 4 node cluster) was designed by Nexenta and a BOM given to Dell, fully certified.
2. migrated produciton data to it, flipped it into production.
3. later that day, it failed.
4. these failures continued every few hours.
5. sent people home for *days*.
6. for the next few weeks continued to send people home and deal with data/system recovery.
7. finally after a few months things settle down, continue to have odd permission and ownership issues.
8. forced to use buggy NFS4 to be 'supported' in the permissions area...
9. switched back to NFS3 because of many NFS4 related issues, many work around had to be implemented, such as a dedicated server that did nothing but check permissions after each job completed on farm.

fast forward a year.

10. Nexenta tells us we should upgrade and pushes us to do it. We do.
11. install 4.x, up from 3.5. Instantly notice fail over issues.
12. Nexenta on phone says "you have to remove your SAS switches to be supported'

this is about 46,000 dollars in investment, and removes our HA / mulitpathing - which was what the system was designed for. We complain, they shrug their shoulders.

Removing these switches also means we can't use all our storage as we have more JBODs than SAS ports, and we can't be dual connected anymore. This is another 120 k in hardware unusable, so now we are up to 160k in hardware we bought on Nexenta's recommendation, which they decided to not support a year or so later.

I lose my shit on how lackadaisical and unconcerned they seem to be about it. Now they get ignorant and stop responding to tickets, sometimes 4 days would go by without an update. I lose my shit more, this system was in production, then on thier recomened we upgrade it. Now they say its "not supportable and we will not help you till you remove those parts". THEY TOOK IT OUT OF A SUPPORTED STATE, not us... And because its not "supportable' they figure they don't have to escuate anything. Again, its been down for a MONTH now, it was in production.

Then to kick us in the nuts again, we have over 45,000 in enterprise SSD drives, "you need to remove those as they are not qualified on this revision". The battle continues. We finally were "approved" for support last week, and the first ticket I opened (fail over marks all disks as bad) at high priority (its been a fucking month now!!!!) still has yet to be resolved. its been open since last week.


the company just sucks, from front to back... There isn't an exec or manager who has worked there for more than 2 years, most 1 year if you look at linkedin. That is a clear indication of the mind set there... They are a horrible collection of idiots. They only people with a clue are the support staff.

We are considering suing them. Its with legal now.

greg






On Sun, Jul 16, 2017 at 9:52 AM, Ben De Luca <bdeluca@gmail.com> wrote:
what happened spill the beans? Can you or does your contract say you cant?
On 14 July 2017 at 01:38, <emailmesiddiqui@gmail.com> wrote:
Hi Greg,
We've been using Nexenta for the last few years on Dell 3060e jbods with dual DellPoweredge 720 heads.
It's been running fine perfomance-wise for us on version3.1.6-FP4 as our primary storage - we've updated and expanded once.

It's not cheap or polished and support has been variable. Sometime very good - sometimes very not good.
We've heard about one or two upgrade/support horror stories about version 4 - so decided not to go there.
Would be really good to speak to you as we're due a refresh.
Sounds like you've been through the wringer with them, thanks for flagging this!
Cheers,George

On Thu, Jul 13, 2017 at 6:42 PM, greg whynott <greg.whynott@gmail.com> wrote:
I just wanted to make sure whoever may be considering them speak to me first before buying any Nexenta solutions. They totally fucked us over very very badly for extended periods of time and money. I'd hate for anyone to go through what happened here, even if I didn't like them very much.
greg

On Thu, Jul 13, 2017 at 11:40 AM, Brian Krusic <brian@krusic.com> wrote:
Hi,
Yes, I think ppl either went Fee - Oracle or Free - FreeBSD/Linux.
I chose mostly ZFS on FreeBSD with a smidgen of ZoL.
- Brian
Every day is a gift Thats why they call it the present"
On Jul 13, 2017, at 6:35 AM, Peter Smith <peter.smith@framestore.com> wrote:
/r/DataHoarder/

On 13 July 2017 at 08:17, Peter Smith<peter.smith@framestore.com>wrote:
I know one single person who's had decent luck with Nexenta.

Everyone else ...

On 12 July 2017 at 21:23, greg whynott<greg.whynott@gmail.com>wrote:
Good day,
Are you or someone you know using or considering a solution from Nexenta? If so what do you think of your experience this far?
greg

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--
FramestorePeter Smith Senior Systems Engineer
LondonNew YorkLos AngelesChicagoMontral
T+44 (0)20 7344 8000 M+44 (0)7816 123009
19-23 Wells Street, London W1T 3PQ
TwitterFacebookframestore.com
https://www.framestore.com/



--
FramestorePeter Smith Senior Systems Engineer
LondonNew YorkLos AngelesChicagoMontral
T+44 (0)20 7344 8000 M+44 (0)7816 123009
19-23 Wells Street, London W1T 3PQ
TwitterFacebookframestore.com
https://www.framestore.com/
To unsubscribe from the list send a blank e-mail tomailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from Greg Whynott @ July 17, 2017, 5:40 p.m.
We are very much considering re-using the hardware if we can't sort something out soon. The intention is to use this for nearline and VTL storage predominately.

I'd very much like to have HA between hosts, but I think for this role we can go without. I'll take 2 of the 4 heads and add them to our Proxmox VM cluster, they are each 390 gigs 24 physical core machines. The other idea is to turn all the JBODS into VTL targets directly connected to the backup server via SAS. Or maybe a Ceph file server?

ScaleIO is a dell product correct? I've not read up on it yet but will. Is that re-branded complelent?

-greg




On Mon, Jul 17, 2017 at 5:22 PM, Brian Krusic <brian@krusic.com> wrote:
Wow, seems to have done all the work for us.
All one has to do is read and follow directions. Nice link!

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:12 PM, William Sandler <william.sandler@allthingsmedia.com> wrote:

If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore. Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct, they did learn. there is an untold amount of money loss, both tangible and intangible. When shots took 40 minutes to open instead of under a minute, that causes the artists to become frustrated, and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'... How can you be creative when your pissed off and stressed out you may not make the deadline? This has a trickle down effect that we could talk for an hour about... reduced confidence, artists start saving work outside of the pipeline to avoid the slowness, and so on and so on. could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective, and the reps are very professional. Its expensive, but it works as advertised. We wouldn't of had to buy the Isilon if the Nexenta was able to deliver. So in my eyes those costs are on Nexenta as well. The company would of gone under if we tried to stick it out in my opinion. Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution. The Isilon just works.


Make cheap, pay twice.

greg






On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg. I know you had an uphill battle to fight on new storage infrastructure. One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.
Cheers,

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDSsolution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.


- Brian

Every day is a gift
Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from William Sandler @ July 17, 2017, 5:15 p.m.
If you want to repurpose the hardware without spending much more money and don't mind getting your hands dirty https://github.com/ewwhite/zfs-ha/wiki .


William Sandler
All Things Media, LLC
william.sandler@allthingsmedia.com
On Mon, Jul 17, 2017 at 5:09 PM, Brian Krusic <brian@krusic.com> wrote:
That's a nice idea.
Seems like a lot of good hardware.

- Brian
Sent from my outstanding iPhone 6 on the craptaculous AT&T network.
On Jul 17, 2017, at 2:05 PM, Todd Smith <todd@sohovfx.com> wrote:

Glad to hear this isn't your production storage array anymore. Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct, they did learn. there is an untold amount of money loss, both tangible and intangible. When shots took 40 minutes to open instead of under a minute, that causes the artists to become frustrated, and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'... How can you be creative when your pissed off and stressed out you may not make the deadline? This has a trickle down effect that we could talk for an hour about... reduced confidence, artists start saving work outside of the pipeline to avoid the slowness, and so on and so on. could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective, and the reps are very professional. Its expensive, but it works as advertised. We wouldn't of had to buy the Isilon if the Nexenta was able to deliver. So in my eyes those costs are on Nexenta as well. The company would of gone under if we tried to stick it out in my opinion. Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution. The Isilon just works.


Make cheap, pay twice.

greg






On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg. I know you had an uphill battle to fight on new storage infrastructure. One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.
Cheers,

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDSsolution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.


- Brian

Every day is a gift
Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from Todd Smith @ July 17, 2017, 5:10 p.m.
Glad to hear this isn't your production storage array anymore.  Have you thought about repurposing that Dell storage/cpu to something like ScaleIO?
Cheers,
Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "greg whynott" <greg.whynott@gmail.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 4:46:12 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS solution from them?

Thanks Todd,

you are correct,  they did learn.   there is an untold amount of money loss,  both tangible and intangible.   When shots took 40 minutes to open instead of under a minute,  that causes the artists to become frustrated,  and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'...   How can you be creative when your pissed off and stressed out you may not make the deadline?     This has a trickle down effect that we could talk for an hour about...  reduced confidence,  artists start saving work outside of the pipeline to avoid the slowness,  and so on and so on.  could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective,  and the reps are very professional.  Its expensive, but it works as advertised.   We wouldn't of had to buy the Isilon if the Nexenta was able to deliver.   So in my eyes those costs are on Nexenta as well.    The company would of gone under if we tried to stick it out in my opinion.   Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution.   The Isilon just works.


Make cheap,  pay twice.

greg




  

On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg.  I know you had an uphill battle to fight on new storage infrastructure.  One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.  
Cheers,

Todd Smith
Head of Information Technology
soho vfx | 40 Hanna Ave. Suite 403, Toronto, Ontario M6K 0C3
office: (416) 516-7863 fax: (416) 516-9682 web: sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDS        solution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made,  it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.

 
- Brian

Every day is a gift
        Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

0 Plus One's     0 Comments  
   

Response from Greg Whynott @ July 17, 2017, 4:50 p.m.
Thanks Todd,

you are correct, they did learn. there is an untold amount of money loss, both tangible and intangible. When shots took 40 minutes to open instead of under a minute, that causes the artists to become frustrated, and more rushed to turn in their work on time resulting in less creative effort and 'pickiness'... How can you be creative when your pissed off and stressed out you may not make the deadline? This has a trickle down effect that we could talk for an hour about... reduced confidence, artists start saving work outside of the pipeline to avoid the slowness, and so on and so on. could talk about that for a long time.


We as of about 1 year ago are the proud owners of a good sized Isilon system and from day one till today we have had ZERO ISSUES with it from both stability and performance perspective, and the reps are very professional. Its expensive, but it works as advertised. We wouldn't of had to buy the Isilon if the Nexenta was able to deliver. So in my eyes those costs are on Nexenta as well. The company would of gone under if we tried to stick it out in my opinion. Its such a joy not having to worry if you are going to get a call at night or walk into another shit storm in the morning because of your storage solution. The Isilon just works.


Make cheap, pay twice.

greg






On Mon, Jul 17, 2017 at 4:34 PM, Todd Smith <todd@sohovfx.com> wrote:
I feel for you Greg. I know you had an uphill battle to fight on new storage infrastructure. One glimmer of hope may be that given the immense cost and pressure the company faced due to storage infrastructure issues, the ownership may be more lenient when given advice from IT professionals.
Cheers,

Todd Smith
Head of Information Technology
soho vfx|40 HannaAve. Suite 403, Toronto, Ontario M6K0C3
office:(416) 516-7863fax:(416) 516-9682web:sohovfx.com

From: "Brian Krusic" <brian@krusic.com>
To: "studiosysadmins-discuss" <studiosysadmins-discuss@studiosysadmins.com>
Sent: Monday, July 17, 2017 3:36:51 PM
Subject: Re: [SSA-Discuss] Nexenta - Anyone still using or considering a SDSsolution from them?

> On Jul 17, 2017, at 12:20 PM, greg whynott <greg.whynott@gmail.com> wrote:
>
> So it was _not_ a choice I made, it was made for me based on economics and nothing to do with experience or technology.

Well my find feathered friend, isnt that how its some what done for most of us?

Depends on framing, i.e.; 1/2-empty or 1/2-full.

My choices are at least 1/2 based/influenced on econ.

I think that in your case, an Oracle-ZFS solution would be a very very good eval option.


- Brian

Every day is a gift
Thats why they call it the present"
To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from Greg Whynott @ July 17, 2017, 3:25 p.m.
Thanks Bobby. :)

On Mon, Jul 17, 2017 at 2:05 PM, Bobby Jain <bobsf40@gmail.com> wrote:
Sales sucked, support sucked, platform was buggy. I did not mix SMB and NFS, and avoided most of the permissions problems others experienced (maybe the one positive story in Toronto).
On Mon, Jul 17, 2017 at 1:50 PM, Brian Krusic <brian@krusic.com> wrote:
Hi,
Eh no worries Greg.
To me it was straight forward, no profanities of unprofessionalism.
In fact, a rather valuable warning to the rest of us in terms of choosing vendors.
- Brian
Every day is a gift Thats why they call it the present"
On Jul 17, 2017, at 10:44 AM, greg whynott <greg.whynott@gmail.com> wrote:
well now. another in a long list of idiot moves on my end. Thought that reply was going direct to Ben.. *sigh*... unprofessional. I am going to go find a job as a cook now...

greg


On Mon, Jul 17, 2017 at 1:39 PM, greg whynott <greg.whynott@gmail.com> wrote:
long story Ben. The coles notes version:

1. inital deployment (1.5 PB, 4 node cluster) was designed by Nexenta and a BOM given to Dell, fully certified.
2. migrated produciton data to it, flipped it into production.
3. later that day, it failed.
4. these failures continued every few hours.
5. sent people home for *days*.
6. for the next few weeks continued to send people home and deal with data/system recovery.
7. finally after a few months things settle down, continue to have odd permission and ownership issues.
8. forced to use buggy NFS4 to be 'supported' in the permissions area...
9. switched back to NFS3 because of many NFS4 related issues, many work around had to be implemented, such as a dedicated server that did nothing but check permissions after each job completed on farm.

fast forward a year.

10. Nexenta tells us we should upgrade and pushes us to do it. We do.
11. install 4.x, up from 3.5. Instantly notice fail over issues.
12. Nexenta on phone says "you have to remove your SAS switches to be supported'

this is about 46,000 dollars in investment, and removes our HA / mulitpathing - which was what the system was designed for. We complain, they shrug their shoulders.

Removing these switches also means we can't use all our storage as we have more JBODs than SAS ports, and we can't be dual connected anymore. This is another 120 k in hardware unusable, so now we are up to 160k in hardware we bought on Nexenta's recommendation, which they decided to not support a year or so later.

I lose my shit on how lackadaisical and unconcerned they seem to be about it. Now they get ignorant and stop responding to tickets, sometimes 4 days would go by without an update. I lose my shit more, this system was in production, then on thier recomened we upgrade it. Now they say its "not supportable and we will not help you till you remove those parts". THEY TOOK IT OUT OF A SUPPORTED STATE, not us... And because its not "supportable' they figure they don't have to escuate anything. Again, its been down for a MONTH now, it was in production.

Then to kick us in the nuts again, we have over 45,000 in enterprise SSD drives, "you need to remove those as they are not qualified on this revision". The battle continues. We finally were "approved" for support last week, and the first ticket I opened (fail over marks all disks as bad) at high priority (its been a fucking month now!!!!) still has yet to be resolved. its been open since last week.


the company just sucks, from front to back... There isn't an exec or manager who has worked there for more than 2 years, most 1 year if you look at linkedin. That is a clear indication of the mind set there... They are a horrible collection of idiots. They only people with a clue are the support staff.

We are considering suing them. Its with legal now.

greg






On Sun, Jul 16, 2017 at 9:52 AM, Ben De Luca <bdeluca@gmail.com> wrote:
what happened spill the beans? Can you or does your contract say you cant?
On 14 July 2017 at 01:38, <emailmesiddiqui@gmail.com> wrote:
Hi Greg,
We've been using Nexenta for the last few years on Dell 3060e jbods with dual DellPoweredge 720 heads.
It's been running fine perfomance-wise for us on version3.1.6-FP4 as our primary storage - we've updated and expanded once.

It's not cheap or polished and support has been variable. Sometime very good - sometimes very not good.
We've heard about one or two upgrade/support horror stories about version 4 - so decided not to go there.
Would be really good to speak to you as we're due a refresh.
Sounds like you've been through the wringer with them, thanks for flagging this!
Cheers,George

On Thu, Jul 13, 2017 at 6:42 PM, greg whynott <greg.whynott@gmail.com> wrote:
I just wanted to make sure whoever may be considering them speak to me first before buying any Nexenta solutions. They totally fucked us over very very badly for extended periods of time and money. I'd hate for anyone to go through what happened here, even if I didn't like them very much.
greg

On Thu, Jul 13, 2017 at 11:40 AM, Brian Krusic <brian@krusic.com> wrote:
Hi,
Yes, I think ppl either went Fee - Oracle or Free - FreeBSD/Linux.
I chose mostly ZFS on FreeBSD with a smidgen of ZoL.
- Brian
Every day is a gift Thats why they call it the present"
On Jul 13, 2017, at 6:35 AM, Peter Smith <peter.smith@framestore.com> wrote:
/r/DataHoarder/

On 13 July 2017 at 08:17, Peter Smith<peter.smith@framestore.com>wrote:
I know one single person who's had decent luck with Nexenta.

Everyone else ...

On 12 July 2017 at 21:23, greg whynott<greg.whynott@gmail.com>wrote:
Good day,
Are you or someone you know using or considering a solution from Nexenta? If so what do you think of your experience this far?
greg

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--
FramestorePeter Smith Senior Systems Engineer
LondonNew YorkLos AngelesChicagoMontral
T+44 (0)20 7344 8000 M+44 (0)7816 123009
19-23 Wells Street, London W1T 3PQ
TwitterFacebookframestore.com
https://www.framestore.com/



--
FramestorePeter Smith Senior Systems Engineer
LondonNew YorkLos AngelesChicagoMontral
T+44 (0)20 7344 8000 M+44 (0)7816 123009
19-23 Wells Street, London W1T 3PQ
TwitterFacebookframestore.com
https://www.framestore.com/
To unsubscribe from the list send a blank e-mail tomailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments