Renderfarms: raw performance, or performance per watt?

[Roger]

I'm nowhere near ready to render anything for my movie, but I have been trying to figure out what makes the most sense for a renderfarm: raw performance or performance per watt?

 

Hypothetical renderfarm 1: Xeon-based renderfarm with high GHz high-dollar Xeon cpus (say, 4 8-core Xeons on a quad socket motherboard). Total cost: Probably in the range of $3000-$4500

 

Hypothetical renderfarm 2: Lower performance AMD cpus with a much lower power requirement (the 25w Kabini quad core at 2ghz comes to mind) --- for an equivalent number of cores, you could do 8 motherboards each with one 25w cpu for a total of 200 watts, compared to 500 watts for the Xeon system (minimum). Total cost: maybe $800-$1000 max?

 

I think the Kabini APU/CPU performs somewhere in the neighborhood of a Core i3 on the Intel side, however if the AMD-based rendering farm only costs $20/month to run 24/7 where the Xeon costs closer to $50-$70.

 

My thinking is even if the AMD cpu is 20-30% slower at a given clock rate, that may be offset by the cost savings of being able to run the farm 24/7 at a lower cost.

 

I'm just sort of thinking out loud at this point, as I'm nowhere near the point of being ready for final rendering of any kind, but I'd be interested if there is anyone out there with experience in designing a system solely for rendering what you considered more important: absolute screaming performance, or performance at a more reasonable price/operating cost?

[robcat2075]

my thoughts...

 

-I think you'd want to know their throughput on frames/time. Clock rate may not be good indicator anymore.

 

-absolute screaming performance tends to be way over-priced

 

-absolute screaming performance tends to be more needed for real-time tasks like image editing, where the operator is waiting for one step to get done before he does the next one.

 

-electricity is so cheap that you'd need a lot of hardware before it became a real concern.

[Roger]

my thoughts...

 

-I think you'd want to know their throughput on frames/time. Clock rate may not be good indicator anymore.

 

-absolute screaming performance tends to be way over-priced

 

-absolute screaming performance tends to be more needed for real-time tasks like image editing, where the operator is waiting for one step to get done before he does the next one.

 

-electricity is so cheap that you'd need a lot of hardware before it became a real concern.

 

Yeah, I figure I'd have to do extensive testing with 2 different test systems to get any sort of true picture of their relative performance.

Electricity is cheap but with a system that draws a half kilowatt at full load, you're talking about 12 kilowatt hours per day, or about $1.20 per day to run, or roughly $30-$35 per month. I suppose you'd have to have 3 or 4 systems running full-blast all the time like that before it started to get too crazy.

 

I suspect the best bang for the buck would be with Xeon cpus from a generation or two ago. These would be cheap enough that the power would probably end up being more expensive. I know that 6 core Xeon 5670 cpus can be found for about $100 or so, compared to modern 12 core cpus that go for over $1500, you could easily pick up 2 of the older model. I don't think the instruction set could have changed that much in 6 years. You could not say the same about a 486 system from 1993 and a Pentium III from 1999. I can't easily get my hands on a system with a modern 12-core xeon to confirm my suspicions, though. Does anyone out there have one of the new Macs with the 10 or 12 core cpu?

[Shelton]

Robert and Roger

 

I have set up and dismantled and rebuilt three different render farms. Believe it or not an i5 4700 series processor runs great for the money. Jason1025 has a new mac pro and i think i had just as good of results with an older intel 920 I7. I have 3 amd bulldozer chips and above 8 core systems and they did good but an older 6 core 1190t amd outperformed the 8 cores. All of the zeon systems i have were 4 and 6 core and performed about the same as the 8 core amd fx chips. I guess what i am saying, do not overspend for wizbang because cost per performance is not there in my opinion. The important thing is memory in the individual machine. I have built all my machines on the standard of 4 gigs per core.

 

 

Here what i have for my current farm. For am server i have a dell poweredge 300t 24 gig system. It has 4 generation zeon 2.5 ghz. Intel 5930 6 core 32gb, i7 920 24 gb Four intel I5 4700 4 core 16 gb of ram. Three amd fx 8350 8 core 32 gb ram, amd 1190t 6 core with 32 gb of ram, which give me currently 56 cores. Now i still have 6 dell modified 4 core poweredge 840 that have second gen zeons, hand full of core 2 duo quads that still perform 85 of the i5. Any way if i had the upgrade am i could have close 100 cores in the farm.

 

The next thing that really needs to happen is some fleshing out of netrender as there are still some problems.

 

Also look at gpu to make sure it supports open gl 3 and above

 

Really just as important is the backbone for the render farm, gig switches a must and the network running with the basics.

 

As i ramble along if you have the money the super system would be great, but look at how much you will use the farm and your cost to get there. I sold a great deal of my last setup after rendering three projects and watching it sit for a year. I had purchased alot of nice equipment and did not get much use from it.

[Roger]

Steve,

 

If you don't mind my asking, what was your electric cost for running your farm?

[John Bigboote]

I have recently been blown-away by A:M's render=nodes and 'farm' software- NetRender. I bought 12 'render nodes' from Hash for some 'scandalously' low price... like $3 per or something... and I am BY NO MEANS a render farm or IT persona... at all. The ease of use(installation) and rendering speed increase is staggering... it really keeps A:M current in this highly evolved 3D world. I have 6 dual-core processors on a 4 year old BOXX so I render on 10 cores... and can still use After Effects with a little lag. I have NO IDEA how fast the wheel is turning on the electric meter outside and never considered it as a factor.