Did you notice that a few weeks ago, AWS changed the web pages that describe their instance types? Specifically, the new AWS instance type page prominently features the new 3 series announced at re:Invent last November and minimizes what are now being called “Previous Generation Instances”. Those older instance types now have a separate page that encourages upgrading to the newer ones. And why wouldn’t you want to upgrade? With Intel Ivy Bridge processors and SSD storage, the new 3 series should perform much, much better than their predecessors.
But how much better?
That’s what we wanted to find out.
CPU Benchmark: Previous Generation vs 3 Series
We were most interested in trying to isolate the processor speed differences between the two generations of AWS instance types. To do that, we decided to run a high CPU load against each so we created a CliQr Application Profile for Blender, the popular open source graphics rendering platform, which takes a set of 5 .blend files describing renderings of houses and generates a series of digital drawings (you can run this yourself here for free). We ran this Application Profile on on 8 different AWS instance types (4, 8, 16, and 32 vCPU for both older and newer instances). While we left GPU testing for another day, this load is CPU intense and pushed the limits of each processor running 5 jobs on each instance type simultaneously.
Here are the instance types we chose:
|Instance Name||vCPU||RAM (GB)||Storage (GB)||Hourly Price|
The intent here was to match the number of vCPU on the different generations. That proved slightly problematic for the 16 vCPU case as the hi1.4xlarge is the only Previous Generation instance type that meets that description. Even after recent price drops, the hi1.4xlarge is a little more expensive given its large disk but for our purposes it was close enough.
This same test was conducted on three consecutive days for each instance type, with the best result among the three being recorded. All tests were run in the AWS US West (Oregon) region. How did each instance type do?
Results: Previous Generation vs 3 Series
First graphically (3 series instances in orange and Previous Generation in black), then in tabular form, here are our results:
|Instance Name||vCPU||RAM (GB)||Storage (GB)||Hourly Price||Job Time (minutes)|
Conclusions: Previous Generation vs 3 Series
If you have a need for speed, the 3 series instance types are unquestionably faster. In our tests they were up to 29% faster compared to their Previous Generation counterparts. The difference isn’t as great for this workload for 32 vCPU instance types, but that may be influenced by how Blender tasks were run in groups of 5 instead of larger groups. At the 4 and 8 vCPU comparison, the improved performance was substantial.
However, price is still an important factor as well. For a batch job like what we tested here, unless you are using Spot Instance pricing you pay for the whole hour of instance use regardless of how quickly it completes its task. The Previous Generation instances are roughly 15% cheaper than their shiny new siblings and for certain workloads that can make a significant impact on your bottom line.
As always, CliQr highly recommends you benchmark your actual workloads on multiple instance types, and even on multiple clouds, to determine the right fit for you. Every organization has different priorities and interpretations of price-performance ratios will vary accordingly. By testing your actual workloads regularly using a toolset like CliQr, as prices change and new instance types get introduced you put all the right data at your disposal so you can make as informed a decision as possible.