Colossus III [Pictures start on Page 2]
Posted: Sun Jan 27, 2013 11:53 am
It is time to update the Colossus, our server.
Now that the population on the server is growing we have been seeing some performance degradation. Mostly due to the CPU. It has become a bottleneck.
As some of you know we do actually have one of the fastest Minecraft servers available. If I had to rate us I'd say we are probably in the top 50 for hardware specifications. But we run a lot of stuff on this server and so the amount of performance dedicated to just Minecraft is lower than the other servers.
So here is our current specification:
CPU: Core i7 940
RAM: 24GB 1333MHz Memory (6x4GB Modules)
Mobo: Asus P6T6 WS Motherboard (Socket 1366, X58 Chipset)
OCZ Vertex 2 SSD (Just for Minecraft)
78TB's of storage with 1.05TB dedicated to just Minecraft backups.
Now the specs here sound good right? Core i7 thats high end. But this is a Core i7 940. This CPU actually came out in 2008. It will be 5 years old in October this year. Put simply the newer chips are much faster.
Another concern is the Vertex 2 SSD, this SSD is not SATA III, it has low IOP's and basically it's just junk. Now it is much better than a hard disk for performance. But its reliability is lower and its performance compared to other solid state drives is very low.
And this is before we get in to all the stuff we are running on this server.
VMWare Workstation with an OS X VM and four Windows XP VM's (This is for development work I do away from Minecraft).
A webserver serving upwards of 50 thousand requests per day
Three WinMX chat rooms with combined 200 users connected
Three separate Minecraft servers
Two separate webservers for DynMAP (Yes these are actually run as their own webservers away from the normal webserver on port 80)
uTorrent with 6000+ torrents seeding 24.7
All our personal storage of our Music, TV Shows, Movies and live streaming & Transcoding to our whole house
Scheduled automated backups of mine and my wifes Notebooks and my Desktop.
So with all this in mind it is time to do a complete system update. Over the past few years I have done mild updates in that I've added a new RAID card, I've added 4TB Disks and I've switched the Motherboard and CPU after I upgraded my desktop. But the architecture of the system has remained the same. I changed it from the Core i7 920 to the Core i7 940. That is just a 266MHz increase in CPU speed. There is no other difference they are from the same series so the performance improvement was about 5%.
What I'm intending to do next month is begin to transition to an entirely new server. New CPU, New Motherboard, New RAM, New SSD.
To do this we are going to move to the X79 platform. This is the latest and greatest platform from Intel for Servers, Workstations and High End Gaming rigs. It uses the same Chipset that is found in my gaming rig which you've all seen on the forum before.
And the chip we are going to be using is the Core i7 3930K. The same processor that is in my desktop. Now there are a few reasons for picking this chip over a XEON and I just want to go over that.
1. XEON's for the same clock speed as a 3930K cost 2.5x as much.
2. We are not going to be using ECC memory anyway so using a XEON would be a waste
3. Both the XEON and the 3930K have the same architecture meaning the same performance per clock
4. We can overclock the 3930K to 4.2GHz on Air without affecting power consumption much.
Now why the Core i7 3930K and not the Core i7 3820? Well the 3820 is a great chip yes. But it is only a four core chip. And really if I'm going to upgrade the whole system I don't want to replace one four core chip with another four core chip. The performance improvements while there wouldn't be as great as if we go to a six core processor like the 3930K. And instead of buying the 3820 now and then upgrading in 6 months time I've decided to simply invest in the 3930K on day one. It's almost twice as much money but it's also a good 50% faster than the 3820 which is already about 50% faster than our current Core i7 940. Theoretically if we used every core on the chip we are getting a 100% performance boost. And this is before I overclock the 3930K 1GHz higher than it already is.
So that is the system and the chip. What about Memory? Well as you know we currently have 24GB of memory. This is also becoming a bottleneck as we gain players and I do more things on the server the RAM is getting filled. It is not uncommon for Survival to use up all its memory and then the server starts to lag. It has happened 3 or 4 times this past week and I've been redistributing where the RAM is allocated to give Survival more leg room. For the new server we are going to push through the 32GB barrier and have 64GB's of memory. I'm intending to use either 1333MHz or 1600MHz memory. The speed isn't too important because the X79 platform is setup in Quad Channel which means it has twice the amount of memory bandwidth over Dual Channel systems like socket 1155.
What this means is, if you used four sticks of 1333MHz in a 1155 system you get the memory bandwidth of 2666MHz memory (1333 x Dual Channel = 2666MHz). But in a 2011 system that same speed would give you 5332MHz of bandwidth. (1333 x Quad Channel = 5332MHz). So the memory speed isn't so important the platform is designed for extremely high throughput.
So that's RAM. What about the Solid State Drive? Well traditionally we have always used OCZ because frankly OCZ is always the cheapest and I didn't really want to spend too much money on a dedicated SSD just for our Minecraft server. But when picking the SSD I did go for the mid-range Vertex series and not the super cheap ones. However the OCZ SSD's are junk, they are unreliable they break fast so we are not going with an OCZ again.
Instead I have decided to buy the best SSD on the market by reviews. It is pricey but it is worth it. We are going to be using the Corsair Neutron GTX 120GB. This SSD has 80,000 IOP's when writing and 85,000 IOP's when reading (4K Blocks) this is 4x higher simultaneous input / output operations per second than our current SSD. It uses the brand new LAMP controller which is a very high end SSD controller usually found in enterprise level SLC NAND SSD's. This means it's very reliable and incredibly fast. Warping, Teleporting, Flying should all be greatly improved by this SSD.
It is also twice as big as our current SSD. We are using right now a 64GB SSD and it has about 25GB of free space. The new SSD is 120GB which will give us a lot more room to grow. It is also SATA III and as our new X79 platform uses a SATA III compatible chipset we will be able to take full advantage of the SSD's throughput. This is something we couldn't do before because even though our current Vertex SSD is SATA III the motherboard in the system is only SATA II.
Our current SSD will still be kept inside the system just not active. It will be used in the event that the Corsair were to fail for any reason. Which reminds me, the Corsair SSD has a 5 year warranty making it a solid investment.
And so that is the system the chip the memory and the storage.
To recap:
Intel LGA 2011 X79 Platform
Core i7 3930K 6 Core / 12MB Cache CPU
64GB of DDR3 Memory
Corsair Neutron GTX 120GB SSD
This will be coupled with our current HX 1000 Corsair PSU, dedicated battery UPS, Asus AC66U Router, 78TB's of storage in our lovely Lian Li PC-343B Case.
When I start getting the parts I will be doing a build log in this thread so stay tuned for that.
Now that the population on the server is growing we have been seeing some performance degradation. Mostly due to the CPU. It has become a bottleneck.
As some of you know we do actually have one of the fastest Minecraft servers available. If I had to rate us I'd say we are probably in the top 50 for hardware specifications. But we run a lot of stuff on this server and so the amount of performance dedicated to just Minecraft is lower than the other servers.
So here is our current specification:
CPU: Core i7 940
RAM: 24GB 1333MHz Memory (6x4GB Modules)
Mobo: Asus P6T6 WS Motherboard (Socket 1366, X58 Chipset)
OCZ Vertex 2 SSD (Just for Minecraft)
78TB's of storage with 1.05TB dedicated to just Minecraft backups.
Now the specs here sound good right? Core i7 thats high end. But this is a Core i7 940. This CPU actually came out in 2008. It will be 5 years old in October this year. Put simply the newer chips are much faster.
Another concern is the Vertex 2 SSD, this SSD is not SATA III, it has low IOP's and basically it's just junk. Now it is much better than a hard disk for performance. But its reliability is lower and its performance compared to other solid state drives is very low.
And this is before we get in to all the stuff we are running on this server.
VMWare Workstation with an OS X VM and four Windows XP VM's (This is for development work I do away from Minecraft).
A webserver serving upwards of 50 thousand requests per day
Three WinMX chat rooms with combined 200 users connected
Three separate Minecraft servers
Two separate webservers for DynMAP (Yes these are actually run as their own webservers away from the normal webserver on port 80)
uTorrent with 6000+ torrents seeding 24.7
All our personal storage of our Music, TV Shows, Movies and live streaming & Transcoding to our whole house
Scheduled automated backups of mine and my wifes Notebooks and my Desktop.
So with all this in mind it is time to do a complete system update. Over the past few years I have done mild updates in that I've added a new RAID card, I've added 4TB Disks and I've switched the Motherboard and CPU after I upgraded my desktop. But the architecture of the system has remained the same. I changed it from the Core i7 920 to the Core i7 940. That is just a 266MHz increase in CPU speed. There is no other difference they are from the same series so the performance improvement was about 5%.
What I'm intending to do next month is begin to transition to an entirely new server. New CPU, New Motherboard, New RAM, New SSD.
To do this we are going to move to the X79 platform. This is the latest and greatest platform from Intel for Servers, Workstations and High End Gaming rigs. It uses the same Chipset that is found in my gaming rig which you've all seen on the forum before.
And the chip we are going to be using is the Core i7 3930K. The same processor that is in my desktop. Now there are a few reasons for picking this chip over a XEON and I just want to go over that.
1. XEON's for the same clock speed as a 3930K cost 2.5x as much.
2. We are not going to be using ECC memory anyway so using a XEON would be a waste
3. Both the XEON and the 3930K have the same architecture meaning the same performance per clock
4. We can overclock the 3930K to 4.2GHz on Air without affecting power consumption much.
Now why the Core i7 3930K and not the Core i7 3820? Well the 3820 is a great chip yes. But it is only a four core chip. And really if I'm going to upgrade the whole system I don't want to replace one four core chip with another four core chip. The performance improvements while there wouldn't be as great as if we go to a six core processor like the 3930K. And instead of buying the 3820 now and then upgrading in 6 months time I've decided to simply invest in the 3930K on day one. It's almost twice as much money but it's also a good 50% faster than the 3820 which is already about 50% faster than our current Core i7 940. Theoretically if we used every core on the chip we are getting a 100% performance boost. And this is before I overclock the 3930K 1GHz higher than it already is.
So that is the system and the chip. What about Memory? Well as you know we currently have 24GB of memory. This is also becoming a bottleneck as we gain players and I do more things on the server the RAM is getting filled. It is not uncommon for Survival to use up all its memory and then the server starts to lag. It has happened 3 or 4 times this past week and I've been redistributing where the RAM is allocated to give Survival more leg room. For the new server we are going to push through the 32GB barrier and have 64GB's of memory. I'm intending to use either 1333MHz or 1600MHz memory. The speed isn't too important because the X79 platform is setup in Quad Channel which means it has twice the amount of memory bandwidth over Dual Channel systems like socket 1155.
What this means is, if you used four sticks of 1333MHz in a 1155 system you get the memory bandwidth of 2666MHz memory (1333 x Dual Channel = 2666MHz). But in a 2011 system that same speed would give you 5332MHz of bandwidth. (1333 x Quad Channel = 5332MHz). So the memory speed isn't so important the platform is designed for extremely high throughput.
So that's RAM. What about the Solid State Drive? Well traditionally we have always used OCZ because frankly OCZ is always the cheapest and I didn't really want to spend too much money on a dedicated SSD just for our Minecraft server. But when picking the SSD I did go for the mid-range Vertex series and not the super cheap ones. However the OCZ SSD's are junk, they are unreliable they break fast so we are not going with an OCZ again.
Instead I have decided to buy the best SSD on the market by reviews. It is pricey but it is worth it. We are going to be using the Corsair Neutron GTX 120GB. This SSD has 80,000 IOP's when writing and 85,000 IOP's when reading (4K Blocks) this is 4x higher simultaneous input / output operations per second than our current SSD. It uses the brand new LAMP controller which is a very high end SSD controller usually found in enterprise level SLC NAND SSD's. This means it's very reliable and incredibly fast. Warping, Teleporting, Flying should all be greatly improved by this SSD.
It is also twice as big as our current SSD. We are using right now a 64GB SSD and it has about 25GB of free space. The new SSD is 120GB which will give us a lot more room to grow. It is also SATA III and as our new X79 platform uses a SATA III compatible chipset we will be able to take full advantage of the SSD's throughput. This is something we couldn't do before because even though our current Vertex SSD is SATA III the motherboard in the system is only SATA II.
Our current SSD will still be kept inside the system just not active. It will be used in the event that the Corsair were to fail for any reason. Which reminds me, the Corsair SSD has a 5 year warranty making it a solid investment.
And so that is the system the chip the memory and the storage.
To recap:
Intel LGA 2011 X79 Platform
Core i7 3930K 6 Core / 12MB Cache CPU
64GB of DDR3 Memory
Corsair Neutron GTX 120GB SSD
This will be coupled with our current HX 1000 Corsair PSU, dedicated battery UPS, Asus AC66U Router, 78TB's of storage in our lovely Lian Li PC-343B Case.
When I start getting the parts I will be doing a build log in this thread so stay tuned for that.
