Completed on August 2007, duration: 1 year

Executive Summary

We have identified opportunity for cost-effective way to reduce provisioning times for new development environments while cutting our operational costs.

My roles

Project manager:

  • Project planning
  • Team management
  • Communication with key stakeholders
  • Preparing and approving budgets

Project sponsor

  • Providing visionary input for team

Elaboration phase

Business case for the project was supported by the following facts:

  • Virtualization technology became mature enough for production workloads
  • Most of hardware servers were idle most of the time
  • Long provisioning times for development environments
  • Growing power, cooling and maintenance costs

Exisitng server infrastructure for the 80% consisted of Linux machines of different computing power and capacity.

High-level requirements were:

  • Reduce time to provision new development environment from weeks to hours
  • Reduce number of hardware servers

Project constraints:

  • Security shouldn’t be sacrificed
  • Limited budget

Project plan and scope definition

Following items were put in scope for our team:

  • Assess available solutions
  • Evaluate shortlisted solutions
  • Approve selected solution with management
  • Upgrade hardware servers where needed
  • Migration of exisiting development environments to VMs

Following items were in scope of Client’s application vendor:

  • Application setup on DR site
  • Data transfer from Primary site

Solution space assesment

We have identified following options:

XEN

Shortlisted due to following advantages:

  • Open-Source solution
  • Ability to run Windows workloads (in addition to Linux)
VMware

Despite availability of a free hypervisor (ESXI) it wasn’t shortlisted due to:

  • Not an open-source
  • Free version had some features disabled
  • License required for central management
OpenVZ

Shortlisted due to following advantages

  • Open-source solution
  • Lowest resource overhead
Virtual Server 2005

Microsoft’s “Mee too” take on free hypervisor. While it was relatively new player, we decided to keep it on shortlist due to remaining 20% of Windows servers as it allowed to also save on licenses.

KVM

Rejected from shortlist as not mature enough at the moment of evaluation

Design choices

Virtualization technology

Considering limited budget (and therefore resources) solution choice was pretty obvious - we decided to go with a mix of OpenVZ for all our Linux environments and Virtual Server 2005 for Windows (switched to Hyper-V when become available).

Over-subscription

Considering most of our workloads were Java-based we were able to leverage RAM over-subscription and double average number of VMs per hardware server.

Challenges

Budget limitations

As our budget was very limited we had to find a way to deliver more with less. After successfull pilot & obtaining approval from the management, we started to consolidate hardware resources at hand - taking two average servers to build one more powerfull and discard the other. OpenVZ was best fit for that kind of approach as it provided minimum overhead on virtualizing workloads and played well with over-subscription for our scenario.

Achievements

Reduced operational costs

We reduced number of physical servers by 30%, effectively cutting 30% off our power, cooling and maintenance costs.

This also reduced our costs on relocation to new office in 2007, as we had to move less hardware.

Spare servers

By the end of the project we ended up with few hardware servers (pretty good specification) we had to turn off as we have nothing to run there. We never been in such a good shape before. This allowed us to keep our bottom-line tight during 2008’s crisis, while our peers in other companies suffered heavily from budget cuts.

P/V ratio

We have easily achived industry average (at that time) ratio of physical-to-virtual servers 1:10. This allowed us to support Company’s growth and continue to provide development environments while reducing number of physical servers.