Oracle's T-Series servers support two virtualization technologies:
This article will take you through both, and help you decide whether to use ldoms, zones, or a mixture of both in your consolidated SPARC environment.
Side note: The marketing people at Oracle (and Sun) had a lot of fun confusing everyone with these features. When talking at a technical level, I prefer to use the terms "ldoms" and "zones" because those are the terms used inside Solaris. The name "Oracle VM for SPARC" is particularly misleading because it is completely different to "Oracle VM for x86", which is built on top of Xen technology.
Technology Overview
The two virtualization technologies complement each other in some ways and duplicate functionality in other ways.
Oracle VM for SPARC ("ldoms")
LDoms provide hardware-assisted separation between multiple Solaris instances on the same server.
This gives you:
- A separate network namespace for each guest (e.g. different hostnames and IP addresses)
- Separation between different guest instances for improved security
- Limits to the CPU, memory, and disk allocated to each guest
- The possibility of lower software licence costs when licences are per-CPU
- The ability to run multiple Solaris versions side-by-side on the same machine (as long as they are supported by the hardware)
- The ability to upgrade Solaris versions independently
- The ability to virtualize applications that won't run within a non-global zone (e.g. NFS servers, DHCP servers)
- The ability to configure delegated resources (such as network interfaces and filesystems) as if each guest instance were a standalone machine
- The ability to live migrate a guest instance between physical servers
Solaris Containers ("zones")
Zones provide a lightweight mechanism for separating different applications running within the same Solaris instance. A Solaris instance may be running bare-metal on the server, or within an ldom.
This gives you:
- A separate network namespace for each zone (e.g. different hostnames and IP addresses)
- Separation between different non-global zones for improved security
- Flexible control over the amount of CPU, memory, and disk allocated to each zone
- The possibility of lower software license costs when licenses are per-CPU
- The ability to run multiple Solaris versions side-by-side on the same machine (including older versions using Legacy Containers)
- The ability (and obligation) to upgrade multiple zones at the same time
Choosing LDoms or Zones (or both!)
Ease of Administration
In my experience, administrators tend to lean towards using ldoms because it is a familiar model of server virtualization: just like VMware, Hyper-V, Xen or VirtualBox. Once the hypervisor (consisting of control/service domains) is set up and resources allocated, most procedures are the same for a guest ldom as they are for a standalone server. For example, you can stop, start, install, upgrade and configure an ldom as if it were a physical server.
Administrators who are willing to invest the time and resources to gain experience with zones may discover new, more efficient ways of doing things. For example, all of the zones in a Solaris instance can be upgraded at the same time (preferably using Live Upgrade). In an environment with many zones, this can save a substantial amount of time and effort. Similarly, storage can be pooled flexibly and efficiently using ZFS and delegated to the non-global zones. These new methods don't fit the old model, but can make ongoing administration easier and faster.
Performance
The ldoms layer introduces latency and may limit throughput for for network and disk IO. There are ways around this performance penalty, such as delegating hardware devices directly into the guest ldoms, but at the cost of adding significant complexity.
In most cases an application within a zone will run just as fast as an application on an equivalent standalone server.
Resource Usage
Both zones and ldoms provide the ability to limit CPU, memory and disk usage. With ldoms, this is done by configuring a certain amount of hardware for each guest, which can mean creating silos of unused resource. Zones are much more flexible in the way this is done:
- CPU can be shared using the Fair Share Scheduler, allowing each zone to use all of the CPU on the server unless there is contention from other zones. Alternatively, CPU pools can be used to dedicate a certain amount of resource to each zone or set of zones
- Memory is shared between all zones up to configured caps
- Disk can be pooled and shared using ZFS
If hardware requirements are well-defined and isolation is important, then ldoms will provide the best solution. However, if application workloads are bursty and you want to avoid making silos of unused resource, zones will provide a better option.
When configured with dedicated CPUs (as per Oracle's instructions), both zones and ldoms can be recognised by Oracle as a hard partitioning technology. This means that if you are running Oracle software, you only need to be licensed for the number of cores (subject to the processor core factor) allocated to that zone or ldom.
Live Migration
The headline feature of ldoms is the ability to live migrate a guest instance from one physical server to another (as long as they are configured correctly using shared storage). This is a great feature for demonstrations, but not particularly useful in practice. Its main purpose would be to avoid downtime during hardware replacement, a firmware upgrade, or a migration to a new server. However, these are relatively rare events, and if your application requires such high availability you should provide it through a different mechanism such as clustering.
For one customer, I used live migration several times to avoid downtime during a series of firmware upgrades -- ironically, however, we would not have needed the firmware upgrades were the system not running ldoms in the first place.
Zones on Top of LDoms
In general I recommend you avoid using both technologies at the same time, because it increases the amount of knowledge required to operate the system effectively. For example, configuring storage for a zone within an ldom feels like passing a thread through several needles simultaneously. However, there are a few use cases:
- When you want to have a virtual server available to practice Solaris upgrades
- When you want to mainly use zones, but need to have some separate virtual environments that can't be run within a zone
- When for business reasons you want to provide a stronger level of separation between two groups of virtual machines (e.g. two departments or tenants using the same physical server)
Summary
Zones and ldoms are both useful technologies that provide separation between applications on the same physical server. For a set-and-forget environment, ldoms provides the easiest transition for administrators who are accustomed to the standard virtual machine model. On the other hand, zones provide the best performance, flexibility, and efficiency. Both can be used at the same time at the cost of added complexity.