In most new environments I like to use 802.3ad (LACP) link aggregation, also known as bonding and port-channeling.
Side note: The term "trunking" can confuse people because it means two different things. It can mean link aggregation: bundling multiple physical links to create one logical link; or it can mean VLAN tagging: sending multiple separate networks over one link. Therefore I try to avoid the the term "trunking" and instead I just talk about link aggregation or VLAN tagging.
- Link aggregation is a set-and-forget configuration; once it is configured in the service domain, you can treat the aggregated link as if it was a single interface. However, it can be difficult to get right if you have limited experience: it requires coordinated, simultaneous work between the switch administrator and the Solaris administrator. Some switches support port-channels that span over multiple switches.
- IPMP (IP MultiPathing) is a host-based configuration option on Solaris that does not require any changes on the switch side and can always be used with each port plugged into a different switch. It needs to be configure within each ldom separately. For full probe-based functionality, IPMP requires 1 IP address per interface plus 1 floating IP address per group of ports. For example, with 2 physical interfaces, the configuration would need a total of 3 IP addresses per ldom.
If you use multiple service domains then you'll need to use IPMP to provide network redundancy. Each guest domain will see two virtual network interfaces, and will need to use IPMP to provide failover and load balance between the two.
It is possible to use a combination of IPMP and link aggregation. For example, when the network switches do not support multi-switch port-channels, but you still want to avoid the switch as a single-point-of-failure, you can create a port-channel to each switch and then layer IPMP on top. However, I generally try to avoid this kind of multi-level, complicated network setup because it increases the burden on future administrators.
Alternatives:
- For the specific case of Oracle datafiles served via NFS, you can achieve load balancing and redundancy by using the Direct NFS feature in Oracle Database 11g and above. This will not protect other general network traffic.
- For the specific case of iSCSI access, you can achieve load balancing and redundancy by using MPXIO and MS/T (multiple sessions per target). This will not protect other general network traffic.
Useful links:
- Cisco documentation: Configuring Port Channels (Nexus 5000) -- note: the key words when communicating with a Cisco administrator are: port-channel, LACP, active, source-dest-mac
- Oracle documentation: Overview of Link Aggregation
- Oracle documentation: Deploying IPMP