Where do I find information about the AIX MPIO and SDDPCM multipath options with DS3000/DS4000/DS5000?

References for AIX MPIO and SDDPCM multipath options with DS3000/DS4000/DS5000

While the RDAC driver is being deprecated with AIX, it is still the default driver on AIX 5.2 and 5.3 for DS4000 devices. With AIX 6.1, the default driver is MPIO for DS4000 devices. All future DS4000/DS5000 models will only be supported with MPIO. For DS3000 and DS5000 subsystems RDAC is no option. These subsystems are always configured as MPIO devices. The MPIO support for these devices removes the RDAC limitation of connecting only one adapter to each subsystem controller port (1:1 FC connection) and allows more than two paths to be configured to the device. This simplifies zoning and improves performance.

Overview of various multipath driver options for DS3000/DS4000/DS5000 subsystems:
  • DS3000: native MPIO only
  • DS4000: RDAC, native MPIO, MPIO w/SDDPCM
  • DS5000: native MPIO, MPIO w/SDDPCM

Note: RDAC support for AIX has been removed starting with 07.60.xx.xx firmware for DS4000 series. With controller firmware 7.60.xx or higher the AIX RDAC fail over driver on DS4000 storage can no longer work. There has been an inquiry change that will cause problems. Customers can continue to use 7.50.xx or lower levels of firmware on DS4000 series with RDAC or with MPIO but if they move to 7.60.xx or higher only MPIO will work. See "ibm_sw_ds4kfc1_10.60.xx.17_aix_anycpu.txt" which comes with the Storage Manager package for more details: 
    New limitations with Storage Manager version 10.60.xx.17 release (controller firmware 07.60.xx.xx): 
    DS4000 storage subsystems with controller firmware 7.60.xx or greater are supported only with the MPIO multipath driver on AIX.  The fcp_array (RDAC) multipath driver support has been removed.  To switch from fcp_array to MPIO consult your AIX documentation on the manage_disk_driver command.    
For more details about prerequisites and supported OS and driver levels take a look at the particular interoperability matrices:

DS3000 family products interoperability matrix

DS4000 family products interoperability matrix

DS5000 family products interoperability matrix

Always refer to the IBM System Storage Interoperation Center for latest and up-to-date information about supported environments and configurations, including supported multipath driver option:

IBM System Storage Interoperation Center (SSIC)

A compact overview of DS4000/DS5000 multipath options for AIX with RDAC, AIX MPIO and SDDPCM can now be found in the newly released IBM Redbook:

IBM Midrange System Storage Implementation and Best Practices Guide (SG24-6363-04, Chapter 12: DS5000 with AIX, PowerVM, and PowerHA) 

General information about DS4000/DS5000 host attachment can be found in the

IBM System Storage DS Storage Manager - Installation and Host Support Guide

(1) DS4000 and AIX MPIO 

AIX MPIO is supported for DS4000 on AIX starting from AIX 5.2 TL10, AIX 5.3 TL6 and AIX 6.1 SP2 though various PTFs and fixes may be required (see interoperability matrix and AIX MPIO documentation below) on these earlier OS levels. It is supported with DS4000 firmware 6.60.xx.xx and higher on DS4800, DS4700, DS4200, DS4500 and DS4300. MPIO is the strategic driver for IBM midrange storage subsystem in AIX 6.1 and up - except for DS4400 which is RDAC only (note that also DS4100 is only listed with RDAC on the SSIC as supported configuration). RDAC is the default driver on all levels of AIX 5.2 and 5.3 for DS4000 devices while in AIX 6.1 the DS4000 products are configured as Multipath I/O (MPIO) devices by default. Please always check the IBM System Storage Interoperation Center (SSIC) for latest information about supported multipath drivers and configurations!

The first published references on how to migrate a DS4K product using the FCPARRAY (RDAC) driver to the MPIO driver, how to manage MPIO on AIX with DS4000 subsystems, the required DS4000 NVSRAM settings and the newly introduced manage_disk_driver and mpio_get_config commands in AIX can be found in the documentation below:

Procedures for AIX 6.1 and DS4000 Interoperability
http://www.ibm.com/systems/resources/systems_storage_disk_ds4000_pdf_aix.pdf (for reference only; unfortunately this link is no longer available)
http://w3.ibm.com/connections/files/app/file/e99a1584-8f81-4fea-af8b-fb9c7a20bc2d (IBM internal link to a copy of the document)

AIX61 TL02 Release Notes (p.19ff)

Please note that the information in these documents was released for early AIX 6.1 and DS4000 firmware releases starting to support AIX MPIO. The information in these documents is outdated for current AIX levels (e.g. AIX 6.1 TL4) and DS4000 firmware releases (V7.xx), for example, regarding the change of the DS4000 AIX HostNVSRAMBYTE at offset 0x27 and the manage_disk_drivers command options.

(2) DS4000 AIX HostNVSRAMBYTE at offset 0x27 for MPIO

Both documents above and even the latest AIX 6.1 TL5 Release Notes (p21ff) describe that the HostNVSRAMBYTE for host type 0x6 (which is AIX) at offset 0x27 has to be set to 0x0 on the DS4k subsystem when using AIX MPIO instead of RDAC as multipath driver by issuing the following SMcli commands to the DS4000 subsystem: 

set controller [a] HostNVSRAMBYTE [6,0x27]=0x0;
set controller [b] HostNVSRAMBYTE [6,0x27]=0x0;

This was true for early DS4000 firmware releases prior to version 06.60.08 only! 

Starting with DS4000 firmware release 06.60.08 and higher the factory default value of the AIX HostNVSRAMBYTE at offset changed to 0x8 which is fine for AIX MPIO. So before applying any change to the DS4000 NVSRAM for AIX MPIO first check if you are already at a firmware level with a default setting of 0x8 for host type 0x6 (AIX) at offset 0x27 by issuing the following SMcli script command:

show controller [a] HostNVSRAMBYTE [0x06,0x27];
show controller [b] HostNVSRAMBYTE [0x06,0x27];

If the output shows

with a default value of 0x8 then _no_ specific change of the DS4000 HostNVSRAMBYTE is required for AIX MPIO!

Please note, that the documents above describing the NVSRAM changes for AIX MPIO are not specific about the host type option required to be changed for AIX MPIO in order to work properly. A change of this HostNVSRAMBYTE effects several host type options each represented by a single bit setting. The DS4000 host type options related to offset 0x27 of the HostNVSRAMBYTE are listed in the table below: 

The real requirement for AIX MPIO to work correctly is to have the RETAIN LOGINS option disabled which is controlled by bit 0 of the HostNVSRAMBYTE at offset 0x27. Disabling the RETAIN LOGINS option means setting bit 0 = 0 for the specific host type (here AIX = 0x6) at offset 0x27. This bit effects the fibre channel PLOGI behaviour. Setting this bit to 0 is required for AIX MPIO. If not set correctly and running with reservation_policy=single_path (which is the default for DS4k hdisk attributes on AIX with MPIO) a failover to another path may not work properly.

With a default value of the HostNVSRAMBYTE at offset 0x27 being 0x8 (8 = 0 x 2^0 + 1 x 2^3 meaning bit 0 = 0 and bit 3 = 1) the RETAIN LOGINS bit is already set to 0 (bit 0 = 0) and no specific change of the NVSRAM is required anymore for using DS4000 with AIX MPIO.

When having applied the NVSRAM change already as described in the AIX 6.1 TL2 (...TL5) Release Notes by setting the whole AIX NVSRAM host byte at offset 0x27 to 0x0still the RETAIN LOGINS bit is set correctly to 0 as required by AIX MPIO. But additionally also bit 3 is set to 0 (bit 3 = 0) compared to the default value of 0x8 (8 = 1 x 2^3 meaning bit 3 = 1). Bit 3 at offset 0x27 of the AIX host byte alters the behavior regarding SCSI-3 persistent reservations, which normally are not used in the AIX MPIO driver by default. The default for AIX MPIO is to use SCSI-2 reservations with algorithm=fail_over and reservation_policy=single_path, so typically a setting of 0x0 should not be of concern in most standard configurations unless persistent reservation policies are applied. When using persistent reservation policies it is recommended to change the AIX host byte in the DS4000 NVSRAM at offset 0x27 back to its factory default value of 0x8:

set controller [a] HostNVSRAMBYTE [0x06,0x27] = 0x8;
set controller [b] HostNVSRAMBYTE [0x06,0x27] = 0x8;

Note that a DS4000 controller reboot is required for any NVSRAM changes to take effect. A controller reboot can be initiated via the GUI (Select the controller and go to Advanced > Recovery > Reset Controller) or by using the additional SMcli script commands:

reset Controller [a];

reset Controller [b]; 

Also note that all applied changes to the default DS4000 NVSRAM will be lost with a DS4000 firmware update as the NVSRAM update (which is part of the firmware update) will reset all settings stored in the NVSRAM to their defaults. So if any changes to the NVSRAM settings have been applied make sure to check and - if required - re-apply them after a firmware update.

(3) Managing multipath driver options with AIX manage_disk_drivers command

The manage_disk_drivers command introduced in previous AIX releases up to AIX 6.1 TL3 displayed a list of DS4K storage models and the driver that manages each model. All disks of a storage model must be managed by the same driver. If you specify the -c flag, the command changes the driver selection to an alternate supported driver for the storage model. The previous release of the command only offers the option to switch between AIX MPIO and RDAC for selected DS4000 models using themanage_disk_drivers -c [1..5] command option.

# manage_disk_drivers
1: DS4100: currently MPIO; supported: RDAC/fcparray, MPIO
2: DS4300: currently MPIO; supported: RDAC/fcparray, MPIO
3: DS4500: currently MPIO; supported: RDAC/fcparray, MPIO
4: DS4700/DS4200: currently MPIO; supported: RDAC/fcparray, MPIO
5: DS4800: currently MPIO; supported: RDAC/fcparray, MPIO

With release of AIX 6.1 TL4 and VIOS the manage_disk_drivers command changed substantially. It now offers a new syntax and more options to configure multipath drivers for XIV, DS3950, DS4100, DS4200, DS4300, DS4500, DS4700, DS4800, DS5100 (including DS5300) and DS5020 storage devices. With SDDPCM 2.5.1 (or higher) filesets installed it even allows to select the following AIX multipath driver options for DS4000 devices:
  • SDDPCM (displayed as AIX_SDDAPPCM for active/passive controller subsystems like DS4k/DS5k; note that SDDPCM version 2.5.1 or higher is required),
  • AIX native PCM (displayed as AIX_APPCM for active/passive controller subsystems like DS4k/DS5k) and
  • RDAC/AIX_FCP array driver (displayed as AIX_fcparray).

Below you can find an example of the manage_disk_drivers command and the available multipath driver options on AIX6.1 TL4 with SDDPCM filesets installed:

# lslpp -Lc | grep -E "sdd|fcp.disk.ibm"
devices.fcp.disk.ibm.mpio:devices.fcp.disk.ibm.mpio.rte: : :C: :IBM MPIO FCP Disk Device: : : : : : :0:0:/:
devices.sddpcm.61:devices.sddpcm.61.rte: : :C: :IBM SDD PCM for AIX V61: : : : : : :0:0:/:

# manage_disk_drivers
Usage :
        manage_disk_drivers [-l]
        manage_disk_drivers -d device -o driver_option
        manage_disk_drivers -h

# manage_disk_drivers -l
Device           Present Driver     Driver Options
2810XIV          AIX_AAPCM          AIX_AAPCM,AIX_non_MPIO
DS4100           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4200           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4300           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4500           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4700           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4800           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM

When using native AIX MPIO the mpio_get_config -Av command can be used to display subsystem specific information for the DS3k/DS4k/DS5k hdisks (as known from thefget_config -Av command for RDAC):

# mpio_get_config -Av
Frame id 0:
    Storage Subsystem worldwide name: 60ab8002643200004afa58c
    Controller count: 2
    Partition count: 1
    Partition 0:
    Storage Subsystem Name = 'DS4700_PFE'
        hdisk      LUN #   Ownership          User Label
        hdisk2         0   A (preferred)      MyLogicalDrive#1
        hdisk3         1   B (preferred)      MyLogicalDrive#2

(4) SAN configuration with AIX MPIO

Examples for SAN attachment and zoning with AIX native MPIO and more than two paths can be found in the 

IBM System Storage DS3000 - Storage Manager v2 Installation and Support Guide for IBM AIX and Linux on POWER 

in chapter 3, Preparing for a SAN attachment (p25ff):

The following examples are a small subset of the connectivity configurations that are supported using the DS3400. The best practice is to use 2 to 4 adapters providing 4 to 8 paths.

Example 1 Create one zone that contains two Fibre Channel HBA ports, one Fibre Channel port from controller A and one Fibre Channel port from controller B of the DS3400. This configuration provides four paths to hdisks. Two of the paths that are associated with the preferred storage controller will service I/Os when the DS3400 is optimal. The other two paths are used if the preferred controller is not accessible by the host. Note: The hdisk attribute “algorithm” should be set to round_robin. If the attribute is set to round_robin, the hdisk attribute “reserve_policy” must be set to no_reserve, pr_exclusive, or pr_shared. 

Example 2 Create separate zones for the connection between the HBA port and one port of the DS3400 controller. Two zones are required for a dual-controller storage configuration. One zone contains an HBA port and controller port from controller A. The other zone contains a different HBA port and controller port from controller B. This configuration provides two paths to hdisks. Note: Attributes “algorithm” and “algorithm of hdisks” do not need to be altered from default values. 

Example 3 Create one zone that contains two Fibre Channel HBA ports and all four Fibre Channel ports from the DS3400. This configuration provides eight paths to hdisks. Four of the paths that are associated with the preferred storage controller will service I/Os when the DS3400 is Optimal. The other four paths are used if the preferred controller is inoperable. Note: The hdisk attribute ″algorithm″ should be set to round_robin. When this attribute is set to round_robin the hdisk attribute ″reserve_policy″ must be set to no_reserve, pr_exclusive, or pr_shared.

The default multipath algorithm for DS3k/DS4K/DS5K devices with native MPIO on AIX is fail_over with a reserve_policy of single_path.

# lsattr -El hdisk3
PCM             PCM/friend/otherapdisk                 Path Control Module              False
PR_key_value    none                                   Persistant Reserve Key Value     True
algorithm       fail_over                              Algorithm                        True
autorecovery    no                                     Path/Ownership Autorecovery      True
clr_q           no                                     Device CLEARS its Queue on error True
cntl_delay_time 0                                      Controller Delay Time            True
cntl_hcheck_int 0                                      Controller Health Check Interval True
dist_err_pcnt   0                                      Distributed Error Percentage     True
dist_tw_width   50                                     Distributed Error Sample Time    True
hcheck_cmd      inquiry                                Health Check Command             True
hcheck_interval 60                                     Health Check Interval            True
hcheck_mode     nonactive                              Health Check Mode                True
location                                               Location Label                   True
lun_id          0x0                                    Logical Unit Number ID           False
lun_reset_spt   yes                                    LUN Reset Supported              True
max_retry_delay 60                                     Maximum Quiesce Time             True
max_transfer    0x40000                                Maximum TRANSFER Size            True
node_name       0x200600a0b81163b8                     FC Node Name                     False
pvid            none                                   Physical volume identifier       False
q_err           yes                                    Use QERR bit                     True
q_type          simple                                 Queuing TYPE                     True
queue_depth     10                                     Queue DEPTH                      True
reassign_to     120                                    REASSIGN time out value          True
reserve_policy  single_path                            Reserve Policy                   True
rw_timeout      30                                     READ/WRITE time out value        True
scsi_id         0xda                                   SCSI ID                          False
start_timeout   60                                     START unit time out value        True
unique_id       3E213600A0B8000118FD000009A1F49D1EDB20F1815 FAStT03IBMfcp Unique device id False
ww_name         0x202700a0b81143a0                     FC World Wide Name               False

In order to use multiple paths to the preferred controller with native AIX MPIO it is required to set the hdisk's algorithm attribute to round_robin and the reserve_policy attribute to no_reserve (no_reserve will be fine for most situations but is depending on your specific SCSI reservation policy) using:

# chdev -l <hdisk#> -a algorithm=round_robin -a reserve_policy=no_reserve -P 

This change requires a reboot. Alternatively one can stop using the hdisks (varyoff any VG with the disks) and eliminate the -P flag to go into effect without a reboot.

The MPIO reserve_policy defines whether a reservation methodology is employed when the device is opened. The values are as follows: 

# lsattr -Rl <hdisk#> -a reserve_policy

no_reserve Does not apply a reservation methodology for the device. The device might be accessed by other initiators, and these initiators might be on other host systems. 

single_path Applies a SCSI2 reserve methodology for the device, which means the device can be accessed only by the initiator that issued the reserve. This policy prevents other initiators in the same host or on other hosts from accessing the device. This policy uses the SCSI2 reserve to lock the device to a single initiator (path), and commands routed through any other path result in a reservation conflict. 
Path selection algorithms that alternate commands among multiple paths can result in thrashing when the single_path reserve value is selected. As an example, assume a device-specific PCM has a required attribute that is set to a value that distributes I/O across multiple paths. When single_path reserve is in effect, the disk driver must issue a bus device reset (BDR) and then issue a reserve using a new path for sending the next command to break the previous reservation. Each time a different path is selected, thrashing occurs and performance degrades because of the overhead of sending a BDR and issuing a reserve to the target device. (The AIX® PCM does not allow you to select an algorithm that could cause thrashing.)

PR_exclusive Applies a SCSI3 persistent-reserve, exclusive-host methodology when the device is opened. The PR_key attribute value must be unique for each host system. The PR_key attribute is used to prevent access to the device by initiators from other host systems.

PR_shared Applies a SCSI3 persistent-reserve, shared-host methodology when the device is opened. The PR_key value must be a unique value for each host system. Initiators from other host systems are required to register before they can access the device.

The PR_key_value is required only if the device supports any of the persistent reserve policies (PR_exclusive or PR_shared).

For more information on native AIX MPIO and the related device attributes please take a look at:

System p and AIX Information Center - Multiple Path I/O

(5) SDDPCM for DS4000/DS5000

Starting with the release of SDDPCM version DS4000 and DS5000 subsystems are also supported with the additional SDD path control module (SDDPCM). SDDPCM is compliant with the MPIO architecture and as such, is an alternative to using the native AIX MPIO PCM. SDDPCM additionally provides I/O load balancing algorithms and the pcmsrv daemon for enhanced path health checks and First Time Data Capture enhancing overall data availability. As such SDDPCM is preferable to just using MPIO. Furthermore, the SDDPCM package also includes the additional pcmpath command providing ease of use for overall SDDPCM device and path management.

Be aware that the default path selection algorithm with SDDPCM is load_balance with the device reserve policy set to no_reserve which means that no reservation methodology for the device is applied. A SCSI-2 reservation which is typically in place when doing a varyonvg of an AIX volume group is _not_ applied actually with SDDPCM default settings for the device (here the hdisks in that VG) which are reserve_policy=no_reserve and algorithm=load_balance. Such an AIX volume group might be varied on and accessed by other initiators and even other host systems in this case!

# lsdev -Cc disk
hdisk5 Available 02-08-02 IBM MPIO DS4700 Array Disk
hdisk6 Available 02-08-02 IBM MPIO DS4700 Array Disk

# lsattr -El hdisk5
PCM             PCM/friend/sddappcm                                       PCM                                     False
PR_key_value    none                                                      Reserve Key                             True
algorithm       load_balance                                              Algorithm                               True
autorecovery    no                                                        Auto Recovery                           True
clr_q           no                                                        Device CLEARS its Queue on error        True
cntl_delay_time 0                                                         Auto Recovery                           True
cntl_hchdeck_in 0                                                         Retry Timeout                           True
dist_err_pcnt   0                                                         Distributed Error Percentage            True
dist_tw_width   50                                                        Distributed Error Sample Time           True
hcheck_interval 60                                                        Health Check Interval                   True
hcheck_mode     nonactive                                                 Health Check Mode                       True
location                                                                  Location Label                          True
lun_id          0x0                                                       Logical Unit Number ID                  False
lun_reset_spt   yes                                                       Support SCSI LUN reset                  True
max_transfer    0x40000                                                   Maximum TRANSFER Size                   True
node_name       0x200400a0b83313f0                                        FC Node Name                            False
pvid            none                                                      Physical volume identifier              False
q_err           yes                                                       Use QERR bit                            True
q_type          simple                                                    Queuing TYPE                            True
qfull_dly       2                                                         delay in seconds for SCSI TASK SET FULL True
queue_depth     16                                                        Queue DEPTH                             True
reserve_policy  no_reserve                                                Reserve Policy                          True
rw_timeout      30                                                        READ/WRITE time out value               True
scbsy_dly       20                                                        delay in seconds for SCSI BUSY          True
scsi_id         0x1b0400                                                  SCSI ID                                 False
start_timeout   60                                                        START unit time out value               True
unique_id       3E213600A0B80002642EA0000C1CF4BE273F40F1814      FAStT03IBMfcp PCM                                False
ww_name         0x200400a0b83313f1                                        FC World Wide Name                      False

If the AIX volume group is shared between hosts (not using any cluster services like PowerHA with enhanced concurrent volume groups) and should _not_ be accessed by other hosts using a SCSI-2 reservation methodology the device reserve_policy can be set set to single_path ( SCSI-2 reserve) after setting the device path selection algorithm to fail_over. Any attempt to set the algorithm to round_robin, load_balance, or load_balance_port with single_path reserve policy will fail.

For a general documentation of SDDPCM support with DS4000/DS5000 take a look at the

Multipath Subsystem Device Driver User's Guide

and also the SDDPCM readme

SDDPCM for AIX Readme

which always provides latest information about prerequisites, APARs and other restrictions that currently apply. 

Note that SDDPCM is not supported on VIOS with DS4k/DS5k devices:
  • VIOS is not supported with SDDPCM on DS4000/DS5000/DS5020 subsystem devices.

However, the SDDPCM fileset might be required on a VIOS partition for attached DS8000 and/or SVC devices and you might want to attach also DS4000/DS5000 devices to the same VIOS. A supported configuration with SDDPCM on VIOS2.1 for DS8000 and/or SVC devices and native AIX MPIO for DS4k/DS5k devices might be achived by using SDDPCM on VIOS (or higher) and manually deselecting the SDDPCM driver option for the DS4k/DS5k devics with the manage_disk_drivers command. Such a configuration would be based on:
  • VIOS (or higher) required by SDDPCM and including the new manage_disk_drivers command
  • SDDPCM (or higher) fileset with appropriate SDDPCM prerequisites (e.g. devices.fcp.disk.ibm.mpio.rte, V1.0.0.20)
  • selecting AIX_APPCM as present driver for attached DS5k/DS4k subsystems with manage_disk_drivers command (i.e. deselect AIX_SDDAPPCM)
  • AIX MPIO using SDDPCM for DS8000 / SVC devices (i.e. only the DS8000/SVC devices are managed by SDDPCM)
  • AIX MPIO using native AIX PCM for all DS4k/DS5k devices (i.e. all DS4k/DS5k devices are managed by the default AIX MPIO driver with the native AIX PCM)

The pcmpath command of the SDDPCM package can be used to easily display and manage SDDPCM devices and paths. Here is an example of the pcmpath query devicecommand with attached DS8000, SVC and DS4800 devices on an AIX 6.1 partition using SDDPCM 2.4.x:

# pcmpath query device

Total Dual Active and Active/Asymmetrc Devices : 2

DEV#:   0  DEVICE NAME: hdisk1  TYPE: 2107900  ALGORITHM:  Load Balance
SERIAL: 75KA3628201
Path#      Adapter/Path Name          State     Mode     Select     Errors
    0           fscsi0/path0           OPEN   NORMAL     844158          0
    1           fscsi0/path1           OPEN   NORMAL     843779          0
    2           fscsi1/path2           OPEN   NORMAL     833541          0
    3           fscsi1/path3           OPEN   NORMAL     833459          0

DEV#:   1  DEVICE NAME: hdisk2  TYPE: 2145  ALGORITHM:  Load Balance
SERIAL: 60050768018082E29000000000000008
Path#      Adapter/Path Name          State     Mode     Select     Errors
    0*          fscsi0/path0           OPEN   NORMAL          6          0
    1           fscsi0/path1           OPEN   NORMAL    2124784          0
    2           fscsi1/path2           OPEN   NORMAL    2148331          0
    3*          fscsi1/path3           OPEN   NORMAL          8          0

Total Active/Passive Devices : 1

DEV#:   3  DEVICE NAME: hdisk3  TYPE: 1815      ALGORITHM:  Load Balance
SERIAL: 600A0B8000118FD00000878E49C33524
Path#      Adapter/Path Name          State     Mode     Select     Errors
    0*          fscsi0/path0           OPEN   NORMAL          0          0
    1*          fscsi1/path2           OPEN   NORMAL          0          0
    2           fscsi0/path1           OPEN   NORMAL      94440          0
    3           fscsi1/path3           OPEN   NORMAL      94168          0

The DS4000 and DS5000 products are dual array controller disk subsystems. Each LUN is assigned to one controller, which is considered the owner, or the active (primary) controller, of a particular LUN. The other controller is considered as an alternate, or passive (secondary), controller. The '*' next to the path ID in the pcmpath command output indicates that the related path is a non-preferred path. The indicator '*' for preferred and non-preferred paths are only shown after the device is opened once. 

SDDPCM distinguishes the following paths to a given LUN on a DS4000 and DS5000 subsystem:
  • Paths to the primary (active) controller that currently owns the LUN and processes the I/Os to that LUN
  • Paths to the alternate (passive/secondary) controller

With this type of active/passive dual-controller subsystem device, I/O can be sent only to the controller that currently owns the LUN. When the SDDPCM selects paths for I/O, it selects paths that are connected only to the active controller for a given LUN. If there is no path on the active controller that can be used, SDDPCM changes the LUN controller ownership to an alternate controller, switches the paths that were passive to active, and then selects these active paths for I/O.

As shown in the example above for a DS4800 (model type 1825) the '*' indicates the paths to the secondary or alternate controller for the LUN. That's why there were no I/Os seen on these paths. Here the I/O load for the given LUN is balanced only across the paths to the primary DS4K/DS5K controller that currently owns the LUN. With two host FC adapters (HBAs) in the server and each host adapter seeing both DS4K/DS5k controllers the load-balancing algorithm with SDDPCM will balance I/Os across both host FC adapters for all I/Os to all DS4K/DS5K LUNs.

The SDDPCM fileset also provides the sddpcm_get_config command for DS4000/DS5000 devices which displays information about all MPIO-based DS4K/DS5K subsystems using SDDPCM and the hdisks associated with them. Specifically, it displays information about the frame (subsystem), including the frame's assigned name and worldwide name, and a list of hdisks (only those currently in the Available state) associated with that subsystem, including the hdisk name, LUN number, current ownership, preferred path, and the user-assigned label for that volume.

# sddpcm_get_config -Av
Frame id 0:
    Storage Subsystem worldwide name: 60ab8001143a0000049b88968
    Controller count: 2
    Partition count: 1
    Partition 0:
    Storage Subsystem Name = 'VIOS_DS4800'
        hdisk      LUN #   Ownership          User Label
        hdisk3         0   A (non-preferred)  lpar3_vol01

The SDDPCM support matrix and the SDDPCM filesets (devices.sddpcm.61.rte) for DS4000/DS5000 are available here:

Support Matrix for Subsystem Device Driver, Subsystem Device Driver Path Control Module

Subsystem Device Driver Path Control Module (SDDPCM)

SDDPCM also requires the AIX host attachment script fileset (devices.fcp.disk.ibm.mpio.rte) to be installed as prerequisite:

Host Attachment for SDDPCM on AIX

The AIX host attachment script provides the device definitions and ODM attributes for the supported SDDPCM devices. Without the AIX host attachment fileset installed the devices will be managed by the default AIX MPIO PCM and simply show up as:

# lsdev -Cc disk
hdisk0 Available Virtual SCSI Disk Drive
hdisk1 Available 30-T1-01 MPIO Other FC SCSI Disk Drive
hdisk2 Available 30-T1-01 MPIO Other FC SCSI Disk Drive
hdisk3 Available 30-T1-01 MPIO Other DS4K Array Disk

With the AIX host attachment script installed the supported SDDPCM devices will show up as:

# lsdev -Cc disk
hdisk0 Available Virtual SCSI Disk Drive
hdisk1 Available 30-T1-01 IBM MPIO FC 2107
hdisk2 Available 30-T1-01 MPIO FC 2145
hdisk3 Available 30-T1-01 IBM MPIO DS4800 Array Disk

If the DS4K/DS5K hdisks still show up as "MPIO Other DS4K Array Disk" after the installation of the AIX host attachment script fileset they will not be managed by SDDPCM and will not be displayed by the pcmpath query command. Here it would be necessary to manually remove and reconfigure the devices so that they show up as "IBM MPIO DS4800 Array Disk" (if the appropriate version of the AIX host attachment script for SDDPCM 2.4.x.x is installed).

The Multipath Subsystem Device Driver User's Guide recommends a SAN attachment with four paths for DS4000 and DS5000 subsystems with SDDPCM:

Because switching a device to another controller is a time-consuming recovery action and affects I/O performance, you can use this redundancy to avoid an unnecessary controller failover if a path fails. Therefore, configure a minimum of four paths for each LUN with two host adapter ports and two storage controller ports where each host adapter port has redundancy to each storage controller port and vice versa. (p.8)