HP P9000 Operation & User’s Manual

Mainframe Systems User Guide Abstract This guide explains how to use HP StorageWorks P9000 Continuous Access Journal for Mainframe Software to replicate data between local and remote HP StorageWorks P9000 disk arrays and to achieve disaster tolerance with maximum application performance. Topics include setting up remote copy connections, configuring the storage system, creating and monitoring remote copies, recovering from a disaster, and troubleshooting. The intended audience is a storage system administrator or authorized service provider with independent knowledge of HP StorageWorks P9000 disk arrays and the HP StorageWorks Remote Web Console. HP Part Number: AV400-96371 Published: May 2011 Edition: Fourth

Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Acknowledgements Microsoft®, Windows®, Windows® XP, and Windows NT® are U.S. registered trademarks of Microsoft Corporation. Export Requirements You may not export or re-export this document or any copy or adaptation in violation of export laws or regulations. Without limiting the foregoing, this document may not be exported, re-exported, transferred or downloaded to or within (or to a national resident of) countries under U.S. economic embargo, including Cuba, Iran, North Korea, Sudan, and Syria. This list is subject to change. This document may not be exported, re-exported, transferred, or downloaded to persons or entities listed on the U.S. Department of Commerce Denied Persons List, Entity List of proliferation concern or on any U.S. Treasury Department Designated Nationals exclusion list, or to parties directly or indirectly involved in the development or production of nuclear, chemical, biological weapons, or in missile technology programs as specified in the U.S. Export Administration Regulations (15 CFR 744). Revision History Description Date Edition Applies to microcode version 70-01-01-00/00 or later. October 2010 First Applies to microcode version 70-01-24-00/00 or later. November 2010 Second Applies to microcode version 70-01-62-00/00 or later. January 2011 Third Applies to microcode version 70-02-01-00/00 or later. May 2011 Fourth

Continuous Access Journal Z software ........................................................................................8 How Continuous Access Journal Z works.....................................................................................8 Hardware and software components...........................................................................................9 P9500 storage systems.......................................................................................................11 Main and remote control units .......................................................................................11 Pair volumes......................................................................................................................11 Journal volumes.................................................................................................................11 Journals............................................................................................................................12 Data path.........................................................................................................................12 Extended consistency groups...............................................................................................12 Remote Web Console ........................................................................................................13 Business Continuity Manager..............................................................................................13 Overview of copy operations...................................................................................................13 Initial copy operation.........................................................................................................13 Update copy operation ......................................................................................................14 Read and Write I/O operations during remote copy operation.....................................................15 S-VOL write option..................................................................................................................15 S-VOL read option..................................................................................................................15 Difference management...........................................................................................................16 Pair status..............................................................................................................................16 2 Requirements and specifications.................................................................17 System requirements................................................................................................................17 Business Continuity Manager specifications ...............................................................................19 Command device ..................................................................................................................21 3 Planning volumes, systems.........................................................................23 Plan and design workflow .......................................................................................................23 Assessing business requirements for data recovery......................................................................23 Determining your RPO........................................................................................................24 Write-workload......................................................................................................................24 Measuring write-workload...................................................................................................24 Sizing journal volumes............................................................................................................25 Determine the required journal volume capacity.....................................................................25 Calculating the journal size.................................................................................................26 Planning journals...............................................................................................................26 Data transfer speed considerations...........................................................................................27 RAID group configuration....................................................................................................27 Fibre Channel port configuration..........................................................................................27 Planning journal volumes.........................................................................................................27 Host I/O time stamp..........................................................................................................28 Planning pair volumes.............................................................................................................28 Data and journal volume specifications.................................................................................28 Duplicating VOLSERs..........................................................................................................29 Maximum number of pairs allowed .....................................................................................30 Calculating maximum number of pairs.............................................................................30 Priority set for initial copy operations and scheduling order.....................................................31 Disaster recovery considerations ..............................................................................................33 Contents 3

Sharing volumes with Continuous Access Synchronous Z, other program products...........................33 Planning for Continuous Access Journal Z with multiple P9500 systems..........................................33 Establishing connections between multiple secondary systems..................................................35 Planning extended consistency groups (EXCTG).....................................................................35 Planning for previous models ...................................................................................................36 Guidelines for preparing systems for Continuous Access Journal Z.................................................37 System option modes..........................................................................................................37 4 Planning the data path..............................................................................40 Data path design workflow......................................................................................................40 Sizing bandwidth...................................................................................................................40 Five sizing strategies...........................................................................................................40 Calculating bandwidth.......................................................................................................41 Sizing bandwidth for peak write-workload........................................................................41 Sizing bandwidth for peak rolling average write-workload..................................................42 Latency .......................................................................................................................43 Packet loss ...................................................................................................................43 Planning ports for data transfer.................................................................................................44 Determining required number of ports ..................................................................................44 On setting up ports ...........................................................................................................45 Distances supported for Fibre Channel type, number of switches ..................................................45 Supported data path configurations .........................................................................................46 5 Using extended consistency groups.............................................................48 Overview .............................................................................................................................48 Register journals in an EXCTG .................................................................................................50 Split, restore, release pairs in an EXCTG ...................................................................................51 Pair-split operation.............................................................................................................51 Operations for a specific journal .........................................................................................51 Remove journals from an EXCTG ..............................................................................................52 Forcibly remove journals.....................................................................................................53 Using EXCTGs with P9500.......................................................................................................53 6 Configuration operations...........................................................................56 Configuration workflow ..........................................................................................................56 Define Fibre Channel port attributes .........................................................................................56 Configure storage systems for Continuous Access Journal Z, define logical paths ............................58 Configure additional logical paths ...........................................................................................60 Specify number of volumes for initial copy, resync ......................................................................61 Register journal volumes in a journal.........................................................................................63 7 Pair operations.........................................................................................67 Pair operations workflow.........................................................................................................67 Check pair status....................................................................................................................67 Create the initial copy ............................................................................................................67 Split a pair ...........................................................................................................................71 Split a mirror..........................................................................................................................73 Create a point-in-time copy .....................................................................................................73 Restore a pair........................................................................................................................74 Resynchronize a mirror............................................................................................................76 Delete a pair ........................................................................................................................77 4 Contents

8 Monitoring the system...............................................................................80 Monitor pair activity, status......................................................................................................80 Pair status definitions..........................................................................................................80 Suspend types..............................................................................................................83 Suspension conditions from the secondary system..............................................................85 Filtering Information in the List in the Pair Operation window...............................................86 Saving pair status information into a text file.....................................................................87 Monitor copy operations data, I/O .....................................................................................88 Select data to be graphed..............................................................................................89 Manipulate graph, save data..........................................................................................91 Monitor journal status..............................................................................................................91 Mirror Journal status definitions............................................................................................91 Monitor logical path status.......................................................................................................92 History of operations for data volume pairs...........................................................................93 9 Maintaining the system ............................................................................94 Pair maintenance—change the pair-split option .........................................................................94 Journal and mirror maintenance...............................................................................................95 Change Continuous Access Journal Z options used by journals................................................95 Change Continuous Access Journal Z options used by mirrors .................................................97 Delete journal volumes from a journal...................................................................................99 Delete a journal...............................................................................................................100 Logical path maintenance......................................................................................................100 Modify data-transfer time threshold ...................................................................................100 Delete logical paths ........................................................................................................101 Delete the Continuous Access Journal Z relationship ............................................................102 Manage power-off for systems and network devices .................................................................102 When power stops unexpectedly.......................................................................................103 When power is removed from primary or secondary system..............................................103 When power is removed from network relay devices .......................................................103 Power-off storage systems intentionally................................................................................103 Power-off the primary or secondary system......................................................................103 Power-off primary and secondary systems at the same time...............................................104 Power-off network relay devices..........................................................................................104 10 Disaster recovery operations ..................................................................105 Preparing for disaster recovery ..............................................................................................105 File and database recovery procedures ..................................................................................105 Switch operations to the secondary site ..................................................................................105 Copy data back to the primary site ........................................................................................106 Resume normal operations at the primary site ..........................................................................106 Disaster recovery for multiple primary and secondary storage systems.........................................107 Consistency of data update sequence when a disaster occurs................................................107 Disaster recovery for multiple primary and secondary storage systems....................................107 Recovery procedures with shared volumes................................................................................107 Recovery in a 3DC cascade configuration ..........................................................................108 Transferring business tasks back to the primary site..........................................................108 Recovering from primary site disaster in 3DC multi target configuration...................................109 Recovering from primary site failures (when delta resync operation is performed)......................111 Recovering from failures in the primary site and the Continuous Access Synchronous Z secondary site................................................................................................................................114 Contents 5

11 Troubleshooting....................................................................................116 General troubleshooting........................................................................................................116 Troubleshooting logical paths ................................................................................................117 Troubleshooting suspended pairs ...........................................................................................119 Troubleshooting using Remote Web Console............................................................................121 Error codes .........................................................................................................................122 Service information messages (SIMs) ......................................................................................122 Enabling or disabling SIM reporting...................................................................................124 Clearing SIMs.................................................................................................................125 12 Support and other resources...................................................................126 Contacting HP......................................................................................................................126 Subscription service..........................................................................................................126 Documentation feedback..................................................................................................126 Related information...............................................................................................................126 HP websites....................................................................................................................127 Conventions for storage capacity values..................................................................................127 Typographic conventions.......................................................................................................127 A Sharing Continuous Access Journal Z volumes ...........................................129 Volume types that can be shared with Continuous Access Journal Z.............................................129 Virtual LVI ...........................................................................................................................131 Cache Residency..................................................................................................................131 Auto LUN............................................................................................................................131 Performance Monitor ............................................................................................................131 Data Retention.....................................................................................................................131 B Continuous Access Journal Z configurations with Continuous Access Synchronous Z.............................................................................................................133 Sharing volumes with Continuous Access Synchronous Z............................................................133 3 data center cascade configuration.......................................................................................133 Prerequisite information for 3DC cascade ...........................................................................134 Procedure for setting up 3DC cascade ...............................................................................135 3 data center multitarget configuration ...................................................................................135 Prerequisite information for 3DC multitarget.........................................................................136 Procedure for setting up 3DC multitarget.............................................................................136 Delta resync configuration .....................................................................................................137 Prerequisite information for creating the delta resync pair......................................................138 Prerequisite information for performing delta resync operation................................................138 Procedure for creating a delta resync pair ..........................................................................139 Configuring delta resync operation environment to support remote command devices................140 Establish immediate communications ............................................................................140 Assign mirrors to remote command devices ....................................................................141 Release mirrors...........................................................................................................142 Perform the delta resync operation .....................................................................................143 C Continuous Access Journal Z configurations with Business Copy Z................144 Overview............................................................................................................................144 Configurations with Business Copy Z primary volumes ..............................................................144 6 Contents

Pair status and data currency.................................................................................................146 Using At-Time Split function when combining Continuous Access Journal Z with Business Copy Z....147 D Continuous Access Journal Z GUI reference...............................................151 Journal Operation window ....................................................................................................151 Journal Detail window ..........................................................................................................154 Change Journal Option dialog box ........................................................................................158 Change Mirror Option dialog box .........................................................................................158 Edit Journal Volumes dialog box.............................................................................................160 Pair Operation window ........................................................................................................161 Detailed Information dialog box ........................................................................................165 Add Pair dialog box ........................................................................................................168 Suspend Pair dialog box ..................................................................................................171 Resume dialog box .........................................................................................................172 Delete Pair dialog box .....................................................................................................173 Change Pair Option dialog box.........................................................................................174 Display Filter dialog box ..................................................................................................175 DKC Operation window .......................................................................................................176 Remote Systems Information ..............................................................................................177 Logical Path Information....................................................................................................178 Port Information for the local system....................................................................................179 DKC Status dialog box ....................................................................................................180 Add DKC dialog box .......................................................................................................182 DKC Option dialog box ...................................................................................................183 Usage Monitor window ........................................................................................................184 History window ...................................................................................................................184 Operations in History window ..........................................................................................186 History window notes.......................................................................................................188 Export operations history .................................................................................................188 Optional Operation window .................................................................................................188 EXCTG Operation window ....................................................................................................189 View the EXCTG List ........................................................................................................191 View Storage Systems in an EXCTG ...................................................................................192 View journals in an EXCTG ..............................................................................................192 Add Journal....................................................................................................................193 Glossary..................................................................................................195 Index.......................................................................................................197 Contents 7

Unless otherwise specified, the term P9000 in this guide refers to the following disk array: • P9500 Disk Array The GUI illustrations in this guide were created using a Windows computer with the Internet Explorer browser. Actual windows may differ depending on the operating system and browser used. GUI contents also vary with licensed program products, storage system models, and firmware versions. With Continuous Access Journal Z (Cnt Ac-J Z), you create and maintain a remote copy of a data volume on a P9500 system. The remote copy is a block-for-block copy of the local storage volume. Remote data is consistent with local data and therefore available for recovery of the local volume should the need arise. This guide provides instructions for planning, implementing, operating, maintaining, and troubleshooting a Continuous Access Journal Z system. The following configurations described in this document are unsupported in version 70-01-0x. • Three data center (3DC) multitarget configuration • Three data center (3DC) cascading configuration • Three data center (3DC) configuration using the delta resync function • Configuration using multiple primary and secondary storage systems Continuous Access Journal Z software With Continuous Access Journal Z, application data is copied to a secondary P9500 system at a remote location. Continuous Access Journal Z is designed to support a remote site hundreds and even thousands of miles from the local site, making recovery from region-wide disasters possible. This guide provides scenarios and procedures for disaster recovery from multiple sites. When a pair is created, the remote system will contain an asynchronous, block-for-block copy of the local storage volume. Impact on host I/O and the primary storage system is limited, since updates sent to the primary volume are also copied to a local journal volume. The remote system “pulls” data from the journal volume across the communication link to the backup-volume. The primary system is free to perform its role as a transaction processing resource rather than as replication engine. How Continuous Access Journal Z works With Continuous Access Journal Z, you enable a data back up from a primary volume (P-VOL) on the local system to a secondary volume (S-VOL) on a remote system. Continuous Access Journal Z operations are performed sequentially as shown below. Figure 1 Basic sequence in Continuous Access Journal Z operations 8 Continuous Access Journal Z overview

• The journal on the local system is the “master journal.” • The journal on the remote system is the “restore journal.” Replication occurs in the following sequence: • Journal obtain - When the host sends an update to the primary volume, the system’s journal-obtain function triggers a copy of the update data to the master journal volume. ◦ The host assigns write-sequence numbers to the data sent to the journal. ◦ Write-sequence numbers and other metadata attached to journal data insure consistency with the data in the primary volume. • Journal copy - data is copied from the master journal to the restore journal. If the master journal has data, the primary system transfers it to the restore journal. When data transfer is complete, master journal data is discarded. ◦ ◦ Data copy to the restore journal is initiated by the read-journal command issued by the remote system. ◦ Data copy occurs on a continual basis unless there is no data in the local journal. On the remote storage system, journal reading operations are performed just after finished the former read operation. • Journal-restore - the secondary volume is updated with changed data from the restore journal. Data is copied to the secondary volume according to the write sequence numbers, insuring data consistency. ◦ ◦ When journal-restore is completed, the data in the restore journal is discarded. ◦ In the configuration of Continuous Access Journal Z, the updating of data volumes and the creating of journal data are processed. Therefore, the performance decreases compared with the configuration only of data volumes, and usage rates rise in the configuration of Continuous Access Journal Z. ◦ The primary storage system does not remove the target journal data from its master journal volume until it receives the sequence numbers of the restored journal that is given to the read journal command from the secondary storage system. This is true even if the primary storage system and secondary storage system are connected using a channel extender product. Hardware and software components A typical configuration consists of a P9500 or externally attached system on both local and remote sites, a host or hosts connected to the systems, Continuous Access Journal Z software on both systems, data path connections, and interface tools for configuring and managing Continuous Access Journal Z. • The local and remote P9500 systems are connected using dedicated fibre channel data paths, which can include fibre channel switches. Data paths are routed from the fibre channel ports on primary system to the ports on the secondary system. • The host is connected to the P9500 using a FICON channel port. • Remote Web Console is connected via a management LAN. Hardware and software components 9

• P-VOLs and S-VOLs on the local and remote P9500 • Master and restore journal volumes on the local and remote P9500 • Master and restore journals on the local and remote P9500 The master journal consists of the primary volume(s) and master journal volume(s). ◦ ◦ The restore journal consists of the secondary volume(s) and restore journal volume(s). When Continuous Access Journal Z journals consist of journal volumes located in multiple systems, a host I/O time stamping function is required. This is provided by MVS DFSMSdfp. An error reporting communications (ERC) feature is also required when Continuous Access Journal Z is to be used for disaster recovery. Management software consists of: • Remote Web Console graphical user interface (GUI) • Business Continuity Manager • Configuration File Loader NOTE: • Continuous Access Journal Z processing continues uninterrupted if the SVP reboots or even if the SVP fails. • Continuous Access Journal Z does not support operations in which one P-VOL is copied to more than one S-VOL, or more than one P-VOL is copied to one S-VOL. Continuous Access Journal Z components are illustrated in the following figure and described in greater detail in the following sections. Figure 2 Continuous Access Journal Z components 10 Continuous Access Journal Z overview

Continuous Access Journal Z is operated using two P9500 storage systems, one at the primary and one at secondary sites. The primary system consists of the main control unit (MCU) and the SVP . The secondary system consists of the remote control unit (RCU) and the SVP. • The primary system communicates with the secondary system over dedicated fibre channel remote copy connections. • The P9500 system can function simultaneously as a primary and secondary system. Main and remote control units The primary and secondary systems are often referred to as the MCU (primary system) and RCU (secondary system). MCU is the main control unit, RCU is the remote control unit. MCUs control the primary storage volume (P-VOL) and the following operations: • Host I/O operations to the P-VOL • Master journal operations • Initial copy and update copy operations between the P-VOL and secondary volume (S-VOL). RCUs control the secondary storage volume (S-VOL) and the following operations: • Issue read-journal commands to the MCU. • Manage the copying of journal data from master to restore journal • Manage the copying of restore journal data to S-VOL • Assist in managing pair status and configuration (for example, rejects write I/Os to the S-VOLs). Pair volumes Original data is stored in the P-VOL and the remote copy is stored in the S-VOL. The pair can be paired, split, resynchronized, and returned to the simplex state. When synchronized, the volumes are paired; when split, new data sent is to the P-VOL but held from the S-VOL. When resynchronized, changed data is copied to the S-VOL. When a disaster occurs, production operations can be transferred to the S-VOL. When the primary site is functional again, operations and data can be transferred and copied back to the P-VOL. The P-VOL remains available to the host for read and write I/O operations. The secondary system rejects write I/Os for the S-VOL, unless the write-enable option is specified for the S-VOL. Then, write I/O is allowed to the S-VOL while the pair is split. In this instance, S-VOL and P-VOL track maps keep track of differential data and uses it to resynchronize the pair. See “Duplicating VOLSERs” (page 29) for further information on volume requirements and support. NOTE: When you configure a journal pair, you have to specify the serial numbers of primary storage systems and secondary storage systems. You have to specify the different serial numbers of the primary storage system and the secondary storage system for the same journal pair. If you have to specify the same serial number, contact HP Technical Support. Journal volumes Journal volumes are required on the primary and secondary systems. • Updates to the P-VOL are copied to the master journal volume in the primary system. See the illustration in “Journals” (page 12). • Master journal data is copied to the restore journal volume on the secondary system. • Journal volumes can have different volume sizes and different RAID configurations. • Journal data is stored sequentially and separately into each journal volume in the same journal. Hardware and software components 11

NOTE: If a path is defined from a host to a volume, you cannot register the volume as a journal volume. Journals Journals help you manage data consistency between multiple P-VOLs and S-VOLs. A journal consists of two or more data volumes and journal volumes. You use journals to create multiple pairs and to split, resync, and release multiple pairs. Journals are required on the primary and secondary systems. Each data volume and its associated journal volume reside in the same journal. This is illustrated below. • The master journal group contains master journal volumes and is associated with the P-VOL. • The restore journal group contains restore journal volumes and is associated with the S-VOL Each pair relationship between journals is called a "Mirror". A Mirror ID identifies a pair relationship between journal groups. When the pair is created, it is assigned a mirror ID. Figure 3 Journal Groups Data path The physical transmission link between the local and remote systems is called the data path. Continuous Access Journal Z commands and data are transmitted through the fibre channel data path and switches. The data path is connected to the primary and secondary systems through two types of Fibre Channel ports, Initiator and RCU Target ports. One data path connection is required for Continuous Access Journal Z. HP recommends two or more independent connections to provide hardware redundancy. A maximum of eight paths can be used. For more information, see “Planning the data path” (page 40). Extended consistency groups To maintain data consistency when performing copy operations between multiple primary systems and multiple secondary systems, the associated journals must be registered in an extended consistency group (EXCTG). An EXCTG is a collection of journals in which data consistency is guaranteed. See “Planning extended consistency groups (EXCTG)” (page 35) for more information. 12 Continuous Access Journal Z overview

Remote Web Console communicates with the SVP of each storage system over defined TCP/IP connections. • Remote Web Console must be LAN-attached to the primary system • Remote Web Console is not required on the secondary system; however having it available allows you to change Continuous Access Journal Z parameters and access the Continuous Access Journal Z S-VOL for maintenance or disaster recovery. NOTE: • Storage Administrator (Remote Copy) role access to the Remote Web Console Java applet program is required to perform operations. Users without Storage (Remote Copy) role write access can only view (Remote Copy) role information. • If the RAID Manager computer is not installed, contact HP Technical Support for information on Continuous Access Journal Z configuration services. Business Continuity Manager Business Continuity Manager provides management of Continuous Access Journal Z software in mainframe environments. Business Continuity Manager software automates Continuous Access Journal Z operations (as well as operations for Business Copy Z and Continuous Access Synchronous Z). Access to system performance metrics and thresholds provides problem avoidance and optimum performance. Overview of copy operations The following sections describe initial and update copy operations including the underlying operations, such as journal processing and differential data management. Initial copy operation The initial copy is executed when the primary storage system copies all the data in sequence in the P-VOL directly to the S-VOL. Though journal volumes are not used during the initial copy, the copy data in this operation is referred to as “base journal data.” • Creating pairs independently of journal activity results in the base journal data being copied to the respective S-VOLs one at a time. This extends the time required for multiple initial copies. • An initial copy operation can be performed to establish the Continuous Access Journal Z pair relationship — with no data copied between the volumes. This can be done when data in the P-VOLs and S-VOLs are the same. • A volume pair can also be created using a Continuous Access Synchronous initial copy operation. See “Planning pair volumes” (page 28) for more information. Overview of copy operations 13

pairs within the same journal, the base journal of one of the pairs is stored in the restore journal volume. Then, the base journal of another pair is stored in the restore journal volume. Therefore, the operation for restoring the latter base journal is delayed. • You can specify None as the copy mode for initial copy operations. If the None is specified, initial copy operations are not performed. Therefore, if you specify None, you are responsible for ensuring that the data in the primary and secondary data volume is completely the same. • You may create a Continuous Access Journal Z data volume pair by using Continuous Access Synchronous Z initial copy. In that case, set the appropriate system option that is system option 474, on both primary and secondary storage system. In addition, the script, which is written especially for this operation, is also required. If you use the script without setting the system option on the secondary storage system, the storage system recognizes the Continuous Access Synchronous Z R-VOL as the Continuous Access Journal Z S-VOL, and the SSB log (SSB=CBED) is generated in the secondary storage system. In this case, the command to create the pair is rejected. The script is executed normally when you use the script without setting the system option on the primary storage system, however, note that the processing speed of the Continuous Access Synchronous Z initial copy slows down if there is the update I/O during the operation. • If you delete all Continuous Access Journal Z data volume pairs in the journal and then create a Continuous Access Journal Z data volume pair, wait for one or more minutes after deleting pairs. • When an RCU is shared with multiple MCUs, perform the pair resume operation after system option mode 593 is set to available if you resume 1025 or more pairs from multiple MCUs (range: mirror) at the same time. Pair resume operations may fail if system option mode 593 is not available. Update copy operation When a host produces new or changed information, the following occurs in the primary storage system: • The update is written to the P-VOL • The update is copied to the master journal along with metadata that includes sequence and other consistency information. Replication to the secondary system is prompted by the read-journal command. This is issued by the remote system independently of host I/O activity. Read-journal commands are repeated every 30 seconds. At this time, the following occurs: ◦ Any data that exists in the master journal on the primary side is sent to the restore journal on the remote system. ◦ The updated data is then copied to the S-VOL. ◦ Data is updated to the remote system continuously until there is no more data in the master journal. ◦ Journal data on the primary and secondary systems is discarded when data consistency is established in the copy. 14 Continuous Access Journal Z overview

system (RCU), called RIO (remote I/O). RIO provides the most efficient type of data transfer. Make sure that your channel extenders are capable of supporting RIO. Contact HP Technical Support for more information. If an update copy operation fails, the remote system suspends the affected pair or all Continuous Access Synchronous Z pairs in the journal. This is dependent on the type of failure. The suspended pair or journal returns to Paired status when the primary and secondary storage systems are resynchronized. Read and Write I/O operations during remote copy operation The primary system reads from the P-VOL when it receives a read I/O. If the read fails, the redundancy provided by RAID1 or RAID5 technology recovers the failure. The primary system does not read the S-VOL for recovery. When a primary system receives a write I/O for a P-VOL in Duplex status, the system performs the update copy operation, as well as writing to the P-VOL. The write operation completes independently of the update copy operations on the S-VOL. The secondary system updates the S-VOL according to the write sequence number in the journal data. This maintains data consistency between P-VOL and S-VOL. If the P-VOL write operation fails, the primary system reports a unit check and does not create the journal data for this operation. If the update copy operation fails, the secondary system suspends either the affected pair or all Continuous Access Journal Z pairs in the journal, depending on the type of failure. When the suspended pair or journal is resumed, the primary and secondary systems negotiate the resynchronization of the pair(s). During normal Continuous Access Journal Z operations, the secondary system does not allow S-VOLs to be online (mounted). Therefore, hosts cannot read from or write to S-VOLs. The S-VOL write enable option allows write access to a secondary data volume while the pair is split. The option is only available when you split the pair from the primary system. When you issue the DEVSERV command to the Continuous Access Journal Z S-VOL, INDETERMINATE FAILING UNIT is returned if the status of Continuous Access Journal Z S-VOL is online. INTERVENTION REQUIRED is returned, if the status of Continuous Access Journal Z S-VOL is offline. S-VOL write option When a pair is split, you can set an option that will allow write I/O to the S-VOL. The S-VOL write option is selected during the Suspend Pair operation when performing the operation from the primary system. When you resynchronize a split pair whose S-VOL is write-enabled, the secondary system sends the S-VOL track bitmap to the primary system, which merges the P-VOL and S-VOL bitmaps to determine which tracks are out of sync. This ensures proper resynchronization of the pair. S-VOL read option You can also have a read option enabled for the S-VOL (mode 20) while the pair is suspended. The secondary system allows you to change the VOLSER of suspended S-VOLs. By setting the mode 20 and the mode 190 to ON, you can change the VOLSER and VTOC of suspended S-VOLs. This allows the S-VOL to be online to the same host as the P-VOL during the suspend. The primary system copies the P-VOL VOLSER and VTOC back to the S-VOL when the pair is resynchronized. Make sure any secondary volumes that were varied online to the host are varied offline before performing the resync operating. Write I/Os are rejected by the secondary storage system during the suspend. Read and Write I/O operations during remote copy operation 15

the S-VOL read option is not enabled and/or the pair is not suspended, the secondary system rejects read and write I/O to a S-VOL. Difference management Differential data (updates during split or suspension) between the P-VOL and S-VOL is stored in a track bitmap. When a split/suspended pair is resumed, the primary system merges the P-VOL and S-VOL bitmaps, and the differential data is copied to the S-VOL. The number of bitmap areas affects the maximum possible number of pairs that can be created in the storage system. Pair status Every pair operation results in a change in pair status. You should monitor pair status to insure that an operation completed successfully. Also, pairs must have a specific status in order for specific operations to be executed. The following provides a brief description of the pair statuses. For complete details, see “Pair status definitions” (page 80) . • Simplex: A volume that is not assigned to a pair. • Duplex: When a pair is in the process of being created, the primary system changes the status of the P-VOL and S-VOL to Pending Duplex. When the initial copy operation is complete, the primary system changes the status of both data volumes to Duplex. • Pending: The initial copy operation for this pair is in progress. This data volume pair is not yet synchronized. When the initial copy operation is complete, the primary system changes the status of both data volumes to Duplex. • Duplex: This data volume pair is synchronized. Updates to the P-VOL are duplicated on the S-VOL. When the initial copy operation is complete, the primary system changes the status of both data volumes to Duplex. • Suspended When a pair is split from the primary system, the primary system changes the status of the P-VOL and S-VOL (if the path status is normal) to Suspended. When a pair is suspended from the secondary system, it changes the S-VOL status to Suspended. The primary system detects the pair suspension (if the path status is normal) and changes P-VOL status to suspended. • When you release (delete) a pair from the primary system, that system changes the status of the P-VOL and S-VOL (if the path status is normal) to simplex. When you release a pair from the secondary system, that system changes S-VOL status to Simplex. The primary system detects the pair release (if the path status is normal) and changes P-VOL status to Suspended. 16 Continuous Access Journal Z overview

This chapter provides system requirements for Continuous Access Journal Z. System requirements Continuous Access Journal Z operations are performed between the host(s) and the primary and secondary storage systems containing the P-VOLs and S-VOLs, using the data path. General requirements for the Continuous Access Journal Z components are listed below. Table 1 General system requirements Requirement Item Two—one at the local site, one at the remote site. Also, any combination of the following can be used with Continuous Access Journal Z: • Four on primary side • Four on secondary side Number of P9500 systems • Must be installed on primary and secondary P9500 systems. • License keys required. • On secondary systems, Continuous Access Journal can coexist with Continuous Access Journal Z. Continuous Access Journal Z • Continuous Access Synchronous Z is required, whether or not Continuous Access Journal Z shares volumes with Continuous Access Synchronous Z • Continuous Access Journal 3DC & 4x4 Open Z MF in the following Continuous Access Journal Z configurations: - When running a 3DC system Other licenses required The track format for the P-VOL and S-VOL must meet the following requirements: • The Continuous Access Synchronous Z P-VOL and S-VOL must have the same track format. • Record zero (R0) must be standard format, with key length of zero and data length of eight. The primary system aborts the initial copy operation if R0 is not standard format. • The CCHH (logical cylinder address and logical head address) of R0 must be identical to the physical cylinder address and physical head address of the track. • The CCHH of each user record in a track must be unique. Disk Track Format The following program products can be used: • Remote Web Console - Required - Storage Administrator or (Remote Copy) role is required. • RAID Manager • Business Continuity Manager Interfaces System requirements 17

Requirement Item • MVS • OS/390 • z/OS • z/VM • z/Linux Also, please observe the following: • Optional error report communications (ERC) function requires MVS/DFP 3.2.0 or later. • If the primary and/or secondary systems consist of several CPU complexes, a SYSPLEX timer is required to provide a common time reference for the host I/O time stamping function. However, the z/Linux operating system does not support a SYSPLEX timer. As a result, I/O from the z/Linux host is not consistent. Also, journal consistency is not maintained. Contact HP Technical Support for the latest information. Supported mainframe host platforms Fibre channel. Data path • P-VOL and S-VOL must be equal in size. • P-VOL and S-VOL must be of same emulation type. • One Continuous Access Journal Z P-VOL may be copied to one S-VOL only. • When Continuous Access Synchronous Z is cascaded with Continuous Access Journal Z, a data volume may be copied to multiple data centers. Volumes Limited per P9500 system. See “Maximum number of pairs allowed ” (page 30). Maximum number of pairs Continuous Access Journal Z operations between P9500 and previous models are supported. See “Planning for previous models ” (page 36). Supported previous models RAID1, RAID5, RAID6 can co-exist in the data volumes and journal volumes. Supported RAID groups Virtual LVI volume is supported. • Can be used for data and journal volumes. • S-VOL capacity must equal P-VOL capacity. Cache Residency volume is supported as follows: • Data volume: yes • Journal volume: no Supported volumes Must be operable for primary and secondary systems to insure pair creation success. The remote system cache should be configured to adequately support Continuous Access Journal Z remote-copy workloads, as well as local workload activity. In general, cache capacity should be increased 25 percent for Continuous Access Journal Z. Also, an additional GB should be added for each journal on the system. Cache and Nonvolatile Storage (NVS) Required for disaster recovery. Host failover software 18 Requirements and specifications

Requirement Item • A system can have a maximum of four EXCTGs. • An EXCTG can contain a maximum of 16 registered journals. • An EXCTG can contain journals from a maximum of four systems. • A journal cannot be registered in more than one EXCTG. Extended consistency groups (EXCTG) RAID Manager consistency groups when multiple primary and secondary storage system • Max. number: 256 (0 to 255) per storage system • Recommended number: Up to 16 • Max. number of journal volumes: 64 per journal • Max. number of data volumes: 8,192 per journal • Journal numbers of master and restore journals that are paired can be different. • A data volume and associated journal volume can belong to only one journal. • Data volumes and journal volumes that belong to different controllers cannot be in the same journal. • The number of journal volumes in the master journal does not have to be equal to the number of volumes in the restore journal. • The P-VOLs and S-VOLs in a journal must be located in one physical primary system and one physical secondary system (1-to-1 requirement). • Each pair relationship in a journal is called a "Mirror". Each pair is assigned a Mirror ID. Max. number of Mirror IDs = 4 (0 to 3). • When Continuous Access Journal co-exists with Continuous Access Journal Z in the same storage system, individual journals must be dedicated either to one or the other, but not both. • Master and restore journals are managed according to the journal number. NOTE: The capacity of journal volume is not included in the accounting capacity. Journals NOTE: Continuous Access Journal can co-exist with Continuous Access Journal Z in the same storage system. Business Continuity Manager specifications Business Continuity Manager commands for Continuous Access Journal Z pair operations are issued from the host system console to the storage system. With Business Continuity Manager commands, you can add, suspend, resume, and release pairs, as well as monitor pair status. You can also add and delete DKC. The two tables that follow provide values for the P9500 system adapter ID (SAID). For more information, see the manual of Business Continuity Manager. Business Continuity Manager specifications 19

SAID Port Package Local SAID Port Package Local SAID Port Package Local SAID Port Package Local X'0084' CL9-E 1BU (Add14) X'008C' CL9-N 1KU (Add10) X'0008' CL1-J 1GU (Add4) X'0000' CL1-A 1EU (Basic) X'00A4' CLB-E X'00AC' CLB-N X'0028' CL3-J X'0020' CL3-A X'00C4' CLD-E X'00CC' CLD-N X'0048' CL5-J X'0040' CL5-A X'00E4' CLF-E X'00EC' CLF-N X'0068' CL7-J X'0060' CL7-A X'0085' CL9-F X'008D' CL9-P X'0009' CL1-K X'0001' CL1-B X'00A5' CLB-F X'00AD' CLB-P X'0029' CL3-K X'0021' CL3-B X'00C5' CLD-F X'00CD' CLD-P X'0049' CL5-K X'0041' CL5-B X'00E5' CLF-F X'00ED' CLF-P X'0069' CL7-K X'0061' CL7-B X'0086' CL9-G 1BL (Add12) X'008E' CL9-Q 1KL (Add8) X'000A' CL1-L 1GL (Add6) X'0002' CL1-C 1EL (Add2) X'00A6' CLB-G X'00AE' CLB-Q X'002A' CL3-L X'0022' CL3-C X'00C6' CLD-G X'00CE' CLD-Q X'004A' CL5-L X'0042' CL5-C X'00E6' CLF-G X'00EE' CLF-Q X'006A' CL7-L X'0062' CL7-C X'0087' CL9-H X'008F' CL9-R X'000B' CL1-M X'0003' CL1-D X'00A7' CLB-H X'00AF' CLB-R X'002B' CL3-M X'0023' CL3-D X'00C7' CLD-H X'00CF' CLD-R X'004B' CL5-M X'0043' CL5-D X'00E7' CLF-H X'00EF' CLF-R X'006B' CL7-M X'0063' CL7-D X'0080' CL9-A 1AU (Add15) X'0088' CL9-J 1LU X'000C' CL1-N 1HU (Add5) X'0004' CL1-E 1FU (Add1) X'00A0' CLB-A X'00A8' CLB-J (Add11) X'002C' CL3-N X'0024' CL3-E X'00C0' CLD-A X'00C8' CLD-J X'004C' CL5-N X'0044' CL5-E X'00E0' CLF-A X'00E8' CLF-J X'006C' CL7-N X'0064' CL7-E X'0081' CL9-B X'0089' CL9-K X'000D' CL1-P X'0005' CL1-F X'00A1' CLB-B X'00A9' CLB-K X'002D' CL3-P X'0025' CL3-F X'00C1' CLD-B X'00C9' CLD-K X'004D' CL5-P X'0045' CL5-F X'00E1' CLF-B X'00E9' CLF-K X'006D' CL7-P X'0065' CL7-F X'0082' CL9-C 1AL (Add13) X'008A' CL9-L 1LL (Add9) X'000E' CL1-Q 1HL (Add7) X'0006' CL1-G 1FL (Add3) X'00A2' CLB-C X'00AA' CLB-L X'002E' CL3-Q X'0026' CL3-G X'00C2' CLD-C X'00CA' CLD-L X'004E' CL5-Q X'0046' CL5-G X'00E2' CLF-C X'00EA' CLF-L X'006E' CL7-Q X'0066' CL7-G X'0083' CL9-D X'008B' CL9-M X'000F' CL1-R X'0007' CL1-H X'00A3' CLB-D X'00AB' CLB-M X'002F' CL3-R X'0027' CL3-H X'00C3' CLD-D X'00CB' CLD-M X'004F' CL5-R X'0047' CL5-H X'00E3' CLF-D X'00EB' CLF-M X'006F' CL7-R X'0067' CL7-H 20 Requirements and specifications

SAID Port Package Local SAID Port Package Local SAID Port Package Local SAID Port Package Local X'0094' CLA-E 2NU (Add14) X'009C' CLA-N 2WU (Add10) X'0018' CL2-J 2TU (Add4) X'0010' CL2-A 2QU (Basic) X'00B4' CLC-E X'00BC' CLC-N X'0038' CL4-J X'0030' CL4-A X'00D4' CLE-E X'00DC' CLE-N X'0058' CL6-J X'0050' CL6-A X'00F4' CLG-E X'00FC' CLG-N X'0078' CL8-J X'0070' CL8-A X'0095' CLA-F X'009D' CLA-P X'0019' CL2-K X'0011' CL2-B X'00B5' CLC-F X'00BD' CLC-P X'0039' CL4-K X'0031' CL4-B X'00D5' CLE-F X'00DD' CLE-P X'0059' CL6-K X'0051' CL6-B X'00F5' CLG-F X'00FD' CLG-P X'0079' CL8-K X'0071' CL8-B X'0096' CLA-G 2NL (Add12) X'009E' CLA-Q 2WL (Add8) X'001A' CL2-L 2TL (Add6) X'0012' CL2-C 2QL (Add2) X'00B6' CLC-G X'00BE' CLC-Q X'003A' CL4-L X'0032' CL4-C X'00D6' CLE-G X'00DE' CLE-Q X'005A' CL6-L X'0052' CL6-C X'00F6' CLG-G X'00FE' CLG-Q X'007A' CL8-L X'0072' CL8-C X'0097' CLA-H X'009F' CLA-R X'001B' CL2-M X'0013' CL2-D X'00B7' CLC-H X'00BF' CLC-R X'003B' CL4-M X'0033' CL4-D X'00D7' CLE-H X'00DF' CLE-R X'005B' CL6-M X'0053' CL6-D X'00F7' CLG-H X'00FF' CLG-R X'007B' CL8-M X'0073' CL8-D X'0090' CLA-A 2MU (Add15) X'0098' CLA-J 2XU (Add11) X'001C' CL2-N 2UU (Add5) X'0014' CL2-E 2RU (Add1) X'00B0' CLC-A X'00B8' CLC-J X'003C' CL4-N X'0034' CL4-E X'00D0' CLE-A X'00D8' CLE-J X'005C' CL6-N X'0054' CL6-E X'00F0' CLG-A X'00F8' CLG-J X'007C' CL8-N X'0074' CL8-E X'0091' CLA-B X'0099' CLA-K X'001D' CL2-P X'0015' CL2-F X'00B1' CLC-B X'00B9' CLC-K X'003D' CL4-P X'0035' CL4-F X'00D1' CLE-B X'00D9' CLE-K X'005D' CL6-P X'0055' CL6-F X'00F1' CLG-B X'00F9' CLG-K X'007D' CL8-P X'0075' CL8-F X'0092' CLA-C 2ML (Add13) X'009A' CLA-L 2XL (Add9) X'001E' CL2-Q 2UL (Add7) X'0016' CL2-G 2RL (Add3) X'00B2' CLC-C X'00BA' CLC-L X'003E' CL4-Q X'0036' CL4-G X'00D2' CLE-C X'00DA' CLE-L X'005E' CL6-Q X'0056' CL6-G X'00F2' CLG-C X'00FA' CLG-L X'007E' CL8-Q X'0076' CL8-G X'0093' CLA-D X'009B' CLA-M X'001F' CL2-R X'0017' CL2-H X'00B3' CLC-D X'00BB' CLC-M X'003F' CL4-R X'0037' CL4-H X'00D3' CLE-D X'00DB' CLE-M X'005F' CL6-R X'0057' CL6-H X'00F3' CLG-D X'00FB' CLG-M X'007F' CL8-R X'0077' CL8-H Command device To use Business Continuity Manager, you can configure a command device for it separately from the command device for an open system. The command device for Business Continuity Manager Command device 21

Continuity Manager. 22 Requirements and specifications

This chapter provides information and instructions for planning Continuous Access Journal Z volumes, P9500 systems, and other important requirements and restrictions. Plan and design workflow Planning the Continuous Access Journal Z system is tied to your organization’s business requirements and production system workload. This means defining business requirements for disaster downtime and measuring the amount of changed data your system produces over time. With this information, you can calculate the size that journal volumes must be and the amount of bandwidth required to transfer update data over the data path network. The plan and design workflow consists of the following: • Assess your organization’s business requirements to determine recovery requirements. • Measure your host application’s write-workload in MB per second and write-input/output per second (IOPS) to begin matching actual data loads with the future Continuous Access Journal Z system. • Use collected data along with your organization’s recovery point objective (RPO) to size Continuous Access Journal Z journal volumes. Journal volumes must have enough capacity to hold accumulating data over extended periods. The size of the journal volumes is dependent on the amount of bandwidth needed. You adjust journal volume size in conjunction with bandwidth to fit the organization’s needs. • Use IOPS to determine data transfer speed into and out of the journal volumes. Data transfer speed is determined by the number of Fibre Channel ports you assign to Continuous Access Journal Z, and by RAID group configuration. You need to know port transfer capacity and the number of ports that your workload data will require. • Use the collected workload data to determine the bandwidth for the Fibre Channel data path. As mentioned, bandwidth and journal volume sizing, along with data transfer speed, are interrelated. Bandwidth may be adjusted in conjunction with the journal volume capacity and data transfer speed you plan to implement. • Design the data path network configuration. This involves: understanding supported configurations, the need for Fibre Channel switches, and the number of ports your data transfer requires. • Plan data volumes (primary and secondary volumes). This involves understanding the sizing of P-VOL and S-VOL, and RAID group considerations. • Understand operating system requirements for data and journal volumes. • Adjust cache memory capacity for Continuous Access Journal Z. Some tasks will be handled by HP personnel. The planning information you need to address is provided in the following sections. Assessing business requirements for data recovery In a Continuous Access Journal Z system, when the data path continues to transfer changed data to the remote site, journals remain fairly empty. However, if a path failure or a prolonged spike in write-data that is greater than bandwidth occurs, data is stored in the journal. Changed data that is no longer moving to the remote system builds up in the master journal. Plan and design workflow 23

according to the following: • The maximum amount of time that journals could accumulate data. You develop this information by determining your operation’s recovery point objective (RPO). • The amount of changed data that your application generates. This is done by measuring write-workload. Determining your RPO Your operation’s recovery point is the maximum time that can pass after a failure or disaster occurs before data loss is greater than the operation can survive. For example, if the operation can survive one hour’s worth of lost data, and a disaster occurs at 10:00 a.m., then the system must be corrected by 11:00 a.m. In regards to journal sizing, the journal must have the capacity to hold the data that could accumulated in one hour. If RPO is 4 hours, then the journal must be sized to hold 4 hours' worth of accumulating data. To assess RPO, the host application’s write-workload must be known. With write-workload and IOPS, you or your organization’s system administrator can analyze the number of transactions write-workload represents, determine the number of transactions the operation could loose and still remain viable, determine the amount of time required to recover lost data from log files or key it in, and so on. The result is your RPO. Write-workload Write-workload is the amount of data that changes in your production system in MB per second. As you will see, write-workload varies. according to the time of day, week, month, quarter. That is why workload is measured over an extended period. With the measurement data, you can calculate workload averages, locate peak workload, and calculate peak rolling averages, which show an elevated average. With one of these base data you will calculate the amount of data that accumulates over your RPO time, for example, 2 hours. This will be a base capacity for your journal volumes or represent a base amount of bandwidth your system requires. Whether you select average, rolling average, or peak workload is based on the amount of bandwidth you will provide the data path (which is also determined by write-workload). Bandwidth and journal volume capacity work together and depend on your strategy for protecting data. Measuring write-workload Workload data is collected using Performance Monitor or your operating system’s performance-monitoring feature. The number of read/write transactions, or input/output per second (IOPS), is also collected by the software. You will use IOPS to set up a proper data transfer speed, which you insure through RAID group configuration and by establishing the number of Fibre Channel ports your Continuous Access Journal Z system requires. Each RAID group has a maximum transaction throughput; the ports and their microprocessors have an IOPS threshold. Workload and IOPS collection is best performed during the busiest time of month, quarter, and year. This helps you to collect data that shows your system’s actual workloads during high peaks and spikes, when more data is changing, and when the demands on the system are greatest. Collecting data over these periods insures that the Continuous Access Journal Z design you develop will support your system in all workload levels. 24 Planning volumes, systems

1. Using your performance monitoring software, collect the following: • Disk-write bytes-per-second (MB/s) for every physical volume that will be replicated. • Data should be collected over a 3 or 4 week period to cover a normal, full business cycle. • Data should be collected at 5 minute intervals. If you use averages, shorter intervals provide more accuracy. 2. At the end of the collection period, convert the data to MB, if needed, and import into a spreadsheet tool. Sizing journal volumes Determine the required journal volume capacity The following factors determine the required journal volume capacity: • The period of time during which data transfer can continue between hosts and the primary storage system when a temporary increase in transferred data occurs or when a communication path failure occurs between the primary and secondary storage system. • The data transfer speed for the period of time mentioned above. To determine the journal volume capacity, use the following formula: Formula 1 : Journal volume capacity > (VH-M - VM-R) × t where: • VH-M is the data transfer speed between hosts and the primary storage system. • VM-R is the data transfer speed between the primary storage system and the secondary storage system. • t is the length of time during which data transfer can continue. To calculate journal volume capacity that is required when a communication path failure occurs between the primary storage system and the secondary storage system, assign zero (0) to VM-R. The total capacity of journal volumes in each journal group must exceed the value illustrated in formula 1. When the data in the primary data volume of the Continuous Access Journal Z delta resync pair is updated, if the journal data that exceeds 70 percent of the journal volume capacity on the primary site of the Continuous Access Journal Z delta resync pair is generated, the delta resync operation will fail. Therefore, adopt the larger value for the journal volume capacity on the primary site of the Continuous Access Journal Z delta resync pair by comparing the values of formula 1 and formula 2. Formula 2 : Journal volume capacity > (V × t) × 1.5 Sizing journal volumes 25

• t is the length of time until the delta resync operation starts. CAUTION: The journal volume capacity is recommended to be over 6 GB. If the capacity is below 6 GB, the system performance is unassured because the following problems may occur. • The new data cannot be stored because the journal volume is full and host performance is reduced. • The performance of the initial copy goes down because the journal volume is full. • The journal group is suspended because the journal volume is full. • The Usage Monitor value is invalid. Calculating the journal size You calculate the size your journal volumes using the write-workload and RPO. To calculate journal size 1. Follow the instructions for “Measuring write-workload” (page 24). 2. Use your system’s peak write-workload and your organization’s RPO to calculate journal size. For example: RPO = 2 hours Write-workload = 30 MB/sec Calculate write-workload for the RPO. In the example, write-workload over a 2 hours period is calculated as follows: 30 MB/second x 60 seconds = 1800 MB/minute 1800 MB/minute x 60 minutes = 108,000 MB/hour 108000 MB/hour x 2 = 416,000 MB/2 hours Basic journal volume size = 416,000 MB (416 GB) Journal volume capacity and bandwidth size work together. Also, your strategy for protecting your data may allow you to adjust bandwidth or the size of your journal volumes. For a discussion on sizing strategies, see “Five sizing strategies” (page 40). NOTE: If you are planning for disaster recovery, the remote array must be large enough to handle the production workload, and therefore, must be the same size as master journals. If not planning a disaster recovery solution, remote journal volumes may be smaller than master journal volumes. Planning journals Continuous Access Journal Z manages pair operations for data consistency through the use of journals. Continuous Access Journal Z journals enable update sequence consistency to be maintained across a group of volumes. Understanding the consistency requirements for an application (or group of applications) and their volumes will indicate how to structure journals. For example, databases are typically implemented in two sections. The bulk of the data is resident in a central data store, while incoming transactions are written to logs that are subsequently applied to the data store. If the log volume “gets ahead” of the data store, it is possible that transactions could be lost at recovery time. Therefore, to insure a valid recovery image on a replication volume, it is important that both the data store and logs are I/O consistent by placing them in the same journal. 26 Planning volumes, systems

• Review journal specifications in “System requirements” (page 17). • Review journal configuration in “Register journal volumes in a journal” (page 63). Data transfer speed considerations The previous sections and the sections later in this chapter on Bandwidth discuss the amount of data that must be stored temporarily in journals and transferred over the data path network. This section discusses the speed that data must be transferred in order to maintain the Continuous Access Journal Z system your are designing. The ability of your Continuous Access Journal Z system to transfer data in a timely manner depends directly on the following two factors: • RAID group configuration • Fibre Channel port configuration Both of these elements must be planned to be able to handle the amount of data and number of transactions your system will process under extreme conditions. RAID group configuration A RAID group can consist of physical volumes with a different number of revolutions, physical volumes of different capacities, and physical volumes of different RAID configurations (for example, RAID1 and RAID5). The data transfer speed of RAID groups is affected by physical volumes and RAID configurations. • The data transfer speed of a journal volume depends on the data transfer speed of the RAID group to which it belongs. A RAID group can consist of one or more volumes, including journal volumes. • Journal volumes must be configured in RAID groups according to the group’s throughput specification and your system’s peak write-workload. If write-workload exceeds the RAID group’s throughput rating, then the number of RAID groups must be increased. • Frequent read/write activity to non-journal volumes in a RAID group results in fewer read/writes by journal volumes in the same RAID group. This can cause a drop in the data transfer speed of journal volumes. To avoid this effect, place journal volumes and frequently accessed non-journal volumes in different RAID groups. Fibre Channel port configuration The Fibre Channel ports on your P9500 system have an IOPS threshold. Use the performance monitoring information for the number of IOPS your production system generates to calculate the number of Fibre Channel ports the Continuous Access Journal Z system requires. See “Planning ports for data transfer” (page 44) for a full discussion on the type and number of Fibre Channel ports required for your system. Planning journal volumes The following information is provided to help you prepare journal volumes: • Identify the journal volumes for your Continuous Access Journal Z system on primary and secondary arrays. Journal volumes should be sized according to RPO and write-workload. See “Sizing journal volumes” (page 25) for more information. • Journal volumes in the same journal can be of different capacity. A master journal volume and the corresponding restore journal volume can be of different capacity. Data transfer speed considerations 27

area is used for storing metadata. • Journal volumes support all RAID configurations that are supported by P9500. Journal volumes also support all physical volumes that are supported by P9500. • Customized volumes can be used for journal volumes. See “Calculating maximum number of pairs” (page 30) for maximum and minimum supported capacity of journal volumes. See system requirements and specifications in “Requirements and specifications” (page 17) for more information. Host I/O time stamp The I/O time stamping function must be installed on the host processor at the primary site in order to provide data consistency in Continuous Access Journal Z journals. The I/O time stamp, which is provided by MVS DFSMSdfp, is the same time stamp that is used by Compatible XRC pairs. The I/O time stamping function should also be installed on the host processor at the secondary site, so that time stamps can be used when copying data in the reverse direction. If the system at the primary and/or secondary site consists of several CPU complexes, a SYSPLEX timer is required to provide a common time reference for the I/O time stamping function. Planning pair volumes The following information is provided to help you prepare volumes for configuration. Also, see system requirements and specifications in “Requirements and specifications” (page 17) for more information. • The emulation and capacity for the S-VOL must be the same as the P-VOL. • When the S-VOL is connected to the same host as the P-VOL, the S-VOL must be defined to remain offline. • The following CU emulation types can be used for MCUs (primary systems) and RCUs (secondary systems): 2105, 2107. • The CU emulation type 2105, or 2107 is required for SMS I/O time stamping of Continuous Access Journal Z journals. When one of these CU emulation types is used, volumes of most 3380 emulation types must not be used. Only 3380-3 can be used. • Continuous Access Journal Z supports the Virtual LVI feature, which allows you to configure custom-size LDEVs that are smaller than standard-size LDEVs. When custom-size LDEVs are assigned to a Continuous Access Journal Z pair, the S-VOL must have the same capacity as the P-VOL. For details about Virtual LVI feature, see HP StorageWorks P9000 Provisioning for Mainframe Systems User Guide. • Identify the volumes that will become the P-VOLs and S-VOLs. • You can create multiple pairs at the same time. For details, see “Pair operations” (page 67). • When you create a Continuous Access Journal Z pair, you will have the option to create only the relationship, without copying data from primary to secondary volumes. You can use this option only when the contents of the primary and secondary data volume are completely the same. Data and journal volume specifications The following table shows maximum capacities and volume types that can be used for data volumes and journal volumes. 28 Planning volumes, systems

Support specifications Type Journal volume Data volume Available Available Virtual LVI volume Unavailable Available Cache Residency Z volume 2.377 GB 2.377 GB 3380-3 Maximum volume capacity 1.26 GB 1.26 GB 3380-E 0.63 GB 0.63 GB 3380-J 1.890 GB 1.890 GB 3380-K 0.964GB 0.964GB 3390-1 1.892GB 1.892GB 3390-2 2.838GB 2.838GB 3390-3 3390-3R 8.510GB 8.510GB 3390-9 27.80GB 27.80GB 3390-L 55.60GB 55.60GB 3390-M Capacity of OPEN-V volumes can be determined freely, and has dependencies on Virtual LVI volume specifications. The minimum capacity is 48.1 MB, and the maximum capacity is the same as the user capacity of one RAID group. The default capacity of an OPEN-V volume is the same as the capacity of a RAID group, and depends on the hard disk drive type and the RAID configuration. Cannot be used as data volumes. OPEN-V 50 cylinders for a non-OPEN-V volume. 48.1 MB for an OPEN-V volume. 1 cylinder Minimum volume capacity Duplicating VOLSERs When the P-VOL is copied to the S-VOL during the initial copy operation, the P-VOL VOLSER (Volume Serial Number) is also copied to the S-VOL. Therefore, the P-VOL and S-VOL have the same VOLSER. Because the host operating system does not allow duplicate VOLSERs, the host Planning pair volumes 29

(such as defining Continuous Access Journal Z S-VOLs not to come online automatically). CAUTION: If the volumes that become Continuous Access Journal Z S-VOLs are physically attached to the same system images as the volumes that become the Continuous Access Journal Z P-VOLs, the potential for error increases. For example, when a Continuous Access Journal Z pair is released, the old S-VOL is usually offline. When a host system is IPL’d (initial program loaded), the operator is offered both volumes and asked which volume should be left offline. You can avoid duplication of the VOLSER as follows: 1. Identify the volumes that are not accessed by the host system. 2. Perform CHP OFF or some other operation to insure that the volumes are inaccessible. 3. When performing IPL, make sure to perform LOAD CLEAR. Maximum number of pairs allowed P9500 has a limit on the number of pairs that can be created. Therefore, it is necessary to calculate the maximum number of pairs on the P9500 storage system. The maximum number is limited according to the following: 1. The number of cylinders in the volumes, which must be calculated. 2. The number of bitmap areas required for Continuous Access Journal data and journal volumes. This is calculated using the number of cylinders. 3. The maximum number of pairs that can be created, calculated using the number of required bitmap areas. CAUTION: The bitmap areas that are used for Continuous Access Journal are also used for Continuous Access Journal Z, Continuous Access Synchronous, Continuous Access Synchronous Z, and External Storage Access Manager. If you use Continuous Access Journal with Continuous Access Journal Z, Continuous Access Synchronous, and Continuous Access Synchronous Z, use the total number of each pair’s bitmap areas to calculate the maximum number of pairs. In addition, if Continuous Access Journal and Continuous Access Synchronous share the same volume, use the total number of both pairs regardless of whether the shared volume is primary or secondary. Calculating maximum number of pairs NOTE: In the calculations below, ceil() indicate that the value between the parentheses should be rounded up to the nearest integer. To determine the number of cylinders The number of cylinders according to the emulation type are listed below. Table 5 Number of cylinders per emulation type Number of cylinders Emulation type 3,339 3380-3 3,339 3390-3, 3390-3R 10,017 3390-9 32,760 3390-L 65,520 3390-M To calculate the number of required bitmap areas: 30 Planning volumes, systems

Required number of bitmap areas = (ceil((number of cylinders x 15) 122,752) ) where: “number of cylinders x 15” indicates the number of slots 122,752 = the number of slots that a bitmap area can manage To calculate the maximum number of pairs that can be created The maximum number of pairs you can create is defined by the following: • The number of LDEVs in the storage system Table 6 The relationship between installed additional shared memory and the total number of LDEVs in the storage system Total number of LDEVs in storage system Installed additional shared memory for Cnt Ac-J Z 16,384 Base(16KLDEV,BC/VM) 65,280 64KLDEV,BC/VM Extension1,FCV2,DP,Snapshot,TPF • The number of bitmap areas required to create pairs (determined above) • The number of bitmap areas of the storage system is 65,536. Use the following formulae to calculate the maximum possible number of pairs that can be created, on the basis of the number of bitmap areas and the required number of bitmap areas you calculated: Maximum number of pairs = floor( Number of bitmap areas required number of bitmap areas ) The maximum number of pairs is limited to 32,768. If the calculated maximum number of pairs exceeds the total number of LDEVs of the storage system, the total number of LDEVs in the storage system is the maximum number of pairs that can be created. Priority set for initial copy operations and scheduling order For performing more initial copy operations than the setting value of the Maximum Initial Copy Activities set on the System Option dialog box, the scheduling order (Priority) can be set for the initial copy operations to be performed. This section explains the assignment of the scheduling order of initial copy operations in cases where two sets of multiple initial copy operations are performed. An example is described in which 4 data volume pairs are created at the same time in cases where the setting value of the Maximum Initial Copy Activities is 2. The Priority which is set for the P-VOLs of data volume pairs is shown in the following table. Table 7 Priority set for P-VOLs for which initial copy operation is performed Value set for Priority P-VOL 2 LDEV 00 3 LDEV 01 1 LDEV 02 4 LDEV 03 The order of starting initial copy and the Priority which is set for the P-VOLs are shown in the following table. Planning pair volumes 31

P-VOLs Value set for Priority P-VOL Order of starting initial copy 1 LDEV 02 1 2 LDEV 00 2 3 LDEV 01 3 4 LDEV 03 4 In this case, as the setting value of the Maximum Initial Copy Activities is 2, initial copy operations for LDEV 02 and LDEV 00 are started. If either one of the initial copy operations for LDEV 02 and LDEV 00 is completed, the initial copy for LDEV 01 is started. The next example explains the assignment of the scheduling order of initial copy operations in cases where initial copy is already performed and two volume pairs are newly added. The P-VOLs of the data volume pairs to be newly added and the Priority are shown in the following table. Table 9 Priority set for P-VOLs of data volume pairs to be added Value set for Priority P-VOL 2 LDEV 10 1 LDEV 11 If initial copy is already started and if any initial copy is newly added, the additional initial copy is started after the previously performed initial copy is completed. The Priority of all the initial copy operations being performed is shown in the following table. Table 10 Order of starting initial copy being performed Remark Value set for Priority P-VOL Order of starting initial copy Data volume pair for which initial copy is already instructed to be performed 1 LDEV 02 1 Data volume pair for which initial copy is already instructed to be performed 2 LDEV 00 2 Data volume pair for which initial copy is already instructed to be performed 3 LDEV 01 3 Data volume pair for which initial copy is already instructed to be performed 4 LDEV 03 4 Data volume pair for which initial copy is newly instructed to be performed 1 LDEV 11 5 Data volume pair for which initial copy is newly instructed to be performed 2 LDEV 10 6 Priority is determined within the range of the number of initial copy operations performed at the same time. Therefore, until the first initial copy operations complying with the order of the Priority are completed, the additional initial copy operations are not started. 32 Planning volumes, systems

You define a disaster recovery solution when planning the Continuous Access Journal Z system. The following are the main tasks for preparing for disaster recovery: • Identify the data volumes that you wand to back up for disaster recovery. • Pair the important volumes using Continuous Access Journal Z. • Establish file and database recovery procedures. • Install and configure host failover software error reporting communications (ERC) between the primary and secondary sites. For more information on host failover error reporting, see the following section. Also, review “Disaster recovery operations ” (page 105) to become familiar with disaster recovery processes. Error reporting communications (ERC) Error reporting communications (ERC) transfers information between host processors at the primary and secondary sites. It is a critical component of any disaster recovery effort. You can configure ERC using channel-to-channel communications, NetView technology, or other interconnect technologies, depending on your installation requirements and standards. Neither Continuous Access Journal Z nor Remote Web Console provides ERC between the primary and secondary sites. When Continuous Access Journal Z is used as a data migration tool, ERC is recommended, but not required. When it is used as a disaster recovery tool, ERC is required to insure effective disaster recovery operations. When a pair is suspended due to an error condition, the primary system generates sense information which results in an IEA491E system console message. This information should be transferred to the primary site via the ERC for effective disaster detection and recovery. Sharing volumes with Continuous Access Synchronous Z, other program products Continuous Access Journal Z volumes can be shared with other program product volumes. Sharing pair volumes results in enhanced replication solutions, for example when Continuous Access Journal Z and Continuous Access Synchronous Z or Business Copy Z volumes are shared. For planning information, see the following: • “Continuous Access Journal Z configurations with Continuous Access Synchronous Z” (page 133) • “Continuous Access Journal Z configurations with Business Copy Z” (page 144) • “Sharing Continuous Access Journal Z volumes ” (page 129) for the following program products: Virtual LVI ◦ ◦ Cache Residency Z Planning for Continuous Access Journal Z with multiple P9500 systems A Continuous Access Journal Z system can be configured with up to four primary P9500 systems and four secondary P9500 system. Any combination of primary and secondary systems, from one to four, may be used. Figure 4 (page 34) shows an example configuration with multiple primary and secondary systems. Disaster recovery considerations 33

When primary hosts write data to P-VOLs, the hosts add a time stamp to the data. Secondary systems check time stamps and then restore data to data volumes in chronological order (older data are restored earlier), so that data update sequence is maintained. For details on the time stamping function, see “Host I/O time stamp” (page 28). The term "arbitration processing" refers to the maintenance of the data update sequence through the execution of the journal restore function based on time stamps. When there is more than one secondary system, one of them is used to perform tasks, including arbitration processing, on the other secondary systems. The secondary system that performs arbitration processing is called the “supervisor DKC.” Systems that are controlled by the supervisor DKC and are targets of arbitration processing are called “subordinate DKCs.” The supervisor DKC compares time stamps of data received by all the secondary systems (including the local system) and then performs arbitration processing. The supervisor DKC must be connected with the subordinate DKCs. For details on connections between secondary systems, see Figure 4 (page 34). Please note the following when working with multiple systems: • Remote Web Console computers must be installed at primary and secondary sites. • BCM should be installed on the host in both the primary and secondary sites. • A remote command device must be created in the supervisor DKC. For details on secondary systems connections and remote command devices, see the next section and HP StorageWorks P9000 External Storage for Open and Mainframe Systems User Guide. • Journal data is updated in the secondary systems based on the same time stamps and sequence numbers issued by the host when the host issues the write requests to the primary system(s). Time and sequence information remains with the data as it moves to the master and restore journals and then to the secondary volume. 34 Planning volumes, systems

When using more than one primary system and more than one secondary system, connections among the secondary systems must be established by configuring paths and ports. To configure ports and paths between secondary systems 1. Create a command device in each of the secondary systems. 2. Configure external ports on the secondary system to be used as the supervisor DKC. 3. Configure paths between the external ports and the target ports on the systems to be used as subordinate DKCs. In the example in Figure 5 (page 35), the secondary system 1 has external ports, each of which is connected with a target port on the secondary systems 2 and 3. For more information about external ports, see HP StorageWorks P9000 External Storage for Open and Mainframe Systems User Guide. For details on configuring paths, see HP StorageWorks P9000 Provisioning for Open Systems User Guide. 4. Create mapping between command devices in the supervisor DKC and the subordinate DKCs. Figure 5 (page 35) shows an example. The supervisor DKC must be able to use command devices in subordinate DKCs via remote command devices. In the example, command devices A and B are created in secondary systems 2 and 3. Remote command devices are created in the secondary system 1 (the supervisor DKC), and are mapped to subordinate DKCs. The emulation type of command devices and remote command devices must be OPEN-V. For more information about remote command devices, see HP StorageWorks P9000 External Storage for Open and Mainframe Systems User Guide. Figure 5 Connections between Supervisor, Subordinate DKCs By using Fibre Channel switches, target ports can also be connected to RCU target ports on the secondary systems. See “Planning ports for data transfer” (page 44) for more information. CAUTION: Performing maintenance operations on remote command devices (for example, the devices A' and B' in above figure) used for connections between secondary systems cause the pairs to be suspended as if a failure had occurred. To avoid this, before performing maintenance tasks, remove all journals in the extended consistency group that use the remote command devices under maintenance. Planning extended consistency groups (EXCTG) An EXCTG is a collection of journals in which data consistency is guaranteed. You register journals in an EXCTG when performing copy operations between multiple primary systems and multiple Planning for Continuous Access Journal Z with multiple P9500 systems 35

information. Planning for previous models Continuous Access Journal Z can be used to perform remote copy operations between P9500 and XP24000/XP20000 Disk Array. Data can be copied from P9500 to XP24000/XP20000 Disk Array, or vice versa. To execute remote copy from P9500 to XP24000/XP20000 Disk Array or vice versa, set up the following: • Configure a logical path between LDKC00 of the P9500 system and the XP24000/XP20000 Disk Array. ◦ More than one XP24000/XP20000 Disk Array can be connected to LDKC00 of P9500. ◦ LDKC01 cannot be used. You can use the configuration instructions in this guide to set up the system. See “Configuration operations” (page 56) for information. Figure 6 Logical path between LDKC00 of P9500 and XP24000/XP20000 Disk Array • P9500 corresponds to a new system of the above-mentioned figure. XP24000/XP20000 Disk Array corresponds to a new system of the above-mentioned figure. • Set up the P9500 volume using a CU:LDEV number between 00:00 to EF:FF. As mentioned above, the volume must be on LDKC00. • Up to 32,768 volumes can be used for volume pairs. • P9500 and XP24000/XP20000 Disk Arrays can be set up in 3-data center (3DC) cascading or multitarget configurations. These configurations are used when combining Continuous Access Synchronous Z and Continuous Access Journal Z systems. See “Continuous Access Journal Z configurations with Continuous Access Synchronous Z” (page 133) to review these configurations. There are no restrictions for combining primary and secondary sites between P9500 and XP24000/XP20000 Disk Arrays. • • If you connect a P9500 with an XP12000 Disk Array, or a P9500 with an XP24000/XP20000/XP10000 Disk Array, contact your HP representative for information on the XP12000 Disk Array and the XP24000/XP20000/XP10000 Disk Array DKCMAIN program verison. • If you connect a P9500 with an XP12000 Disk Array, CU:LDEV number from 00:00 to 3F:FF can be used for P9500, but 40:00 or larger cannot be used. 36 Planning volumes, systems

Use the following guidelines to insure that your P9500 systems are ready for Continuous Access Journal Z: • Identify the locations where your Continuous Access Journal Z primary and secondary data volumes will be located, then install and configure the P9500 systems. • Make sure that primary and secondary systems are configured for P9500 Remote Web Console operations. See HP StorageWorks P9000 Remote Web Console User Guide for information. • Make sure that primary and secondary systems are properly configured for Continuous Access Journal Z operations; for example, cache memory considerations. See the item for Cache and Nonvolatile Storage in the requirements table, “System requirements” (page 17). Also consider the amount of Cache Residency data to be stored in cache when determining the required amount of cache. • Make sure that primary and secondary systems have the system option modes specified that may be required for your Continuous Access Journal Z configuration. See “System option modes” (page 37), below for more information. • Make sure that primary systems are configured to report sense information to the host. Secondary systems should also be attached to a host server to enable reporting of sense information in the event of a problem with a S-VOL or secondary system. If the remote system is not attached to a host, it should be attached to a primary site host server so that monitoring can be performed. • If power sequence control cables are used, set the power select switch for the cluster to LOCAL to prevent the primary system from being powered off by the host. Make sure the secondary system is not powered off during Continuous Access Journal Z operations. • Install the Continuous Access Journal Z remote copy connections (optical Fiber cables, switches, etc.) between the primary and secondary systems. • When setting up data paths, distribute them between different storage clusters and switches to provide maximum flexibility and availability. The logical paths between the primary and secondary systems must be separate from the logical paths between the host and secondary system. System option modes HP provides system option modes to tailor customer requirements on the P9500 system. System option modes are used with Continuous Access Journal Z for the following: • Delta Resync configuration using Continuous Access Journal Z and Continuous Access Synchronous Z • Configuring split options for mirrors System option modes can only be changed by a HP Technical Service representative. Table 11 System option modes Description Mode Mode 20 ON: Enables the read option and VOLSER of the suspended secondary data volume to be updated. Mode 20 OFF: Normal operations are performed. 20 Mode 190 ON: By setting both this mode and the mode 20 to ON, you can update VOLSER and VTOC of the suspended secondary data volume. Mode 190 OFF: Normal operations are performed. 190 Guidelines for preparing systems for Continuous Access Journal Z 37

Description Mode Mode 448 ON: If the SVP detects a blocked path, the SVP assumes that an error occurred, and then immediately splits (suspends) the mirror. Mode 448 OFF: If the SVP detects a blocked path and the path does not recover within the specified period of time, the SVP assumes that an error occurred, and then splits (suspends) the mirror. NOTE: The mode 448 setting is available only when mode 449 is set to OFF. 448 Mode 449 ON: The SVP does not detect blocked paths. Mode 449 OFF: The SVP detects blocked paths and monitors the time until the mirrors get split (suspended). 449 Allows you to perform the initial copy operation of the Continuous Access Journal Z pair for the suspended Continuous Access Journal Z pair. Use the script file in which the following operations are described: Initial copy operation of the Continuous Access Journal Z pair (None) -> Suspend the Continuous Access Journal Z pair -> Initial copy operation of the Continuous Access Journal Z pair -> Delete the Continuous Access Journal Z pair -> Resynchronize the Continuous Access Journal Z pair. The performance of the initial copy processing of the Continuous Access Journal Z pair can be improved by issuing the command from Business Continuity Manager. Mode 474 ON: The performance of the initial copy processing of the Continuous Access Journal Z pair can be improved by issuing the command from Business Continuity Manager. Mode 474 OFF: Normal operations are performed. 474 To create a Continuous Access Journal Z pair for delta resync operation in the 3DC multitarget configuration, set this mode to ON. Mode 506 ON: The delta resync operation is performed if there are no update I/Os. Mode 506 OFF: The copy processing of all data is performed if there is no update I/Os. 506 When creating a Continuous Access Journal Z pair for delta resync operation in the 3DC multitarget configuration, and if you want to control the ratio of the usage rate of difference journal data in the journal volume in the primary site, set this mode to ON. Mode 511 ON: The upper limit of the usage rate of the journal data in the journal volume is 40%. Mode 511 OFF: The upper limit of the usage rate of the journal data in the journal volume is 70%. 511 To change the displayed consistency time (C/T) when the Continuous Access Journal Z pair status is Duplex, set this mode to ON. Mode 530 ON: The copy completion time from the master journal volume to the restore journal volume of the journal data is displayed in C/T. Mode 530 OFF: The completion time when the journal data is restored to the data volume is displayed in C/T. 530 38 Planning volumes, systems

Description Mode In the configuration where multiple MCUs share an RCU (Continuous Access Journal Z Mx1 configuration), the mirror resynchronization or delta resync operation may fail due to SSB=EB9D. SSB=EB9D is output as SSB log when the parameter of the communication command is not saved in the shared memory. To reduce the number of multiprocessing of the communication command issued from MCU to RCU for the status transition of each pair, set this mode to ON. By setting this mode to ON, you can prevent mirror resynchronization or delta resync operation from failing. Mode 593 ON: The multiprocessing of the communication command issued from MCU to RCU is up to 128 pairs per DKC. Mode 593 OFF: The multiprocessing of the communication command issued from MCU to RCU is up to 1024 pairs per DKC. 593 The host I/O response might decrease while the Continuous Access Journal Z pair is resynchronized. To suppress increase in the MP utilization rate by the differential bitmap retrieval processing when resynchronizing the pair, set this mode to ON. You can reduce host I/O response by setting this mode to ON. Mode 760 ON: Decrease in host I/O response during the Continuous Access Journal Z resync operation is reduced. Mode 760 OFF: Normal operations are performed. 760 Use this mode in the Continuous Access Journal Z extended consistency group configuration. In the case where Business Continuity Manager issues the Flush request to the primary data volume of the Continuous Access Journal Z pair, and the Continuous Access Journal Z pair receives the suspend command, if you want to reflect the data up to the time specified in the command to the secondary data volume, set this mode to ON. Mode 762 ON: When the Continuous Access Journal Z pair receives the suspend command, the data up to the time specified in the suspend command is reflected to the secondary data volume. Mode 762 OFF: The time stamp of the latest data received in each restore journal group may be different. In this case, the time stamps of data in restore journals are compared, and data up to the oldest time stamp is reflected to secondary data volumes. 762 Guidelines for preparing systems for Continuous Access Journal Z 39

A data path must be designed to manage your organization’s throughput to the remote site. Appropriate bandwidth, required number of ports, and the Fiber Channel data path configuration you use help insure that your update data arrives at the remote site in a time consistent with your organization’s RPO. This chapter provides instructions for calculating bandwidth and designing the data path network. Data path design workflow To set up a data path, you need to establish the following: • The amount of bandwidth necessary to move the data generated by your host application under all I/O conditions. • Ports that can either send or receive data • Understand Fibre Channel types and number of switches required • A data path Sizing bandwidth You purchase bandwidth according to the amount of data that will be transferred from the primary to the secondary system within a certain amount of time. If the data path network cannot keep pace with the flow of data, the data is saved in the journal until additional bandwidth capacity becomes available. If the journal also cannot keep up, the integrity of the pairs is lost and a new initial copy must be created. In general, bandwidth is expensive. Adding capacity to a journal volume is relatively inexpensive. But the more data that accumulates in the journal, the further the secondary image is from the production volumes. Therefore, sizing bandwidth is a trade-off between expense and keeping your secondary volume as consistent as you need it to be with the primary volume. Five sizing strategies The following sizing strategies are provided to help you work out an approach to sizing bandwidth. Be aware that these are not the only strategies you can use. • Size bandwidth to peak workload. This results in the smallest difference between data in the P-VOL and S-VOL. Identify peak workload on the production disks, then add extra capacity to accommodate packet loss and protocol overhead. RPO is at or near zero when bandwidth is sized to peak workload. • Size bandwidth to peak workload rolling average. The rolling average is less than peak but more than average. This guarantees that at some point data will accumulate in the journal, but most of the time it will not. Your system can afford to journal for the amount of time planned for and still maintain RPO. • Size bandwidth to typical workload. When bandwidth is sized to typical write-workload, and an extended peak workload is experienced, excess write-data is written to journal. This excess data is delayed for subsequent FIFO transmission to the remote site when network capacity becomes available. Differential data is proportional to the amplitude and duration of the workload surge. • If you cannot determine a “typical” workload, sizing should be to the average or mean workload, plus a small compensation for network overhead. In this scenario, excess data in 40 Planning the data path

below average write-workload, the journals never fully drain and eventually overflow. • You can size bandwidth and journal size for long-haul data migration, to be used where data consistency is not required. In this strategy, you alternate pair status between resync and suspend in order to “batch copy” point-in-time copies. When pairs are suspended, journals are not used to queue write operations. Rather, a bitmap is used to track which cylinders have changed on the physical disks. For access patterns that favor multiple writes to a relatively small region of disk, this technique can provide especially efficient transfer of data, since multiple writes to one region are not sent each and every time. Only the last update before resync is sent. The disadvantage of this strategy is that it does not guarantee I/O consistency on the secondary system until the resync completes. Calculating bandwidth To determine bandwidth for Continuous Access Journal Z, write-workload must be measured. Production system workload data is collected using performance monitoring software. Please see “Measuring write-workload” (page 24). When you have collected write-workload data, size your bandwidth according to your sizing strategy. In the following procedures, bandwidth is sized for peak and peak rolling average write-workload. Sizing bandwidth for peak write-workload 1. Make sure that write-workload data is imported into a spreadsheet tool. Column C in the figure below shows an example of collected raw data over 10-minute segments. Figure 7 Write-workload spreadsheet 2. Locate the highest peak. Based on your write-workload measurements, this is the greatest amount of data transferred during the collection period. It indicates the base amount of data that your bandwidth must be able to handle for near 0 RPO. Though the highest peak is used for determining bandwidth, you should take notice of extremely high peaks. In some cases a batch job, defragmentation, or other process could be driving workload to abnormally high levels. It is sometimes worthwhile to review the processes that are running. After careful analysis, it may be possible to lower or even eliminate some spikes Sizing bandwidth 41

may lower workload. 3. With a base bandwidth value established, make adjustments for growth and a safety factor. • Projected growth rate accounts for the increase expected in write-workload over a 1, 2, or 3 year period. • A safety factor adds extra bandwidth for unusually high spikes that did not occur during write-workload measurement but could. Sizing bandwidth for peak rolling average write-workload 1. Using write-workload data imported into a spreadsheet and your RPO, calculate write rolling-averages. For example, if RPO time is 1 hour, then 60-minute rolling averages are calculated. Do this by arranging the values in six 10-minute intervals, as follows: a. In cell E4 type =average(b2:b7), and press Enter. (Most spreadsheet tools have an average function.) This instructs the tool to calculate the average value in cells B2 through B7 (six 10-minute intervals) and populate cell E4 with that data. (The calculations used here are for example purposes only. Base your calculations on your RPO.) b. Copy the value that displays in E4. c. Highlight cells E5 to the last E cell of workload data in the spreadsheet. d. Right-click the highlighted cells and select the Paste option. Excel maintains the logic and increments the formula values initially entered in E4. It then calculates all the 60-minute averages for every 10-minute increment, and populates the E cells. This is shown in the following figure, which is an example using Microsoft Excel. For comparison, 24-hour rolling averages are also shown. Figure 8 Rolling averages calculated using 60-minute RPO For another perspective, you can graph the data, as shown in Figure 9 (page 43). 42 Planning the data path

2. From the spreadsheet or graph, locate the largest or highest rolling average value. This is the peak rolling average, which indicates the base amount of data that your bandwidth must be able to handle. 3. With a base bandwidth value established, make adjustments for growth and a safety factor. • Projected growth rate accounts for the increase expected in write-workload over a 1, 2, or 3 year period. • A safety factor adds extra bandwidth for unusually high spikes that did not occur during write-workload measurement but could. Latency Network latency affects replication. It is the amount of data that can be present in the data path. In the event of network failure, a certain number of transmitted records will not yet be resident in the secondary system’s journal because they are still in-route within the data path. During periods of low workload, there may be no records in the path, but during periods of heavy workload, the network is fully used. This amount represents the minimum difference between data in the primary and secondary systems. Packet loss Packet losses have the effect of reducing overall bandwidth because lost packets must be retransmitted, which consumes network capacity that would otherwise be occupied by new data traffic. Also, a network can elongate consistency time, since journals are not applied until a contiguous sequence of records has arrived at the secondary site. Sizing bandwidth 43

Updated data is transferred along the data path from the initiator and RCU target ports on the primary and secondary systems. • Initiator ports send data. • RCU target ports receive data. • The amount of data each of these ports can transmit is limited. You must know the amount of data that will be transferred during peak periods. Then, you need to ensure that a sufficient number of your systems’ ports are specified as initiator and RCU target ports to handle the potential amount of data. The Fibre Channel ports on P9500 systems can have the following attributes: • Target port: Connects the storage system and an open systems host. When the host issues a write request, the request is sent to a volume on the P9500 system via a target port on the storage system. Target ports are not used with mainframe volumes or Continuous Access Journal Z operations. This port can be changed to an initiator, or RCU target port. • Initiator port: Sends Continuous Access Journal Z commands to the paired storage system. initiator ports must be configured on both primary and remote storage systems for Continuous Access Journal Z operations. • RCU Target port: Receives Continuous Access Journal Z commands and data. RCU target ports must be configured on both primary and remote storage systems for Continuous Access Journal Z operations. • External port: Required for External Storage copy operations. This port is not used for Continuous Access Journal Z copy operations. This port can be changed to a target, initiator, or RCU target port. NOTE: Two or more initiator ports must be configured before you can add the secondary storage systems and create the Continuous Access Journal Z volume pairs. Determining required number of ports The primary and secondary systems require both initiator and RCU target ports. Both systems send and receive traffic during active replication. • The initiator port on the primary side is connected to the RCU target port on the secondary side. • The initiator port on the secondary side is connected to the RCU target port on the primary side. Adding more ports increases the amount of data that can be transferred between systems. To determine the required number of initiator and RCU target ports P9500 ports are designated as target ports by default. When you determine the number of initiator ports and RCU target ports your Continuous Access Journal Z system requires, you change port attributes from target to initiator and RCU target. 1. Maximum IOPS threshold is set by the factory for the Fibre Channel ports and their microprocessors. One microprocessor manages the data traffic for two Fibre Channel ports. Contact your factory representative to find out the IOPS threshold of your port microprocessors. 2. Measure IOPS for your production system. IOPS is provided when write-workload is measured. (See “Measuring write-workload” (page 24).) 44 Planning the data path

write-IOPS. Using peak IOPS and the IOPS threshold of your systems’ port microprocessors, calculate the number of microprocessors (and Fibre Channel ports) the Continuous Access Journal Z system requires. For example: if your performance monitoring software shows peak IOPS at 10,000, and if factory specifications show that microprocessors have a threshold of 7,000 IOPS, then the Continuous Access Journal Z system would require another port. The following shows the calculation: Peak IOPS / IOPS threshold = Number of microprocessors required ( x 2 = number of Fibre Channel ports required) On setting up ports The following tips are provided to help you design your port configuration: • When new data exists, the initiator port on the local system sends informational commands to the secondary RCU target port. • The remote system initiator port sends journal query commands to the local RCU target port. When new data exists in the primary system, the data is sent through these ports — that is, from primary side RCU target port(s) to secondary system initiator port(s). An example configuration could consist of the following: ◦ Two initiator ports on the local side, with two matching RCU target ports on the remote side ◦ Four initiator ports on the remote side, with four matching RCU target ports on the local side. An initiator port on one side must have an RCU target port on the other side. • If your system will support a disaster recovery failover/failback environment, initiator and RCU target ports should be sized equivalently on primary and secondary systems. • Up to eight paths can be established in both directions. HP strongly recommends that you establish at least two independent data paths to provide hardware redundancy. • Host paths cannot be connected to an initiator port. However, they can be attached to RCU target ports. NOTE: Two or more initiator ports must be configured before you can add the secondary storage systems and create the Continuous Access Journal Z volume pairs. Distances supported for Fibre Channel type, number of switches Fibre channel multimode or single-mode optical fiber cables are required on primary and secondary systems. The type of cable and number of switches depends on the distance between primary and secondary sites. • 0 to 0.5 km: multimode optical shortwave fiber cables are required. • 0.5 km to 1.5 km (1,640 to 4,920 feet): multimode shortwave Fibre Channel interface cables are required; one switch is required, two maximum. • 1.5 km to 10 km: single optical long wave fiber cables are required. • 10 km to 30 km (6.2 to 18.6 miles): single-mode long wave fiber-channel interface cables are required; one switch is required, two maximum. • Greater than 30 km (18.6 miles): approved third-party channel extender products and telecommunications lines are required. Distances supported for Fibre Channel type, number of switches 45

Figure 10 Data path types, switches, and distances Supported data path configurations The remote data path can be configured in one of the following connection types. But for switch connection, the port should be set to Fabric on, Point-to-Point (F-port). • Direct connection, as shown in Figure 11 (page 46). Direct connection (loop only) is a direct link between the local and remote arrays. NL-port (Node Loop) are supported for the data path and host-to-system path. • Switch connection, as shown in Figure 12 (page 47). Switch connections push data from the local switch through a Fibre Channel link across a WAN to the remote switch and Fibre Channel to the remote system. F-port (Point-to-Point) and FL-port (Loop) switch connections are supported. • Extender connection, as shown in Figure 13 (page 47). Figure 11 Direct connection configuration 46 Planning the data path

Figure 13 Extender connection configuration NOTE: • When an MCU and RCU are connected using switches with channel extenders, and multiple data paths are assembled, the capacity of data to be transmitted may concentrate on particular switches, depending on the configuration and the settings of switch routing. • Make sure that your channel extenders are capable of supporting remote I/O. For further details, contact HP technical support. Supported data path configurations 47

When performing Continuous Access Journal Z operations between multiple primary and secondary storage systems, the journals must be in extended consistency groups (EXCTGs). This chapter provides information and instructions for setting up and maintaining journals in EXCTGs. Overview An EXCTG is a collection of journals in which data consistency is guaranteed. You register journals in an EXCTG when performing copy operations between multiple primary storage systems and multiple secondary systems. The primary EXCTG is the primary system EXCTG and the secondary EXCTG is the secondary system EXCTG. Normally, only the secondary EXCTG requires configuration. However, because the P-VOLs in the primary EXCTG could be used as S-VOLs if a failure occurs, primary EXCTGs should also be configured. Configuring both primary and secondary EXCTGs provides the following benefits: • Guaranteed Consistency of Data Update Sequence. Continuous Access Journal Z restores journal data to S-VOLs by taking the following steps: 1. Checks the EXCTG for the time stamps of all journal data not restored to S-VOLs; then identifies the latest time stamp for each journal. In the example in the next figure, the latest time stamp for each journal is shown as follows: ◦ In Journal 1, the latest time stamp is 15:00. ◦ In Journal 2, the latest time stamp is 15:02. ◦ In Journal 3, the latest time stamp is 15:03. ◦ In Journal 4, the latest time stamp is 15:04. Figure 14 EXCTG time stamps 48 Using extended consistency groups

time to the secondary volumes. In the example shown in figure, the oldest time stamp is 15:00. Continuous Access Journal Z restores all data that have a time stamp 15:00 or earlier to the S-VOLs. ◦ For Journal 1, Continuous Access Journal Z restores all data up to 15:00. ◦ For Journal 2, Continuous Access Journal Z restores all data up to 14:02. ◦ For Journal 3, Continuous Access Journal Z restores all data up to 14:03. ◦ For Journal 4, Continuous Access Journal Z restores all data up to 14:04. • Consistency time. Remote Web Console displays the consistency times of EXCTGs, journals, and pairs in the Continuous Access Journal Z windows. ◦ The consistency time of an EXCTG is the latest time stamp of the restored data in the group in which consistency is guaranteed. In the example shown above, the consistency time of the EXCTG is 15:00. ◦ The consistency time of a journal is the latest time stamp of the restored data. In the example figure, the consistency times of journals 1 to 4 are as follows: – The consistency time of Journal 1 is 15:00. – The consistency time of Journal 2 is 14:02. – The consistency time of Journal 3 is 14:03. – The consistency time of Journal 4 is 14:04. ◦ The consistency time of a pair is the latest time stamp of the data that has been restored when the pair becomes suspended. In the example figure, if a pair in the journal 1, 2, 3 or 4 is suspended immediately after data are restored, the consistency time of the pair is as follows: – If a pair in Journal 1 is suspended, the consistency time is 15:00. – If a pair in Journal 2 is suspended, the consistency time is 14:02. – If a pair in Journal 3 is suspended, the consistency time is 14:03. – If a pair in Journal 4 is suspended, the consistency time is 14:04. If a failure occurs in a primary system, restore journal data with time stamps later than the consistency time of the EXCTG, to the S-VOLs. In the example figure, the consistency time of the EXCTG is 15:00; therefore, you must restore the following data to S-VOLs: ◦ Data with the time stamp 15:02 in journal 2. ◦ Data with the time stamp 15:03 in journal 3. ◦ Data with the time stamp 15:04 in journal 4. If a failure occurs in a secondary system, compare the consistency times of all journals in the EXCTG, identify the oldest consistency time, then restore all data with time stamps later than the oldest consistency time, to the S-VOL. In the example figure above, the consistency time of journal 2 is the oldest among journals 1 to 4. Since the consistency time of journal 2 is 14:02, you must restore the following data to S-VOLs. ◦ Data with the time stamp 15:00 in journal 1. ◦ Data with the time stamp 15:02 in journal 2. Overview 49

◦ Data with the time stamp 14:04, and data with the time stamp 15:04 in journal 4. Register journals in an EXCTG When performing Continuous Access Journal Z copy operations between multiple primary and secondary storage systems, you must register the journals in an EXCTG. Prerequisite information • Set up primary and secondary EXCTGs using Business Continuity Manager prior to registering journals. • journals must be in Active or Stopped status. • Register master and restore journals in the separate EXCTGs. • When Continuous Access Journal Z pairs are cascaded with Business Copy Z, and the Business Copy Z T-VOL and a Continuous Access Journal Z P-VOL share the volume, data consistency cannot be maintained when between primary and secondary systems, even when journals are registered in an EXCTG. • When Continuous Access Journal Z pairs are cascaded with Continuous Access Synchronous Z, and the Continuous Access Synchronous Z R-VOL and Continuous Access Journal Z P-VOL share the volume, make sure you enable the time stamp transferring option of all the Continuous Access Synchronous Z pairs in the journal for data consistency. To register journals in an EXCTG 1. In Remote Web Console, click Actions > Remote Copy > Continuous Access Journal Z > EXCTG Operation. The EXCTG Operation window displays. 2. Click to change the mode from View to Modify. 3. Right-click the desired EXCTG in the tree or in the list and select Add Journal to EXCTG. 4. To register journals for the supervisor DKC, select the check box(es) on the far left of the DKCs (DKC-1 to DKC-4). If you are not registering journals for the supervisor DKC, clear the check box(es) on the far left of the DKCs (DKC-1 to DKC-4). 5. Select one from the DKCs (DKC-1 to DKC-4). To register journals of the supervisor DKC, proceed to Step 9. 6. Enter the system serial number in the SN box. If the serial number is already displayed and unchangeable, you do not need to enter it. 50 Using extended consistency groups

not need to enter it. 8. Select the Controller ID. 9. Specify the LDKC number, CU number, and the LDEV number of the remote command device. If the LDKC number, CU number, and the LDEV number are already specified and unchangeable, you do not need to specify them. 10. On the lower half of the dialog box, select the journal or groups that you want to register from the JNL list on the left and click Add. The selected journals move to the JNL list on the right. If the JNL list on the right contains a journal that you do not want to register, select the journal and click Cancel. 11. Select a mirror ID from the Mirror ID list. If a mirror ID is already specified and unchangeable, you do not need to select it. 12. If needed, repeat Step 4 to Step 11. 13. Click Set to close the Add Journal dialog box. 14. In the Preview list, check your settings. To change a setting, right-click and select Modify. To delete a setting, right-click and select Delete. 15. When satisfied, click Apply. If an error occurs during the operation, right-click the error code in the right-most column and select Error Detail. An error message displays. Split, restore, release pairs in an EXCTG With Business Continuity Manager, you can split, restore, and release all pairs in an EXCTG collectively. Pair-split operation When you perform a pair-split operation in an EXCTG, you have the choice of specifying Flush or Purge for Suspend Mode. • With Flush, the pairs in the EXCTG are split with the consistency of the data update sequence maintained. • With Purge, consistency of the data update sequence is not maintained within the EXCTG, but within each journal. With Purge, the pair-split operation is completed faster than with Flush. NOTE: When you split all pairs in a specific journal in an EXCTG, you cannot use the REVERSE option of the Business Continuity Manager YKSUSPND command. Operations for a specific journal When you split, restore, or release all pairs in a specific journal in an EXCTG, the supervisor DKC executes arbitration processing on the other journals. For example, in Figure 15 (page 52), if you collectively split all pairs that belong to journal 1, the supervisor DKC executes arbitration processing on journals 2 and 3, and therefore consistency of data update sequence is maintained only within the primary storage systems 2 and 3 and the secondary storage systems 2 and 3. If you restore pairs in journal 1, the supervisor DKC executes arbitration processing on journals 1 and 3. Split, restore, release pairs in an EXCTG 51

Remove journals from an EXCTG You can release journals as follows: • Single, multiple, or all journals from an EXCTG or from all EXCTGs. • All journals in the storage system or LDKC. You can also forcibly remove journal that could not be removed normally. Prerequisite information • You must be logged in to the supervisor DKC to remove a journal. • You can remove journals from an EXCTG at any time, regardless of the status of the journals. • If you delete all data volumes from a journal in an EXCTG, the journal is automatically removed from the EXCTG. • Removal of journal(s) from an EXCTG does not stop remote copy operations. Update sequencing by the EXCTG is affected for the journals, though not for others in the EXCTG. To remove journals from an EXCTG 1. In Remote Web Console, click Actions > Remote Copy >Continuous Access Journal Z > EXCTG Operation. The EXCTG Operation window displays. 2. Click to change the mode from View to Modify. 3. Select the journal or groups that you want to remove in the following ways: • To remove one or more individual journals, right-click the journal(s) from the list. • To remove all journals from one EXCTG, right-click the EXCTG in the tree or the list. • To remove all journals from all EXCTGs, right-click Registered in the tree. 52 Using extended consistency groups

the list. • To remove all journals in an EXCTG belonging to an LDKC, right-click the LDKC from the tree. 4. Click Remove JNL from EXCTG in the menu that displays 5. In the Preview list, check the journal(s) to be removed. To cancel removal of a journal, right-click and select Cancel. 6. When satisfied, click Apply. If an error occurs during the operation, right-click the error code in the right-most column and select Error Detail. An error message displays. If you cannot remove journals, follow the instructions in the next section to remove the journals forcibly. Forcibly remove journals Follow the procedure below when you cannot remove the journals from an EXCTG due to a communication failure between storage systems or other reason. CAUTION: To remove journals from an extended consistency group, first log in to the supervisor storage system and then follow the instructions in “Remove journals from an EXCTG ” (page 52). Follow the procedure below only when you cannot remove journals due to a communication failure between storage systems or some other reason. To remove journals forcibly 1. Follow the instructions in previous section, “Remove journals from an EXCTG ” (page 52). 2. If the operation fails, navigate to the Journal Operations window from the Actions menu. 3. Locate the journal or groups that you want to forcibly remove from an EXCTG, right-click and select Journal and Remove JNL from EXCTG (Force). 4. In the Preview list, check the journal(s) to be forcibly removed. To cancel removal of a journal, right-click and select Cancel. 5. When satisfied, click Apply. If an error occurs during the operation, right-click the error code in the right-most column and select Error Detail. An error message displays. Using EXCTGs with P9500 Continuous Access Journal Z can perform remote copy operations from more than one primary storage system to more than one secondary system using EXCTGs. Both the primary and secondary systems can be a mix of P9500 and XP24000/XP20000 Disk Arrays. • There is no limitation on the combination of P9500 and XP24000/XP20000 Disk Array systems for either the primary or secondary systems. Any combination of system type is allowed. • There is no limitation on the journals that can be registered in the EXCTG when the supervisor DKC is a P9500 and the subordinate DKC is an XP24000/XP20000 Disk Array. All journals in P9500 and XP24000/XP20000 Disk Array can be specified as EXCTGs. This is shown in Figure 16 (page 54). • However, when the supervisor DKC is an XP24000/XP20000 Disk Array and the subordinate DKCs are P9500, journals of LDKCs other than LDKC00 cannot be registered in the EXCTGs. This will be true even if the number of LDKCs is expanded in a future version. This is shown in Figure 16 (page 54). To clarify, please observe the following: ◦ The journal of LDKC00 of P9500 (subordinate DKC) can be registered in the EXCTG. ◦ The journal of LDKC01 of P9500 (subordinate DKC) cannot be registered in the EXCTG. Using EXCTGs with P9500 53

Disk Array) are the subordinate DKC 54 Using extended consistency groups

(XP24000/XP20000 Disk Array) is the supervisor DKC Using EXCTGs with P9500 55

This chapter provides the configuration tasks that will make your primary and secondary systems ready for Continuous Access Journal Z pair operations. Configuration workflow You will set up primary and secondary systems for Continuous Access Journal Z operations as follows. Check the prerequisites for each procedure. Some tasks need to be performed from both local and remote storage systems and other tasks need to be performed from the local or remote storage system. 1. Log in to Remote Web Console. Make sure Remote Web Console is in Modify mode. Click to change the mode from View to Modify. 2. Define the Fibre Channel port attributes that will be used for Continuous Access Journal Z. See “Define Fibre Channel port attributes ” (page 56), below. 3. Set up logical paths between the primary and secondary systems. See “Configure storage systems for Continuous Access Journal Z, define logical paths ” (page 58). 4. Register journal volumes in a journal. See “Register journal volumes in a journal” (page 63). You may also perform these additional procedures prior to the initial copy: • Add additional logical paths. See “Configure additional logical paths ” (page 60). • Specify the number of volumes to copy. See “Specify number of volumes for initial copy, resync ” (page 61). Define Fibre Channel port attributes Initiator, and RCU target ports must be set up on the primary and secondary systems for Continuous Access Journal Z command and data transfer. Prerequisite information Before changing port attributes, you must perform the following prerequisites. • If initiator ports and RCU target ports are not configured on both primary and remote storage systems, the following problems occurs and remote copy operations are not performed: ◦ The administrator at the primary site will be unable to create any pair of a primary data volume and a secondary data volume. ◦ The secondary storage system will be unable to read data from journal volumes in the primary storage system. • Before a fiber channel target port can be changed to an initiator port, the following must be performed: ◦ Disconnected from the host. ◦ All affected pairs must be released. ◦ All logical paths from the initiator port to remote storage system must be deleted ◦ After deleted logical paths, all cables to the port must be removed. 56 Configuration operations

must be performed: ◦ All affected data volume pairs must be released. ◦ All logical paths must be deleted from the initiator port to the remote storage system. ◦ After logical paths are deleted, cables connecting the local system to the remote system must be removed. • The number of hosts connected to a target port must be limited to 128 or fewer to avoid illegal disconnection. See “Planning ports for data transfer” (page 44) for information on target, initiator, and RCU target ports and to determine the number of initiator and RCU target ports required by your system. To define port attributes 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > DKC Operation. 2. Click to change the mode from View to Modify. 3. In the Display box, select Port. The ports associated with the local storage system are listed. 4. Right-click the port that you want to change and select the desired port type, Initiator, RCU Target, or Target). 5. Select all the port attributes that you want to change, review the Preview list. WARNING! Read and follow the important warnings and caution above before applying the port attribute changes. • To change the attribute that you set for a port attribute, right-click the port in the upper right list and select the attribute. • To release a the modification that you made for a port attribute, right-click the port in the Preview list and select Delete. 6. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Define Fibre Channel port attributes 57

paths Prerequisite Information • This procedure must be performed on both primary and secondary systems. • The primary and secondary systems must already be configured for Continuous Access Journal Z operations. See “System requirements” (page 17) for more information. • The data path must be set up. See “Planning the data path” (page 40) for more information. • The port attributes on the primary and secondary systems must be configured for Continuous Access Journal Z. See “Define Fibre Channel port attributes ” (page 56). • The secondary system’s serial number, SSID, and the CU will be required during this procedure. To set up the Continuous Access Journal Z relationship and define logical path(s) 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > DKC Operation. The DKC Operation window displays. 2. Click to change the mode from View to Modify. 3. In the Display box, select DKC. The tree and full list on the right-side display information about all remote storage systems. 4. Locate the list on the right-side, right-click, and select DKC Operation > Add DKC. The Add DKC dialog box is displayed. 58 Configuration operations

6. In the LDKC box, select the remote system’s LDKC number. Enter 00 when the remote system is a P9500, XP24000/XP20000, or XP12000/XP10000 disk array. 7. In the Controller ID box, select the remote system’s controller ID. The controller ID for a P9500 is 6. The controller ID for an XP24000/XP20000 Disk Array is 5. The controller ID for an XP12000/XP10000 Disk Array is 4. 8. The Path Gr. ID check box is intended for a future release. Continue to the next step. 9. In the M-R Path box, define the logical path between local and remote systems: 1. In the Port column, select a local system initiator port. 2. In the Pair-Port column, select the remote system RCU target port. This establishes a logical path with the selected initiator port. You may enter port numbers using the keyboard. You may abbreviate the number into two characters. For example, “1A” may be entered instead of “CL1-A.” Letters may be uppercase or lowercase. 10. Click the Option. The DKC Option dialog box displays, as shown below. Configure storage systems for Continuous Access Journal Z, define logical paths 59

12. RIO MIH means “remote I/O missing interrupt handler.” In the RIO MIH Time box, enter the amount of time to wait before the data transfer operation is flagged as failed by the system. The range of time is from 10 seconds to 100 seconds (15 seconds is the default). When using a 3DC cascading configuration, it recommended that RIO MIH is set at 20 seconds. 13. Click Set and display the DKC Operation window. 14. See the Preview list to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 15. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Configure additional logical paths You can configure additional logical paths as needed. A maximum of eight logical paths are supported. Prerequisite information Please review the prerequisites for configuring logical paths in “Configure storage systems for Continuous Access Journal Z, define logical paths ” (page 58). To configure additional logical paths 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > DKC Operation. 2. Click to change the mode from View to Modify. 3. In the Display box, select DKC. The tree and full list on the right-side display information about remote storage systems. 4. In the tree, select the remote system below the associated LDKC. The logical paths display in the full right-side list. 5. Right-click a logical path and select Add Path from the menu. 60 Configuration operations

1. In the Port column, select a local system initiator port. 2. In the Pair-Port column, select the remote system RCU target port. This establishes a logical path with the selected initiator port. You may enter port numbers using the keyboard. You may abbreviate the number into two characters. For example, “1A” may be entered instead of “CL1-A”. Letters may be uppercase or lowercase. 7. Click Set and close the Add Path dialog box. 8. See the Preview list to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 9. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Specify number of volumes for initial copy, resync You can specify the number of volumes to be copied concurrently during an initial copy and resynchronization operations. Specify number of volumes for initial copy, resync 61

1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Optional Operation. 2. Click to change the mode from View to Modify. 3. Right-click the value that displays under Activities in the right-side pane, then select Change System Option from the menu. The System Option dialog box displays. 4. In the Maximum Initial Copy Activities box, enter the number of volumes that you want to be copied concurrently during initial copy and resynchronization operations, then click Set. The number can range from 1 to 128 volumes. The default is 64 volumes. • This setting may impact the performance of the primary storage system. This depends on the number and the amount of I/O activity. By using the default, 64 volumes, you limit the impact of initial copy operations on storage system performance. • When you specify more than 64, the system starts the first 64 pairs but does not start the 65th pair until one of the pairs is synchronized. 5. Click Set when finished. 6. See the Preview list in the Journal Operation window to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 7. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. 62 Configuration operations

You set up journal volumes by registering them in a journal. The procedure is provided below. Prerequisite information • A journal volume may be either a 3390 emulation volume (except for data exchange emulations and 3390-A) or it may be an OPEN-V emulation. The number of volumes registered does not have the same between the primary and secondary storage systems. However, the emulation of the journal volumes in the primary and secondary storage system be the same when the journals are paired. If the volumes in the primary storage system are 3390 emulation, then the journal volumes in the paired journal group in teh secondary storage system must also be 3390 emulation. • Journal volumes must be registered in a journal before the initial pair-copy operation is performed. • Prior to registering journals, set up primary and secondary EXCTGs, if necessary. See “Using extended consistency groups” (page 48). • Journal volumes must be registered on both local and remote systems. • A journal can contain up to 64 journal volumes. • Journal volumes must be set up and sized according to your Continuous Access Journal system’s requirements. For more information, see “Sizing journal volumes” (page 25). • Journal volumes can be added to a journal under the following conditions: The journal is not used. ◦ ◦ When pair operations are not in progress and the status of the journal of the mirror is Active, Stopped, Hold, Holding, or Hold(Failure). • Journal volumes cannot be added to a journal under the following conditions: When pair operations are in progress and the status of the journal of the mirror is Active, Stopped, Hold, Holding, or Hold(Failure). ◦ ◦ When Cache Residency settings are defined for the volumes. • Do not register a journal volume during quick formatting. Doing so stalls quick formatting operation. • Data volumes and journal volumes that belong to different LDKC cannot coexist in a journal. Register journal volumes in a journal 63

1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. The Journal Operation window is displayed. 2. Click to change the mode from View to Modify. 3. To register journal volumes in an existing journal, select the journal from the Registered tree. To register journal volumes in a new journal, select the journal from the Free tree. The selected journal displays in the data pane. Only mirrors of journals with either of the following can be registered: Status: Blank, Initial, Active, Stopped, Hold, Holding, or Hold(Failure). 4. Right-click the selected journal then select Edit Journal Volumes from the menu. The Edit Journal Volumes dialog box displays. 64 Configuration operations

volume in the list. If you cannot find the volumes, select one of the following in the PG/CU change area: • PG, then enter a parity group number in the box, and click Show. A list of volumes in the specified parity group displays. Select the volumes that you want to register. • PG(Ext.), then enter the external parity group number in the box, and click Show. A list of volumes in the specified external parity group displays. Select the volumes that you want to register. Though parity group numbers for external volumes start with the letter "E", you do not need to enter "E" in the text boxes. For example, to specify the parity group number "E1-2", you would enter 1 and 2 into the text boxes. • CU, then select a CU number from the list. A list of volumes in the specified CU displays. Select the volumes that you want to register. 6. In the Timer Type box, select the type of clock used for consistency time. This option can only be selected when no journal volume is registered. The selected option must be the same for primary and secondary systems. • System When the System timer option is selected, the primary storage system acquires the time stamp information for each journal data as follows. When a Continuous Access Journal Z pair is established, the primary storage system reports state change interrupt (SCI) to all hosts. The host then issues a series of sense group commands to determine the device status change, and the primary storage system returns the same response as if the device had been added to an XRC session to activate I/O time stamping for the device. Once I/O time stamping is activated, the MVS IOS routine attaches the time stamp information (contents of time of day (TOD) clock) to each write I/O operation for the device. The time stamp indicates the time that the update was generated during start sub Register journal volumes in a journal 65

storage system at the beginning of each I/O operation. • Local When the Local timer option is selected, the primary storage system does not acquire time stamp information from the host I/O time stamping function. • None This timer option can be selected only when the copy direction of a Continuous Access Journal Z volume pair is in reverse direction (i.e., from the secondary site to the primary site). When the None option is selected, the primary storage system acquires time stamp information from the host I/O time stamping function. 7. Click Add. The added display in the Journal Volumes list, and the Operation column of the list displays Add. 8. Click Set to close the Edit Journal Volumes dialog box. 9. See the Preview list in the Journal Operation window to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 10. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. 66 Configuration operations

This chapter provides procedures and prerequisite information for performing Continuous Access Journal Z copy operations. Pair operations workflow Basic Continuous Access Journal Z operations consists of the following. • Always check pair status. Each Continuous Access Journal Z operation requires the pair to be in a specific status. • Create the pair, in which the S-VOL becomes a duplicate of the P-VOL. • Split the pair, which separates the P-VOL and S-VOL and allows read/write access to the S-VOL if desired. • Restore the pair, in which the S-VOL again mirrors the ongoing, current data in the P-VOL. • Delete a pair, which removes the pair relationship from the volumes. For disaster recovery procedures, see “Disaster recovery operations ” (page 105). Check pair status Every Continuous Access Journal Z operation requires that the pairs have a specific status. Before performing any operation, check pair status. • Find status requirements under the Prerequisites sections for each operation. • To view a pair’s current status in the GUI, or review status definitions, see “Monitor pair activity, status” (page 80). Create the initial copy When you create the initial copy, all data in the P-VOL on the primary storage system is copied to the S-VOL on the secondary storage system. The P-VOL remains available to the host for I/O operations. Prerequisite information • The initial copy is performed from the primary storage system. • The initial copy can be performed when the volumes are in the status of Simplex. • S-VOLs must be offline to all hosts. • You will be required to select the volumes you will assign to a pair or pairs by the individual volume’s CU and LDEV numbers. • Connect the primary and secondary systems, and all logical paths must be defined. See “Configure storage systems for Continuous Access Journal Z, define logical paths ” (page 58) for more information. • Journal volumes must be registered to the journals that are associated with pairs. See “Register journal volumes in a journal” (page 63) for more information. • You will assign master and restore journals to the P-VOL and S-VOL during the operation. • If you are creating more than one pair at one time, you must specify the number of volumes that initial copy operations can be performed concurrently on. See “Specify number of volumes for initial copy, resync ” (page 61) for more information. Pair operations workflow 67

requirements are met. See “Procedure for creating a delta resync pair ” (page 139). • If this program product is running on a primary storage system or secondary storage system, stop Performance Monitor before performing the initial copy to avoid overloading with TCP/IP traffic. For details about Performance Monitor, see HP StorageWorks P9000 Performance for Open and Mainframe Systems User Guide. To create an initial copy or copies 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z> Pair Operation. 2. Click to change the mode from View to Modify. 3. In the tree, select a CU group number or CU number. The volumes available for pairing display in the right-side list area. 4. Right-click a volume that you want as a P-VOL and select Add Pair from the menu. • Volumes with the pair icon are already used as P-VOLs. • Volumes with the pair icon are already used as S-VOLs. • You can create more than one pair at one time. In this case, all the S-VOLs must be in the same secondary system 68 Pair operations

LDKC number, CU number, and LDEV number. • One P-VOL displays at a time, even if you selected multiple volumes for pairing. The volume with the lowest number displays first. 5. Specify an S-VOL as follows: 1. Select the CU number and LDEV number from the two S-VOL , left to right. If you selected multiple volumes as P-VOLs, the S-VOL you specify by the list is paired with the P-VOL displayed in P-VOL. Create the initial copy 69

S-VOLs. If you selected multiple volumes as P-VOLs, you can instruct the system to automatically assign S-VOLs for the remainder of P-VOLs. Do this from the Select Other S-VOL(s) list, as follows: - Select Increment to cause Continuous Access Journal Z to add the secondary system LDEV incrementally. Your S-VOL LDEV numbers must be numbered incrementally. For example, if you create 3 pairs and specify 11 on the LDEV of P-VOL, the LDEVs on the secondary system are incremented at 11, 12, and 13. - Select Input Next to specify S-VOL LDEV numbers one at a time for each P-VOL. For example, if after setting the S-VOL LDEV for the current P-VOL you click the Set, then you would reopen the dialog box to set the next S-VOL, and so on. - Select Relative to P-VOL to cause Continuous Access Journal Z to add S-VOL LDEV numbers closest in number to the P-VOL’s LDEV numbers. For example, if the LDEV numbers of the three P-VOLs are 001, 005, and 006; and you select LDEV numbers 02 for the first S-VOL, LDEV numbers 06 and 07 will be assigned by the system as the next two S-VOLs. If you select and right click more than one volume in the Preview list and then select Modify, the Paircreate dialog box opens and the Select Other S-VOL(s) list indicates No Change. If you want to change the setting with this list, select another option from the list. 6. In the Mirror box, proceed as follows. 1. Select the master journal from the M-JNL list. In the M-JNL list, the number of unused or existed master journal on the primary storage system is displayed. 2. Select a mirror ID (0 to 3) from the Mirror ID list. 3. Select a restore journal from the R-JNL list. The M-JNL list does not display journal numbers used by Continuous Access Journal Z. 7. In the DKC list, select the secondary system. In the DKC list, display the serial number, the LDKC number, the controller ID, and the model name about the secondary storage system. The secondary system must be the same for all pairs being created during one operation. 8. In the Initial Copy list, select one of the following: • Entire, to create the Continuous Access Journal Z pair relationship and copy the data from primary to secondary volumes immediately. • None, to create the Continuous Access Journal Z pair relationship but copy no data from primary to secondary volumes. You must be insure that data in the P-VOL and S-VOL are identical when selecting None. • Delta, to create the Continuous Access Journal Z pair for a delta resync. No initial copy operation starts after the volume pair is created. The status of the volume pair changes to Hold or Holding. 9. In the Priority list, specify the scheduling order for the initial copy operation. The default value is 32, and 1 to 256 can be specified. For performing more initial copy operations than the setting value of the Maximum Initial Copy Activities set on the System Option dialog box, the scheduling order (Priority) can be set for the initial copy operations to be performed. The Priority is determined within the range of the number of initial copy operations performed at the same time. Therefore, until the first initial copy operations complying with the order of the Priority are completed, the additionally specified initial copy operations are not started. If a time-out error occurs during the Add Pair operation, the copy operation may not be executed correctly in the order specified in the Priority field. A time-out error may be caused 70 Pair operations

the error, and then retry the Add Pair operation. 10. In the Error Level list, select one of the following options to take place in the event a failure occurs during this operation: • Mirror, to cause all pairs in the failed pair’s mirror to be split. Mirror is the default. • Volume, to cause only the pair that failed to be split. 11. In the CFW box, select one of the following to specify whether to copy cache-fast-write data to the S-VOL or not: • Only P-VOL: does not copy cache-fast-write data to the S-VOL (the default). • Copy to S-VOL: copies cache-fast-write data to the S-VOL. 12. Click Set to close the Paircreate dialog box. 13. See the Preview list to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 14. Click Apply to apply the settings and create pairs. If an error occurs during creating pairs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. 15. In the Pair Operation window, verify that pair status is correct based on the initial copy option you selected in Step 7. Current Pair Status Should Be Initial Copy Option Pending or Duplex Entire or None HOLD, Holding Delta 16. You can monitor the progress of the paircreate operation by clicking File > Refresh in the menu bar. This updates the information in the list. You can also review current pair activity and pair status on the Detailed Information dialog box. See “Monitor pair activity, status” (page 80) for more information. Split a pair When you split a pair, write-data is no longer sent to S-VOL and the pair is no longer synchronized. Splitting a pair or mirror gives you a point-in-time copy. For details about a point-in-time copy, see “Create a point-in-time copy ” (page 73). Prerequisite information • A pair can be split from the primary or secondary system. • Pair status must be Duplex to perform the split operation. • If you want to split multiple pairs at the same time, the pairs must belong to the same mirror. Doing this insures sequence consistency among S-VOLs in the mirror. • Best practice is to perform pair split when I/O load is low. This reduces impact on performance. Operations on multiple pairs in the same consistency group with different statuses may result in suspension during high periods of write I/O. Split a pair 71

• During normal Continuous Access Journal Z operations, the secondary system rejects write operations to an S-VOL. However, you can enable write operations to the S-VOL during the pair split operation. When you do this, then resync the pair, the secondary system sends the S-VOL track bitmap to the primary system to insure proper resynchronization of the pair. This S-VOL Write option is described in the pair-split procedure. To split one or more pairs 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Pair Operation. 2. Click to change the mode from View to Modify. 3. In the tree, select a CU group number or CU number. Paired volumes are displayed in the upper-right-side list area. 4. Select the pair that you want to split, right-click, and select Suspend Pair from the menu. The dialog box displays. 5. From the S-VOL Write list, select one of the following: • Enable to permit hosts to write data to the S-VOL. • Disable (the default) to prevent hosts from writing to the S-VOL 6. From the Range list, select one of the following: • Volume to split only the selected pair or pairs. • Mirror to split all pairs in the same mirror(s) as the selected pairs If the following two conditions are satisfied and you select Apply, a warning message appears and processing cannot continue. - The Preview list contains two or more pairs belonging to the same mirror. - The Range column shows Mirror for at least one of the above pairs. To be able to continue processing, do either of the following: - Ensure that the Range column shows Volume for all pairs in the same mirror. - In the Preview list, select one pair in one mirror, and then other pairs in the mirror is excluded. 7. From the Suspend Mode list, select one of the following: • Flush to cause all uncopied update data to be copied to the S-VOL before the split operation is completed. • Purge to cause all uncopied update data not to be copied to the S-VOL before the split operation is completed. When you resynchronize, uncopied update data is then copied to the S-VOL(s). 8. Click Set when finished. 72 Pair operations

• If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 10. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. You can verify the Suspend status when the operation is completed. Split a mirror If a mirror is split, all the pairs in the mirror are split, and the data copy from the primary journals to the secondary journals is suspended. Prerequisite information • A mirror can be split from the primary or secondary system. • Mirrors must be in Active status. To split a mirror 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. 2. Click to change the mode from View to Modify. 3. In the tree, expand Registered then select the desired LDKC. Journals display in the tree and mirrors display in the right-side pane. 4. Select one or more mirror with the desired mirror ID(s). The status of the mirror must be Active. 5. Right-click and select Mirror >Suspend Pair from the menu. The Suspend Pair dialog box displays. 6. Select the desired options and complete the procedure (see Step 4. to Step 9. on “Split a pair ” (page 71). However, you cannot change Mirror of the Range list in the step 5.) 7. Click Apply to split the mirror. If an error occurs during creating pairs, right-click the item in the Preview list and select Error Detail. You can verify the Stopped status of the mirror when the mirror split operation is completed. Create a point-in-time copy Continuous Access Journal Z allows you to make Point-in-Time (PiT) duplicates of groups of volumes. The term "Point-in-Time (PiT)" refers to an act of obtaining a copy of data immediately when you want to obtain it. Continuous Access Journal Z can be used to create a PiT copy of data volumes belonging to a journal. To produce a PiT duplicate of the volumes in a journal 1. Stop all write I/Os from hosts to P-VOLs in the journal. 2. After all write I/Os to P-VOLs have stopped, split the mirror. Use the Suspend Pair dialog box and then specify Mirror for Range and specify Flush for Suspend Mode. 3. When the status of all pairs in the journal has changed to Suspend, the duplicate set of volumes is complete. If desired, you can restart your business application at the primary site to resume business activities. Split a mirror 73

If the pair is restored, the suspended data copy from the primary data volumes to the secondary data volumes is resumed. If the delta resync pair is restored, the delta resync processing is performed. This section describes the pair resync operation. Also, the instructions are used when performing the delta resync operation. Prerequisite information • The pair resynchronizing operation is performed from the primary storage system only. • Pairs in Suspend status are resynchronized. • Best practice is to resume the pair when I/O load is low. This reduces impact on performance. Operations on multiple pairs in the same consistency mirror with different statuses may result in suspension during high periods of write I/O. Prerequisites for delta resync operation The delta resync operation is executed by selecting delta-resync only options during the resync operation. If executing delta resync, please observe the following: • Status of the delta resync pair must be Hold or Holding. If the pair status is in Hlde status, you can change the status to Hold by performing the pairresync operation. • Status of the Continuous Access Synchronous Z secondary volume (S-VOL) must be Suspend. • The Continuous Access Journal Z S-VOL uses two mirror IDs; one for the pair for 3DC multitarget pair, one for the pair for the delta resync. ◦ The status of the Continuous Access Journal Z pair S-VOL must be Duplex or Suspended. ◦ The status of the delta resync Continuous Access Journal Z pair S-VOL must be Hold. • The data path between primary volume (P-VOL) and S-VOL must be available. • If data in the restore journal exceeds 80% of capacity, old journal data is automatically deleted. This can occur when the Continuous Access Journal Z pair is suspended and has not been resynchronized for a long time. If the journal data that is not restored to the secondary volume of Continuous Access Journal Z exceeds 80% of the journal volume capacity, the S-VOL cannot be restored only copying the journal data to the restore journal volume in the Continuous Access Journal Z secondary site. In this case, based upon journal option settings, either the entire P-VOL is copied to the S-VOL or delta resync operation finishes without any processing. • Even if the capacity of the journal data does not exceed 80% of the journal volume, the journal data may be destroyed in the following cases: ◦ When you restore the Continuous Access Synchronous Z pair, then updated the P-VOL. ◦ When you restore the Continuous Access Journal Z pair between the primary site and the Continuous Access Journal Z secondary site, then updated the P-VOL. ◦ When the retry processing occurs because of a delay of the P-VOL update. ◦ When the update of the Continuous Access Synchronous Z S-VOL is delayed. • The status of the delta resync Continuous Access Journal Z pair might become Suspend by splitting the pair or by failure while copying all data from P-VOL to S-VOL when the delta resync Continuous Access Journal Z pair performs the operation as a Continuous Access Journal Z pair before performing the delta resync operation. In this case, all data might be copied from P-VOL to S-VOL regardless of the journal option settings when the delta resync operation is performed. 74 Pair operations

journal after a power supply error occurred at the primary storage system, first resynchronize the pair and then execute I/O operation from the host. • After the delta resync operation (when pair status has changed to Duplex), the delta resync P-VOL must be updated from the host for longer than five minutes. This is done to insure internal communications between the sites. However, if remote command devices are used, the update operation needs not be performed for five minutes or more. See “Prerequisite information for performing delta resync operation” (page 138) for details. To restore a pair 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Pair Operation. 2. Click to change the mode from View to Modify. 3. In the tree, select the CU group number or CU number for the pair. Associated pair volumes display in the right-side list area. NOTE: The primary system does not resynchronize a pair or pairs that are suspended due to an error until the cause of error is removed. 4. In the list, select a pair to resume, perform the delta resync operation, or change the status from Hlde to Hold, right-click and select Resume Pair from the menu. The Resume Pair dialog box displays. 5. From the Range list, select one of the following: • Volume, to resync only the selected pair or pairs. NOTE: If the journal to which the specified pair belongs is in the Halting or Stopping status, the pairresync command is rejected. • Mirror, to resync all pairs in the same mirror(s) as the selected pairs. Specify the restore range. The default is Volume if two or more pairs in the same mirror are selected. The default is Mirror if not. Note that the default setting is Mirror when the pair status is Hold, Holding or Hlde, and in that case you cannot change the default setting. If the following two conditions are satisfied and you click Apply, a warning message appears and processing cannot continue: • The Preview list contains two or more pairs belonging to the same mirror. • The Range column displays Mirror for at least one of the above pairs. Restore a pair 75

• Ensure that the Range column displays Volume for all pairs in the same mirror. • In the Preview list, select all but one pair in the same mirror, right click the selected pairs, and then select Delete. 6. From the Priority list, specify the scheduling order for resynchronizing selected pairs. If Range is Mirror, the Priority list is disabled. 7. The DKC box displays the following information: • The serial number with the LDKC number. • The controller ID with the model name. • The path type. 8. The Resync Mode box indicates the type of pair resync that will take place when the operation is executed. • Normal. The split pair whose status is Suspend is recovered (resynchronized). • Delta. The delta resync operation is performed. • Return to standby. The pair status changes from Hlde to Hold. 9. In the Error Level list, select one of the following options to take place if a failure occurs during resync. Note that if the Range option is set to Mirror, Error Level options are disabled. • Mirror, to cause all pairs in the failed pair’s mirror to be split. • Volume, to cause only the specified pair or pairs that failed to be split. 10. Click Set. The Pair Operation window is redisplayed. 11. See the Preview list in the Journal Operation window to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 12. When satisfied, click Apply. This starts the operation. Verify the status when the operation is completed. To monitor the progress of the operation, see “Monitor pair activity, status” (page 80). If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. NOTE: If the status of the Continuous Access Journal Z pair for delta resync operation does not change after you perform the delta resync operation, some of the requirements may not have been fulfilled. Check pair status of the Continuous Access Synchronous Z pair, Continuous Access Journal Z pair, and Continuous Access Journal Z pair for delta resync to make sure they are in the required status for delta resync. Resynchronize a mirror When you resynchronize a mirror, the data copy from the master journal to the restore journal restarts. Prerequisite information • This operation is performed from the primary system only. • Best practice is to resume the mirror when I/O load is low. This reduces impact on performance. Operations on multiple pairs in the same consistency group with different statuses may result in suspension during high periods of write I/O. 76 Pair operations

1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. 2. Click to change the mode from View to Modify. 3. In the tree, expand Registered then select the desired LDKC. Journals display in the tree, Mirrors display in the right-side pane. 4. In the list, select a pair to resume, perform the delta resync operation, or change the status from Hold(Failure) to Hold, right-click and select Mirror > Resume Pair from the menu. The Pairsplit-r dialog box displays. • If Stopped is selected, all pairs in the mirror is resynchronized and then the data copy restarts. • If Hold or Holding is selected, the delta resync operation is performed on all pairs in the mirror. • If Hold(Failure) is selected, the status of the mirror is changed to Hold. 5. See the Preview list to check the mirrors that you want to restore. • If you want to cancel restoring a mirror, select and right click the mirror and then click Cancel. • If necessary, you can repeat steps 2 to 5 to specify other mirrors. 6. Click Apply to restore the mirror(s). If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Delete a pair When you delete a pair, the Continuous Access Journal Z relationship between the P-VOL and S-VOL is released. The data volumes and their data remain, but in Simplexstatus. Prerequisite information • This operation can be performed from either the primary or secondary systems. • When a pair deletion is initiated, transfer of differential data from P-VOL to S-VOL is completed, then the pair relationship is deleted and the volume’s status becomes Simplex. • A pair can be deleted regardless of its status. However, data consistency is not guaranteed unless status is Duplex. • If the operation fails, the P-VOL nevertheless becomes Simplex. Transfer of differential data to S-VOL is terminated. • Best practice is to perform pair deletion when I>O load is low. This reduces impact on performance. Operations on multiple pairs with different statuses and in the same mirror may result in suspension during high periods of write I/O. • In a delta resync configuration, if you release the Continuous Access Synchronous Z pair, the Continuous Access Journal Z delta resync pair will be released as well. If you release the Continuous Access Journal Z pair, the Continuous Access Journal Z delta resync S-VOL is released. • If Continuous Access Journal Z secondary data volumes are physically attached to the same host server(s) as the Continuous Access Journal Z primary data volumes, the following problem can occur: When a Continuous Access Journal Z pair is released, the older, secondary data volume is usually offline. If the host is then restarted, the system administrator may be offered both Delete a pair 77

to error. If the Continuous Access Journal Z secondary data volumes and primary data volumes are connected to the same host(s), HP strongly recommend that the secondary data volumes are defined to remain offline to avoid this problem. To delete a pair or pairs 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Pair Operation. 2. Click to change the mode from View to Modify. 3. In the tree, a CU group number or CU image number for the pair. Associated pairs display in the right-side list area. 4. In the list, select the pair or pairs that you want to delete, right-click, and select Delete Pair from the menu. The Delete Pair dialog box displays. 5. From the Range list, select one of the following. Note that the Range list is disabled if a pair’s status is Simplex; In this case, Range is automatically set to Volume. If a pair’s status is Deleting or Suspending, Range is automatically set to Mirror. • Volume, to resync only the selected pair or pairs. If deleting multiple selected pairs, they must be in the same mirror. Review the Force option for the Delete Mode field when specifying Range as Volume. • Mirror, (default) to resync all pairs in the same mirror(s) as the selected pairs. CAUTION: Do not use the Mirror option when deleting pairs from the secondary system during disaster recovery. 6. From the Delete Mode list, select one of the following: • Normal, to delete the pair or pairs only if the primary system is able to change the pair status of both P-VOL and S-VOL to Simplex. • Force, to forcibly delete the pair or pairs, even when the primary system is unable to communicate with the secondary system. This option could be used, for example, when the primary system is unable to communicate with the secondary system. Specifying Force would thus allow host operations to continue. If Force is specified when pair status is other than Simplex, Range is automatically set to Mirror. If Force is specified when Range is set to Volume, the other pairs in the same journal are also suspended. 7. Click Set. 78 Pair operations

• If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 9. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Delete pair volumes from a mirror When you delete the pairs in a mirror, data copying between master and restore journals end. Prerequisite information • This operation can be performed from either the primary or secondary systems. • When one journal uses two mirror IDs, the result of the operation differs according to the status of the specified mirror. ◦ When you specify a mirror in Hold, Holding, or Hold(Failure) status, only the Continuous Access Journal Z pairs of the specified mirror are deleted. ◦ When you specify a mirror that is not in Hold, Holding, or Hold(Failure) status, Continuous Access Journal Z pairs of both mirrors (including the mirror that you did not specify) are deleted. To delete pairs from a mirror 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z> Journal Operation. 2. Click to change the mode from View to Modify. 3. Operate one of the following: • In the tree, select a master journal or a restore journal from below Registered. • In the tree, select Journals or Registered and then select a master journal or a restore journal from the list to the right. A list of mirrors appears. One row in the list represents one mirror. 4. Select the desired mirror or mirrors. 5. Right-click and select Mirror >Delete Pair from the menu. 6. See the Preview list in the Journal Operation window to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 7. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Delete pair volumes from a mirror 79

Monitoring pairs, journals, and data paths should be checked frequently to insure that Continuous Access Journal Z pairs are operating correctly. Pair status should be checked before performing a Continuous Access Journal Z operation. Each operation requires specific pair status. Monitor pair activity, status Monitoring the Continuous Access Journal Z system is an ongoing operation that should be performed frequently to maintain your pairs. • When you want to perform a pair command, first check the pair’s status. Each operation requires a specific status or statuses. • Pair-status changes when an operation is performed. Check status to see that pairs are operating correctly and that data is updated from P-VOLs to S-VOLs in the Paired status, or that differential data management is performed in the Split status. Monitoring using the Remote Web console GUI is done at the user’s discretion. Monitoring should be repeated frequently. Email notifications of problems can be set up using the GUI. To monitor pair status 1. In Remote Web Console, select the storage system where the pair or pairs are located. Click Actions > Remote Copy > Continuous Access Journal Z > Pair Operation. The Pair Operation window displays. 2. Click to change the mode from View to Modify. 3. Locate the pair whose status you want to review. Review pair status in the Status column. 4. Select and right click a pair and then select Pair Status from the menu. You cannot select more than one pair. 5. In the Detailed Information dialog box, view detailed information about the pair. Pair status definitions Status descriptions are provided as follows. Description Pair Status The volume is not assigned to a Cnt Ac-J Z pair. The volume does not belong in a journal. When the volume is added to a Cnt Ac-J Z pair, the status changes to Pending. Simplex The initial copy operation is in progress. Pending The pair is synchronized. Updates to the P-VOL are duplicated in the S-VOL. Duplex This pair is not synchronized. • When the primary system detects a suspension condition, the system changes P-VOL status and S-VOL status (if possible) to suspended. • When the secondary system detects a suspension condition, the system changes S-VOL status to suspended. • When you suspend a pair from the primary system, the system changes P-VOL and S-VOL status (if possible) to suspended. • When you suspend a pair from the secondary system, the system changes the S-VOL status to suspended. When the primary system detects that the pair is suspended or released from the secondary system, the primary system changes the status of the P-VOL to suspended. Suspend (see “Suspend types” (page 83) for more information) 80 Monitoring the system

to suspended. When the suspension is requested (by user, primary storage system, or secondary storage system), the status of all affected pairs changes to suspending. When the suspension is complete, the status changes to suspended. Suspending This pair is not synchronized. This pair is in transition from duplex, pending duplex, or suspended to simplex. When the delete pair operation is requested (by user), the status of all affected pairs changes to deleting (releasing). When the delete pair operation is complete, the status changes to simplex. Deleting (releasing) • The pair is ready for the delta resync operation. When the status of the Cnt Ac-J Z P-VOL is Hold, incoming write data for the Cnt Ac-S Z S-VOL is stored in the master journal volume. • Operations allowed on the pairs in Hold status: - Delta resync - Pair-delete - Changing pair options • If you are using remote command devices for delta resync, this status indicates that the conditions to successfully execute the delta resync operation are satisfied. Hold • The pair status is changing to Hold because a pair for delta is being created. • Operations allowed in Holding status: delta resync, pair release, and changing pair option. • This status is displayed when the environment of the delta resync operation to support remote command devices is used. • If you execute the delta resync operation on a pair in Holding status, the operation may fail. If you set the Delta resync Failure option to Entire to in the Change Journal Option dialog box, all deferential data will be copied to a S-VOL. Holding Monitor pair activity, status 81

• When the status of P-VOL is Hlde, incoming write data for the Cnt Ac-S Z S-VOL is not stored in the master journal volume. • Operations allowed on pairs in Hlde status are: - Recovering pair status to standby (Hold) - Pair-delete - Changing pair options Hlde *Holding indicates that differential data to be used for a delta resync operation does not exist in the storage system, or the storage system cannot judge whether the delta resync operation can be performed without the differential data. When the differential data to be used for a delta resync operation does not exist in the secondary site of Cnt Ac-S Z even if you update the data from the host on the primary site, the differential data might have been discarded due to the following: • The cache or shared memory was added to or reduced from the storage system during the maintenance work. • The storage system was powered off during the maintenance work. • The Cnt Ac-J Z pair or Cnt Ac-S Z pair was suspended, and then the pair has been resynchronized. • The journal volume could not be accessed because the disaster or failure occurred in the secondary site of Cnt Ac-S Z. When a problem is fixed, differential data are stored in the Cnt Ac-S Z secondary site again if you update the data in the primary site. The status in which the delta resync operation can be executed without the differential data for the delta resync means that the data update to the primary site is not executed or the both data of the Cnt Ac-S Z secondary site and the Cnt Ac-J Z secondary site are coincident because the data update is stopped. To make the above mentioned condition, make the configuration to execute the delta resync operation to support the remote command devices, and then change the status of all Cnt Ac-J Z pair and Cnt Ac-S Z pair which belong to the desired journal in the primary site to PAIR by the resynchronizing operation. Even if the "delta resync operation" without the deferential data is executable, the pair status will change to Hold regardless of the differential data existing if the conditions of the differential data discarding are satisfied as mentioned above. To change the status of the pair to Hold, update the data in the primary disk system after the recovery from the condition of the differential data discarding. If you cannot connect to the secondary site of Continuous Access Synchronous due to failures or disasters, the transition from Holding to Hold is not performed correctly. Additional information on pair status • When a pair is split or suspended, the primary system notifies the host(s) with a service information message (SIM). If SNMP is installed and operational for P9500, this SIM results in an SNMP trap indicating the reason for suspension. • Transitional statuses occur when a request to change Continuous Access Journal Z pair status to Suspended or Simplex has been accepted, but is not complete. Transition states are not reported to the host. • The user and the primary or secondary systems can initiate the Suspended status change. • Only the user can perform release a pair. • If a user performs an operation, the final status is reported at the end of the operation. • If an error causes the status to change to Suspended, the status is reported at the beginning of the transition. • A pair in Flush mode (remaining local update data is flushing to the secondary system) remains in Suspending or Deleting status until data in the master and restore journals is the same and the pair is completely split or released. To calculate the time during which the pair remains Suspending or Deleting, use the following equation: C x (u / 100) x 1,024 / V (The unit is seconds) 82 Monitoring the system

◦ u is the usage rate of data (%) in the master journal volume. ◦ V is data transfer speed (MB/sec) between the primary and the secondary systems. See “Usage Monitor window ” (page 184) for instructions on viewing the usage rate of a journal volume. The Continuous Access Journal Z SEQCHK status is indicated when a Continuous Access Journal Z pair assigned to a mirror with the System timer type accepts a non time stamped update from the primary system. The SEQCHK status does not affect Continuous Access Journal Z copy activities and is removed when the next time stamped update is successfully copied to the S-VOL. However, if a disaster or system failure occurs before the next time stamped update, the update sequence consistency between the S-VOL and other S-VOLs in the mirror is not ensured. To ensure effective disaster recovery, you should detect and remove the source of the SEQCHK status. The SEQCHK status can be caused by any of the following. ◦ An application may issue update I/Os bypassing the MVS standard I/O procedure. ◦ The I/O time stamping function may not be active at the primary site. • When the status of the Cnt Ac-J Z pair status is Pending, neither cache nor shared memory can be added to or removed from the storage system. Before adding or removing the cache or shared memory, check the pair status and split pairs whose status is Pending; then, resynchronize the pair when the memory operation completes. Suspend types You can suspend a pair after the initial copy operation is complete. When you suspend a pair, the primary and secondary systems insure synchronization by completing or discarding any pending update copy operations. This depends on the Suspend Mode option that you specify in the Suspend Pair operation. A pair can also be suspended by the primary or secondary storage system. System suspensions occur for the following conditions: • When the primary system detects that the user has released the volume pair from the secondary storage system • When the primary system detects an error condition related to the secondary system, S-VOL, or a journal data operation • When the secondary system cannot execute DFW (DASD fast write (DFW)) to the S-VOL (if DFW is specified) • When the primary system is unable to communicate with the secondary system When a pair is suspended, the primary system stops updating the S-VOL. The primary and secondary systems keep track of journal data discarded during suspension. Also, the primary system continues accepting write I/Os for the P-VOL and keeps track of the P-VOL tracks which are updated while the pair is suspended. In addition to the Suspended status, an S-VOL also has a consistency status, which indicates its update sequential consistency with respect to the other S-VOLs in the same group. The consistency status is available only on the secondary system. See Table 13 (page 85) for descriptions of suspended S-VOL consistency statuses. When a pair is suspended, the primary system generates sense information to notify the host(s). Table 12 (page 84) provides suspension types and descriptions. Monitor pair activity, status 83

Description Applies to Suspend type The user suspended the pair from the primary or secondary system using the S-VOL option. P-VOL, S-VOL Secondary data volume by operator The secondary system received a request from the primary system to suspend the pair. The P-VOL suspend type is Primary data volume by Operator or Secondary data volume by Operator. S-VOL By MCU The primary system detected an error condition at the secondary system which caused the primary system to suspend the pair. The S-VOL suspend type is By MCU. P-VOL By RCU The primary system detected that the S-VOL status changed to simplex because the user released the pair from the secondary system. The pair cannot be resumed because the S-VOL does not have the suspended status. P-VOL Delete Pair to RCU The primary system detected an error during communication with the secondary system or an I/O error during update copy. In this case, the S-VOL suspend type is usually By MCU. P-VOL Secondary Data Volume Failure The primary system could not find valid control information in its nonvolatile memory during the IMPL procedure. This condition occurs only if the primary system is completely without power for more than 48 hours (e.g., power failure and fully discharged backup batteries). P-VOL, S-VOL MCU IMPL The pair was suspended before the initial copy operation was complete. The data on the S-VOL is not identical to the data on the P-VOL. P-VOL, S-VOL Initial Copy failed The pair was suspended because it was highly likely that journal data will overflow. P-VOL, S-VOL JNL Cache Overflow Table 13 (page 85) provides consistency statuses for suspended S-VOLs. These statuses indicate the S-VOL’s sequential update consistency with respect to the other S-VOLs in the same group. 84 Monitoring the system

Description Consistency status This Cnt Ac-J Z volume pair was suspended alone. Update sequence consistency between this S-VOL and other S-VOLs in this mirror is not insured. This S-VOL cannot be used for disaster recovery at the secondary system. This status is indicated when: This volume pair was suspended by a user-initiated suspend pair operation with the Cnt Ac-J Z Suspend option set to Volume. This volume pair was suspended due to a failure that did not affect the entire mirror, and the Error Level pair option for this pair is set to Volume. Volume This Cnt Ac-J Z volume pair was suspended along with the other pairs in its mirror. Update sequence consistency between this S-VOL and other S-VOLs in this mirror is insured. This S-VOL can be used for disaster recovery at the secondary system (after releasing the Cnt Ac-J Z volume pair from the secondary storage system). This status is indicated when: This volume pair was suspended by a user-initiated suspend pair operation with the Cnt Ac-J Z Suspend option set to Mirror. All volume pairs in this mirror were suspended due to a failure that affected the entire mirror (not just one pair) (e.g., primary/secondary storage system communication failure). The volume pair was suspended due to a failure that did not affect the entire mirror. Mirror Suspension conditions from the secondary system The following table provides conditions reported by the secondary system for suspending Continuous Access Journal Z pairs. Table 14 Suspension condition Pairs to be suspended Detected by Suspension condition All S-VOLs in the journals, or the affected S-VOL. RCU The secondary system could not copy the journal data successfully due to a hardware failure or logic error. All S-VOLs in the journal, or the affected S-VOL, depending on the type of failure. RCU The secondary system detected a logical error while selecting the journal data to be restored. RCU The secondary system could not restore the journal data due to a hardware failure, track condition, or logical error. See “Troubleshooting suspended pairs ” (page 119) for more information. The primary system stores the differential bitmap for each P-VOL in the system’s shared memory, and the secondary system stores the differential bitmap for each S-VOL in its shared memory. The tracks containing the following journal data are marked in the differential bitmap as modified and will be copied during the resume pair operation: • Journal data created by the primary system but not yet sent to the secondary system. After marking these P-VOL tracks as modified, the primary system discards these journal data. • Journal data sent to the secondary system but not acknowledged by the secondary system. After marking these P-VOL tracks as modified, the primary system discards these journal data. This insures that journal data lost during transmission are identified and marked. Monitor pair activity, status 85

these S-VOL tracks as modified, the secondary system discards these journal data. • The P-VOL records updated by host-requested write I/Os after the pair was suspended. When a suspended pair is resumed (resynchronized), the contents of the secondary system’s cylinder/track bitmap are sent to the primary system and merged into its bitmap. The primary system then performs the resync operation according to the merged bitmap. This insures that all the tracks, including discarded journal data, are resynchronized at this time. Filtering Information in the List in the Pair Operation window Use the Display Filter dialog box to filter information in the list in the Pair Operation window. The list shows only the volumes that satisfy certain conditions. To filter information in the list in the Pair Operation window 1. In the Pair Operation window, click Display Filter. 2. Use the Display Filter dialog box to specify information that you want to display in the list. 3. Click Set to close the Display Filter dialog box. The list in the Pair Operation window displays the specified information only. To restore the list to the default status (i.e., the status before filtering), click Reset in the Display Filter dialog box and then Set in the same dialog box. If you change settings in the Display Filter dialog box, the changes remain in effect during the session until you reset the settings. The Display Filter dialog box shows the following: • Journal: Select the journal to be displayed. If you select All, all journals appear in the list. • Mirror: Select the mirror to be displayed. If you select All, all mirrors appear in the list. • P-VOL/S-VOL: Select the type of volumes to be displayed. If you select P-VOL, P-VOLs appear in the list. If you select S-VOL, S-VOLs appear in the list. If you select All, all P-VOLs and all S-VOLs appear in the list. • CLPR: Select the CLPR to be displayed. If you select All, all CLPRs appear in the list. • Internal/External VOL: Specify whether internal volumes or external volumes appear in the list. If you select All, all the internal and external volumes appear in the list. • Status: Specify pair statuses. The volumes of the specified statues appear in the list. • Sub Status: Allows you to specify consistency status. The volume pairs in the specified status appear in the list. • SEQCHK Only: Specify whether to display only the volume pairs in SEQCHK status. If this check box is selected, volumes in Simplex status do not appear in the list. • Reset: Restores all options in the Display Filter dialog box to the default state. All lists show All and all check boxes are selected. 86 Monitoring the system

• Cancel: Discards the settings and closes the dialog box. Saving pair status information into a text file You can use the export function only while the client file access is enabled. When the client file access is not enabled, the Export is not available on the Pair Operation window. For the further information of the client file access settings, see HP StorageWorks P9000 Remote Web Console User Guide. The following information for pair status is displayed in the exported file: Descriptions Item Volume in the local storage system. The LDKC number, the CU number, and the LDEV number Vol Status of the pair Status Sub status (i.e., consistency status) of the pair Sub Journal number in the local storage system JNL Mirror ID Mr Serial number of the remote storage system S/N(LDKC) Controller ID and model name of the remote storage system CNTL Volume in the remote storage system. The number to the left of the colon (:) is a CU number. The number to the right of the colon is an LDEV number. Vol Journal number in the remote storage system. JNL Type of volumes in the local storage system P-VOL: primary data volume S-VOL: secondary data volume Type Error level ErrLv Progress of copy operation. Or, synchronization rate (i.e., concordance rate) between the data volume before the pair was split and the data volume after the pair was split. Sync CLPR number and CLPR name CLPR Copy time. Copy Time An LDEV number ending with # indicates the volume is an external volume. For detailed information about external volumes, see HP StorageWorks P9000 External Storage for Open and Mainframe Systems User Guide. "00" appears as the LDKC number when the volume is a volume of the XP12000/XP10000 Disk Array and the XP24000/XP20000 Disk Array. To save information about pairs in a text file 1. If necessary, filter information in the list in the Pair Operation window. 2. In the Pair Operation window, click Export. A message appears and asks whether to save volume pair information in a file. 3. Click OK to close the message. A dialog box appears and prompts you to save a file. Monitor pair activity, status 87

A message appears. 5. Click OK to close the message. If Subsystem or an LDKC is selected in the tree, you can abort the exporting by clicking Cancel in the dialog box that is open during the export processing. Monitor copy operations data, I/O You can monitor and use data for remote copy operations data and I/O statistics for monitored volumes on the storage system in the Usage Monitor window. This section explains how to do the following: • “Select data to be graphed” (page 89) • “Manipulate graph, save data” (page 91) In Remote Web Console, click Actions > Remote Copy > Continuous Access Journal Z > Usage Monitor. The Usage Monitor window displays as shown in Figure 18 (page 88). The fields on the Usage Monitoring window are described in Table 15 (page 88). You start and stop monitoring in Performance Monitor. See HP StorageWorks P9000 Performance for Open and Mainframe Systems User Guide for instructions. Figure 18 Usage Monitor Window Table 15 Fields on the Usage Monitor window Description Field • Enable: Monitoring is on. Graph displays. • Disable: Monitoring is off. Graph is disabled. Monitoring Switch The data collection interval. Gathering Interval The most recent data sample time of data on the graph. Update Remote I/O statistics and status of remote copy monitor. Usage Monitor Graph 88 Monitoring the system

The usage monitor graph plots the I/O data that you specify. On the graph: • The x-axis indicates time. • The y-axis indicates the number of I/Os during the sampling period. • The legend on the right side shows the data being displayed. The value on the y-axis varies according to the maximum value of the statistical data appearing in the graph. If the y-axis value exceeds 10,000,000, the value is shown in exponential notation (for example, 1E7 = 1 x 107 = 10,000,000; 2E8 = 2 x 108 = 200,000,000)). To specify I/O data to be graphed 1. Make sure that usage monitoring is running (Monitoring Switch = Enable). The usage monitor graph can be viewed only when monitoring is on. 2. Right-click the graph and select Display Item from the menu that appears. The Display Item dialog box displays. 3. In the Select Volume box, select one of the following: • ALL Volumes, to view I/O statistics for all LDEVs in the system. When selected, the LDKC number, CU number, and LDEV number appear above the graph. - A device ID ending in # (e.g., 00:00:3C #) indicates the LDEV is an external volume (see HP StorageWorks P9000 External Storage for Open and Mainframe Systems User Guide for more information about external drives). - A device ID ending in X (e.g., 00:00:3C X) indicates the LDEV is a Thin Provisioning virtual volume (see HP StorageWorks P9000 Provisioning for Open Systems User Guide for more information on a virtual volumes). • Journal, to view I/O statistics for a specific journal. Enter a journal number (00-FF) in the Journal box. • Volume, to view I/O statistics for a specific LDEV. Specify the LDEV by entering an LDKC number (00-01), a CU number (00-FE) and an LDEV number (00-FF). 4. In the Monitor Data boxes, select the I/O statistics data that you want to appear on the graph. You must select at least one box. Table 16 (page 90) describes the I/O statistics data. Monitor pair activity, status 89

of the selected I/O statistics data for the selected LU(s). Table 16 Remote copy I/O statistics data Description Data type Host I/O The number of write I/Os per second Write Record Count The amount of data that are written per second. The unit is kilobytes per second. Write Transfer Rate Initial Copy The initial copy hit rate. The unit is percent. Initial Copy Hit Rate The average transfer rate for initial copy operations. The unit is kilobytes per second. Average Transfer Rate Asynchronous Copy The number of asynchronous remote I/Os per second at the primary storage system. M-JNL Asynchronous RIO count The number of journals at the primary storage system. M-JNL Total Number of Journal The average transfer rate for journals in the primary storage system. The unit is kilobytes per second. M-JNL Average Transfer Rate The remote I/O process time on the primary storage system. The unit is milliseconds. M-JNL Average RIO Response The number of asynchronous remote I/Os per second at the secondary storage system. R-JNL Asynchronous RIO count The number of journals at the secondary storage system. R-JNL Total Number of Journal 90 Monitoring the system

Description Data type The average transfer rate for journals in the secondary storage system. The unit is kilobytes per second. R-JNL Average Transfer Rate The remote I/O process time on the secondary storage system. The unit is milliseconds. R-JNL Average RIO Response M-JNL Data usage rate for master journals. The unit is percent. Data Used Rate Metadata usage rate for master journals. The unit is percent. Meta Data Used Rate R-JNL Data usage rate for restore journals. The unit is percent. Data Used Rate Metadata usage rate for restore journals. The unit is percent. Meta Data Used Rate Manipulate graph, save data • To enlarge the graph, right-click it and select Large Size. • To close the graph, right-click it and select Close. • To save monitoring data in text files, use the Performance Monitor Export Tool. Monitor journal status Journal status relates to pair operations performed on the journal. Monitoring these statuses can help you resolve problems and maintain the Continuous Access Journal Z system. Status for journals is viewed on the Journal Operation window. This section explains how to navigate to the Journal Operation window and provides definitions for statuses. To view journal status 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. 2. In the tree, open the Registered child-tree, then select a journal. The group’s information displays in the right-hand pane. See the status field. Mirror Journal status definitions Description Status Journal volumes are registered in the journal but data volumes are not registered. Initial The journal has Cnt Ac-J pairs that are synchronized, or the initial copy is in progress. If a journal in Active status contains pairs that are split, a warning appears. The split pairs must be resynchronized. Active The status at the beginning of a mirror-split operation. The status changes immediately to Halting. HaltAccept indicates status of restore journals only. HaltAccept: An operation for splitting or deleting the mirror is in progress. • Splitting a mirror changes the status in the following order: Halting, Halt, Stopping, and finally Stopped. • Deleting a mirror changes the status in the following order: Halting, Halt, Stopping, Stopped, and finally Initial. Halting Monitor journal status 91

Halt An operation for splitting or deleting the mirror is in progress. Stopping An operation for splitting or deleting the mirror is finished. Stopped A Cnt Ac-J delta resync pair has been created. It is ready for the delta resync operation. Hold An error occurred with the Cnt Ac-J pair during the delta resync operation. Hold(Failure) This is the unused journal which is not registered data volumes. Blank Monitor logical path status To view logical path status 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > DKC Operation. 2. In the Display box, select DKC and select LDKC. 3. Locate the remote system in the list on the right side and view the Status column. 4. To see in-depth path status, right-click the remote system and select DKC Status. Description Item Number of the row. No Status of a logical path. • Normal. The path is established and ready to use for copy operations. • Nothing. An operation for configuring or deleting the path is in progress. • Initialization Failed. An error occurred when the connection between local and remote systems was initializing. The probable causes are: - No cable is connected to the local system. - No cable is connected to the remote system. - No cable is connected to the network device between the two systems. • Communication Time Out. This status indicates one of the following: - A timeout error has occurred between the primary and the secondary storage systems. - A logic error is detected between the primary and the secondary storage systems. • Resource Shortage (Port). The local system rejected configuration of the logical path connection. Logical path resources in the local system might be in use for other connections. • Resource Shortage (Pair-Port). The remote system rejected configuration of the logical path connection. Logical path resources in the remote system might be in use for other connections. • Serial Number Mismatch. The serial number of the storage system connected to this logical path does not match the serial number specified by the Add DKC dialog box. • Invalid Port. The port is not an initiator port. • Pair-Port Number Mismatch. Indicates one of the following: - The specified port number is incorrect. - The port in the remote system is physically disconnected from the local system. • Pair-Port Type Mismatch. The port on the remote system is not an RCU target port. • Communication Failed. A communication timeout error has occurred on the path between the local and remote systems. Path Status Port on the local system. Port 92 Monitoring the system

Pair-Port Serial number and LDKC number of the remote system. S/N Controller ID and model name (in parenthesis) for the remote system. Controller ID Path group ID Path Gr. ID Type of channel interface between local and remote systems. Always displays column displays “Fibre”. M-R Path Minimum possible number of paths between the local and the remote systems. Minimum Paths Remote I/O missing interrupt handler timer value—the wait time for data transfer from the local to remote system to complete. RIO MIH Date and time when local and remote systems were associated to each other DKC Registered Date and time the last operation on a logical path to the remote system was performed. Last Updated When clicked, the DKC Operation window refreshes when it is redisplayed. Refresh the DKC Operation tab after this panel is closed History of operations for data volume pairs The History window provides the history of operations for data volume pairs. For example, the window shows the date and time when data volume pairs are created or released. To open the History window, do either of the following: If Continuous Access Journal Z has not been started: 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > History. The History window opens. The History window may not show the latest operation history. To view the latest operation history, go to the next step. 2. Select File > Refresh. The operation history is updated. If Continuous Access Journal Z has already been started: 1. Click the History tab to open the History window. The History window may not show the latest operation history. To view the latest operation history, go to the next step. 2. Select File > Refresh. The operation history is updated. Monitor logical path status 93

Some maintenance tasks are a response to behavior discovered during system monitoring. However, you can also change certain settings to keep the system in tune with your changing requirements. Pair maintenance—change the pair-split option You can change the option that specifies whether a pair-split operation that is triggered by pair failure is applied to all the pairs in the mirror, or only to the affected pair. This option was first set during the pair create operation. Prerequisite information You can change this option only for pairs in Duplex, Suspend, Hold, Holding, or Hlde status. To change the option 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Pair Operation. 2. Click to change the mode from View to Modify. 3. In the tree, select the port that you have configured for the pair, or a host group (displayed below the ports). The volumes for the port or host group display in the right-side list area, 4. In the list, right-click the desired pair, or mirror of pairs, then select Change Pair Option from the menu. NOTE: If you selected multiple pairs and want to change one of the options on the Change Pair Option dialog box – but do not intend to change the other option, leave blank the option that you do not want to change. 5. In the Change Pair Option dialog box, click the Error Level list and select one of the following: • Mirror, to cause all pairs in the failed pair’s mirror to be split. • Volume, to cause only the pair that failed to be split. 6. In the CFW box, select one of the following to specify whether to copy cache-fast-write data to the S-VOL or not: • Only P-VOL: does not copy cache-fast-write data to the S-VOL (the default). • Copy to S-VOL: copies cache-fast-write data to the S-VOL. In a 3DC multitarget configuration, specify the same CFW setting for both Continuous Access Journal Z and Continuous Access Synchronous Z pairs. If you specify Copy to S-VOL for either the Continuous Access Journal Z or Continuous Access Synchronous Z pair, the cache-fast-write data is copied to the S-VOL of the other pair. 7. Click Set when finished. 8. See the Preview list to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 9. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. 94 Maintaining the system

Maintaining journals and mirror consists of the following operations: • Changing journal options • Changing mirror options • Deleting journal volumes from journals • Deleting journals Change Continuous Access Journal Z options used by journals Most Continuous Access Journal Z pair operations are performed on multiple pairs. This is done using journals. You can change the following options that affect the pairs in journals: • Inflow Control—allows you to restrict the inflow of data to allow other journals with higher priority data unrestricted inflow • Data Overflow Watch—establishes a timeout period • Use of Cache—which enables using store journal data in the cache on the secondary system • Timer Type—establishes the type of clock used for consistency time. Prerequisite information Please be aware of the following when changing journal options: • Journal options must be changed on both systems, primary and secondary. • To change journal options, one of the following conditions must be satisfied: - The attribute of the journal is Initial. - The status of the journal in the mirror is one of the following: ◦ Initial ◦ Stopped ◦ Hold(Failure) When one journal uses multiple mirrors, whether you can select Change Journal Option depends on the status of the journal whose mirror is not in Hold, Holding or Hold(Failure) status. For example, if mirrors in the journal are in the Hold and Active status, you can not change the journal option. If mirrors in the journal are in the Hold and Stop status, you can change the journal option. • When changing options for multiple journals, you can leave some options unchanged by entering or selecting no value. If you specify a value, it is applied to the selected journals. To change options for a journal 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. The Journal Operation window displays. 2. Click to change the mode from View to Modify. 3. Operate either of the following: In the tree, right click a journal from below Registered and then select Journal > Change Journal Option. In the upper right list, right click the desired journal and then select Journals > Change Journal Option. Journal and mirror maintenance 95

(slows delay response to hosts). For example, you could have a 100 GB-bandwidth data path with three journals using it. If an important database is saved to a primary volume in one of the journals, with data of lesser importance in the other journals, you may decide to restrict the Inflow Control for the less important journals while not restricting control to the important journal. Yes indicates inflow will be restricted. No indicates inflow will not be restricted. If Yes is selected and the metadata or the journal data is full, the update I/Os may stop. 5. For Data Overflow Watch, specify the number of seconds for the system to monitor write data to the journal volume when the journal volume threshold (80%) is reached. Range is 0-600 seconds. Default is 60 seconds. If Inflow Control is No, Data Overflow Watch is disabled. 6. For Use of Cache, specify whether to store journal data (initial copy or resynchronization data) in the cache memory on the remote side (restore journal). - Use: Journal data will be stored in the cache. When there is insufficient space in the cache, journal data will also be stored into the journal volume. This setting only takes effect on the journal volumes of RAID-5 or RAID-6. Not Use: Journal data is not stored in the cache. - Blank: The current setting of Use of Cache will remain unchanged. NOTE: This setting does not effect master journals unless the Business Continuity Manager YKRESYNC REVERSE command is used to change a master journal into a restore journal. 7. For Timer Type, specify the type of clock used for consistency time. This option can only be selected when no journal volume is registered. The selected option must be the same for primary and secondary systems. - System, the system clock of the mainframe host is used. The host MVS IOS routine attaches timestamp information from the time-of-day (TOD) clock to each write I/O operation. The timestamp is transferred to the primary system at the beginning of each I/O operation. - Local, the mainframe system clock is not used. - None is available only when copy direction of the pair is reversed, S-VOL-to-P-VOL. With None, the mainframe system clock is used. 8. Click Set when finished. 96 Maintaining the system

• If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 10. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Change Continuous Access Journal Z options used by mirrors Most Continuous Access Journal Z pair operations are performed on multiple pairs. This is done using a journal or mirrors. You can change the following options that affect the pairs in a mirror: • Copy Pace—establishes the pace at which data is copied • Path Watch Time—establishes the amount of time before a blocked path results in the mirror being split • Forward Path Watch Time—establishes whether to forward the Path Watch Time value of the master journal to the restore journal. • Transfer Speed—which specifies the speed for data transfer • Delta Resync Failure—which specifies the processing that takes place in the event of a failure Prerequisite information Please be aware of the following when changing mirror options: • Mirror options must be changed on both systems, primary and secondary. • To change mirror options, the status of the mirror is one of the following: Initial ◦ ◦ Active ◦ Stopped ◦ Hold ◦ Holding ◦ Hold(Failure) • The Transfer Speed option can be changed if the mirror’s status is Active, Hold, or Holding. • When changing options for multiple mirrors, you can leave some options unchanged by entering or selecting no value. If you specify a value, it is applied to the selected mirrors. To change options for a mirror 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. The Journal Operation window displays. 2. Click to change the mode from View to Modify. 3. Operate either of the following: In the tree, right click a journal from below Registered and then select Mirror > Change Mirror Option. In the upper right list, right click the desired journal and then select Mirror > Change Mirror Option. Journal and mirror maintenance 97

specified on the remote system. Low is the default. When specifying Medium, insure that write I/O is 10 MB/s or less per parity group. If it exceeds 10 MB/s, pairs may be suspended. When specifying High, insure that I/O will not occur. If update I/O occurs, pairs may be suspended. 5. For Unit of Path Watch Time, specify the unit of time for Path Watch Time, minute, hour, or day. 6. For Path Watch Time, specify the interval from the time a path becomes blocked to when the mirror is split (suspended). The interval must be the same for master and restore journals in the same mirror (see next item). Important: If you want a mirror to split immediately after a path becomes blocked, ask HP Technical Support to set the system option mode 448 to ON and set the system option mode 449 to OFF. 7. For Forward Path Watch Time, specify whether to forward the Path Watch Time value of the master journal to the restore journal. If the Path Watch Time value is forwarded from the master journal to the restore journal, the two journals will have the same Path Watch Time value. - Yes: The Path Watch Time value will be forwarded to the restore journal. - No: The Path Watch Time value will not be forwarded to the restore journal. No is the default. - Blank: The current setting of Forward Path Watch Time will remain unchanged. CAUTION: This option cannot be specified in the secondary site. For a pair using the delta resync configuration, select Yes in the primary site. 8. For Transfer Speed, specify the line speed (in Mbps (megabits per second)) of data transfer. Specify one of the following: 256, 100, or 10. Recommended values are as follows: - 10 is recommended if the transfer speed is from 10 Mbps to 99 Mbps. - 00 is recommended if the transfer speed is from 100 Mbps to 255 Mbps. - 256 is recommended if the transfer speed is 256 Mbps and more. NOTE: This setting does not effect master journals unless the Business Continuity Manager YKRESYNC REVERSE command is used to change a master journal into a restore journal. 9. For Delta resync Failure, specify the processing that takes place when the delta resync operation cannot be performed. (This option specified on the primary site only.) - Entire, the default. The entire P-VOL is copied to the S-VOL. 98 Maintaining the system

10. Click Set when finished. 11. See the Preview list in the Journal Operation window to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. 12. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Delete journal volumes from a journal You can delete journal volumes as needed. This section provides instructions and important information for doing so. Prerequisite information • A journal volume can only be deleted if either of the following conditions exist: If the pair belonging to the mirror in the journal is suspended. ◦ ◦ If the pair belonging to the mirror in the journal is deleted. • If you delete a mainframe journal volume from a journal where a data volume pair has ever been registered, the deleted volume (LDEV) is blocked. To reuse the volume as a data volume, you must format the volume. Unless you format the volume, data in the volume is not guaranteed. For instructions on formatting volumes, see the HP StorageWorks P9000 Provisioning for Mainframe Systems User Guide. • The following must be true to delete a journal volume: The status of all mirrors in the journal is Initial, Stopped, or Hold (Failure). However, if a journal uses multiple mirrors, journal volumes can only be deleted as shown below. Table 17 Deleting journal volumes with multiple mirror IDs Can the journal volumes be deleted? Status of journal Mirror ID 2 Mirror ID 1 No. Hold Active No. Hold(Failure) Active Yes. Hold Stopped Yes. Hold(Failure) Stopped To delete journal volumes 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. The Journal Operation window displays. 2. Click to change the mode from View to Modify. 3. From the tree, click Registered. Journals display in the list on the right. 4. Select and right-click the desired journal. Only one journal can be selected. Click Edit Journal Volumes from the menu that displays. The Edit Journal Volumes dialog box displays. 5. From the Journal Volumes list, select the volumes that you want to delete. 6. Click the Delete. Journal and mirror maintenance 99

8. See the Preview list to check the settings that you have made. • If you want to modify a setting, select and right click the setting in the Preview list and then select Modify. • If you want to cancel a setting, select and right click the setting and then select Cancel. • If necessary, repeat steps from 2 to 6 to specify volumes that should be deleted from other journals. 9. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. Delete a journal Journals can be deleted from either storage system. Prerequisite information • When all mirrors in the journal are in the Initial status, you can delete the journal. • Master and restore journals cannot be deleted. To delete a journal 1. In Remote Web Console, select Actions > Remote Copy > Continuous Access Journal Z > Journal Operation. The Journal Operation window displays. 2. Click to change the mode from View to Modify. 3. From the tree, click Registered. Journals display in the list on the right. 4. Select and right-click the desired journal. Only one journal can be selected. Click Journal > Delete Journal from the menu that displays. 5. See the Preview list to check the settings that you have made. If you want to cancel a setting, select and right click the setting and then select Cancel. 6. Click Apply to apply the settings. If an error occurs, the error code appears in the rightmost column of the Preview list. To view detailed information about the error, right click the error code and select Error Detail. 7. The deleted journal displays in the Preview list. You can cancel the operation by clicking the Cancel. To complete the deletion, click Apply. Logical path maintenance This section provides the following: • Modifying Path Volume Thresholds • Deleting Logical Paths • Deleting the Continuous Access Journal Z Relationship Modify data-transfer time threshold You can modify the threshold for data transfer to complete. If the threshold value is reached, the transfer is flagged as failing by the system. 100 Maintaining the system