IBM DS3500 Installation, User & Maintenance Manual

and see the Warranty Information document that comes with the storage subsystem. First Edition (May 2010) © Copyright IBM Corporation 2010. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

. . . . . . . . . . . . . . . . . . . . . 1 DS3500 storage subsystem overview . . . . . . . . . . . . . . . . . 2 EXP3500 expansion enclosure overview . . . . . . . . . . . . . . . . 3 Notices and statements in this document. . . . . . . . . . . . . . . . 3 Features and operating specifications . . . . . . . . . . . . . . . . . 4 Models and optional devices . . . . . . . . . . . . . . . . . . . . 5 Operating-system support . . . . . . . . . . . . . . . . . . . . . 5 Product updates . . . . . . . . . . . . . . . . . . . . . . . . . 5 Best practices guidelines. . . . . . . . . . . . . . . . . . . . . . 5 DS3500 storage subsystem and EXP3500 expansion enclosure components . . 6 Disk drives and bezels . . . . . . . . . . . . . . . . . . . . . 7 Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Environmental service modules . . . . . . . . . . . . . . . . . . 10 Power supply and fans . . . . . . . . . . . . . . . . . . . . . 10 Battery units . . . . . . . . . . . . . . . . . . . . . . . . . 11 Software and hardware compatibility and upgrades . . . . . . . . . . . 11 Software and firmware support code upgrades . . . . . . . . . . . . 11 Determining firmware levels . . . . . . . . . . . . . . . . . . . 12 Specifications . . . . . . . . . . . . . . . . . . . . . . . . . 12 Area requirements . . . . . . . . . . . . . . . . . . . . . . 12 Dimensions . . . . . . . . . . . . . . . . . . . . . . . . 12 Weight . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Temperature and humidity . . . . . . . . . . . . . . . . . . . . 14 Electrical requirements . . . . . . . . . . . . . . . . . . . . . 14 Site wiring and power . . . . . . . . . . . . . . . . . . . . 15 AC power recovery . . . . . . . . . . . . . . . . . . . . . 15 Power cords and receptacles. . . . . . . . . . . . . . . . . . 15 Heat output, airflow, and cooling . . . . . . . . . . . . . . . . . 15 Chapter 2. Installing the storage subsystem and expansion enclosure. . . 17 Installing the DS3500 storage subsystem . . . . . . . . . . . . . . . 17 Inventory checklist . . . . . . . . . . . . . . . . . . . . . . 17 DS3500 installation overview. . . . . . . . . . . . . . . . . . . 18 Handling static-sensitive devices . . . . . . . . . . . . . . . . . 19 Preparing for installation . . . . . . . . . . . . . . . . . . . . 19 Required tools and hardware. . . . . . . . . . . . . . . . . . 20 Preparing the site . . . . . . . . . . . . . . . . . . . . . . 20 Installing the DS3500 storage subsystem in a rack. . . . . . . . . . . 21 Installing the EXP3500 expansion enclosure . . . . . . . . . . . . . . 21 Inventory checklist . . . . . . . . . . . . . . . . . . . . . . 21 Installing the EXP3500 expansion enclosure in a rack . . . . . . . . . 22 Chapter 3. Cabling the storage subsystem and expansion enclosure . . . 23 Cabling the DS3500 storage subsystem. . . . . . . . . . . . . . . . 23 Controller connectors (with Fibre Channel host port adapters) . . . . . . 23 Controller connectors (with iSCSI host port adapters). . . . . . . . . . 23 Controller connectors (with SAS host port adapters) . . . . . . . . . . 24 © Copyright IBM Corp. 2010 iii

. . . . . . . . . . . . . . . . . . . . . 25 Working with SAS cables . . . . . . . . . . . . . . . . . . . . 25 Working with SFP modules and fiber optic cables . . . . . . . . . . . 27 Handling fiber optic cables . . . . . . . . . . . . . . . . . . 27 Installing SFP modules . . . . . . . . . . . . . . . . . . . . 28 Removing SFP modules . . . . . . . . . . . . . . . . . . . 30 Using LC-LC Fibre Channel cables . . . . . . . . . . . . . . . 31 Using LC-SC Fibre Channel cable adapters . . . . . . . . . . . . 34 Cabling the EXP3500 expansion enclosure . . . . . . . . . . . . . 37 ESM connectors . . . . . . . . . . . . . . . . . . . . . . 37 Single-ESM configuration . . . . . . . . . . . . . . . . . . . 37 Dual-ESM configuration. . . . . . . . . . . . . . . . . . . . 38 Connecting expansion enclosures to the DS3500 . . . . . . . . . . . 40 Redundant drive channel pair . . . . . . . . . . . . . . . . . 40 Overview of steps to connect expansion enclosures to a storage subsystem. . . . . . . . . . . . . . . . . . . . . . . . 41 DS3500 storage subsystem drive cabling topologies . . . . . . . . . 42 Adding an expansion enclosure to a running dual-controller configuration 44 Upgrading a single-controller to a dual-controller when there is no attached EXP3500 enclosure . . . . . . . . . . . . . . . . . . . . . 46 Upgrading a single-controller to a dual-controller when there is one or more EXP3500 enclosures attached to the storage subsystem. . . . . . . . 46 Connecting secondary interface cables . . . . . . . . . . . . . . . 46 Configuring the storage subsystem . . . . . . . . . . . . . . . . 47 Storage subsystem management methods. . . . . . . . . . . . . 47 Installing the storage subsystem configuration . . . . . . . . . . . 49 Connecting SAS hosts to the DS3500 . . . . . . . . . . . . . . 49 Connecting Fibre Channel hosts to the DS3500 . . . . . . . . . . . 53 Connecting iSCSI hosts to the DS3500 . . . . . . . . . . . . . . 59 Overview of installing an IBM BladeCenter configuration to connect to a DS3500. . . . . . . . . . . . . . . . . . . . . . . . . . 64 Installing the BladeCenter configuration . . . . . . . . . . . . . . 64 Connecting BladeCenter hosts to the DS3500 . . . . . . . . . . . 64 Cabling the DS3500 and EXP3500 power supplies . . . . . . . . . . . 65 Chapter 4. Operating the storage subsystem and expansion enclosure 67 Performing the health check process . . . . . . . . . . . . . . . . . 67 Hardware inspection . . . . . . . . . . . . . . . . . . . . . . . 68 Turning on the storage subsystem . . . . . . . . . . . . . . . . . . 68 Installing the DS Storage Manager Client . . . . . . . . . . . . . . . 70 Monitoring status through software . . . . . . . . . . . . . . . . . 71 Firmware updates . . . . . . . . . . . . . . . . . . . . . . . 72 Troubleshooting the storage subsystem . . . . . . . . . . . . . . . 72 Checking the LEDs . . . . . . . . . . . . . . . . . . . . . . . 73 Power-supply LEDs . . . . . . . . . . . . . . . . . . . . . . 73 Front LEDs . . . . . . . . . . . . . . . . . . . . . . . . . 74 Controller LEDs . . . . . . . . . . . . . . . . . . . . . . . 75 Seven-segment numeric display LEDs . . . . . . . . . . . . . . . 78 ESM LEDs . . . . . . . . . . . . . . . . . . . . . . . . . 80 Turning off the storage subsystem . . . . . . . . . . . . . . . . . . 81 Performing an emergency shutdown . . . . . . . . . . . . . . . . 83 Restoring power after an unexpected shutdown . . . . . . . . . . . . 83 Recovering from an overheated power supply . . . . . . . . . . . . . 84 Cache memory and cache battery . . . . . . . . . . . . . . . . . . 86 Cache memory . . . . . . . . . . . . . . . . . . . . . . . . 86 Controller cache battery . . . . . . . . . . . . . . . . . . . . 87 iv System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

. . . . . . . . . . . . . . . . . . . 87 EXP3500 expansion enclosure power features . . . . . . . . . . . . . 88 Turning on the expansion enclosure . . . . . . . . . . . . . . . . 88 Turning off the expansion enclosure . . . . . . . . . . . . . . . . 89 Turning off the expansion enclosure in an emergency . . . . . . . . . 90 Turning on the expansion enclosure after an emergency . . . . . . . . 90 Chapter 5. Replacing components . . . . . . . . . . . . . . . . . 93 OK to remove LED . . . . . . . . . . . . . . . . . . . . . . . 93 Working with controllers . . . . . . . . . . . . . . . . . . . . . 93 Removing a controller . . . . . . . . . . . . . . . . . . . . . 93 Removing and installing a cover . . . . . . . . . . . . . . . . . 94 Installing a controller . . . . . . . . . . . . . . . . . . . . . . 95 Replacing a controller . . . . . . . . . . . . . . . . . . . . . 96 Removing and disposing of the system-board lithium battery . . . . . . . 99 Installing an optional host port adapter. . . . . . . . . . . . . . . . 101 Working with hot-swap hard disk drives . . . . . . . . . . . . . . . 103 Removing a hard disk drive . . . . . . . . . . . . . . . . . . . 105 Installing a hard disk drive . . . . . . . . . . . . . . . . . . . 106 Replacing a hot-swap hard disk drive . . . . . . . . . . . . . . . 108 Replacing multiple drives. . . . . . . . . . . . . . . . . . . . 108 Replacing all drives at the same time . . . . . . . . . . . . . . 110 Replacing the drives one at a time . . . . . . . . . . . . . . . 112 Replacing a power supply . . . . . . . . . . . . . . . . . . . . 114 Replacing a battery . . . . . . . . . . . . . . . . . . . . . . . 118 Replacing the memory cache DIMM . . . . . . . . . . . . . . . . 120 Removing the DIMM . . . . . . . . . . . . . . . . . . . . . 120 Installing the DIMM . . . . . . . . . . . . . . . . . . . . . . 121 Replacing the bezels . . . . . . . . . . . . . . . . . . . . . . 122 Removing the bezels . . . . . . . . . . . . . . . . . . . . . 123 Installing the bezels. . . . . . . . . . . . . . . . . . . . . . 123 Working with environmental service modules . . . . . . . . . . . . . 123 Installing an additional ESM. . . . . . . . . . . . . . . . . . . 123 Replacing an ESM . . . . . . . . . . . . . . . . . . . . . . 124 Replacing a midplane assembly . . . . . . . . . . . . . . . . . . 125 Chapter 6. Solving problems. . . . . . . . . . . . . . . . . . . 127 Solving problems in the DS3500 storage subsystem. . . . . . . . . . . 127 Solving problems in the EXP3500 expansion enclosure . . . . . . . . . 132 Chapter 7. Parts listing, DS3500 storage subsystem and EXP3500 expansion enclosure . . . . . . . . . . . . . . . . . . . . . 135 Replaceable components . . . . . . . . . . . . . . . . . . . . 135 DS3512 storage subsystem and EXP3512 expansion enclosure parts listing 135 DS3524 storage subsystem and EXP3524 expansion enclosure parts listing 137 Power cords . . . . . . . . . . . . . . . . . . . . . . . . . 138 Appendix A. Records . . . . . . . . . . . . . . . . . . . . . 141 Identification numbers . . . . . . . . . . . . . . . . . . . . . . 141 Hard disk drive locations . . . . . . . . . . . . . . . . . . . . . 142 Storage subsystem and controller information record . . . . . . . . . . 143 Appendix B. Getting help and technical assistance . . . . . . . . . . 145 Before you call . . . . . . . . . . . . . . . . . . . . . . . . 145 Using the documentation. . . . . . . . . . . . . . . . . . . . . 145 Getting help and information from the World Wide Web . . . . . . . . . 145 Contents v

. . . . . . . . . . . . . . . . . . 146 Hardware service and support . . . . . . . . . . . . . . . . . . . 146 IBM Taiwan product service. . . . . . . . . . . . . . . . . . . . 146 Appendix C. Notices . . . . . . . . . . . . . . . . . . . . . . 147 Trademarks. . . . . . . . . . . . . . . . . . . . . . . . . . 147 Important notes . . . . . . . . . . . . . . . . . . . . . . . . 148 Particulate contamination. . . . . . . . . . . . . . . . . . . . . 149 Documentation format . . . . . . . . . . . . . . . . . . . . . . 149 Electronic emission notices . . . . . . . . . . . . . . . . . . . . 150 Federal Communications Commission (FCC) statement . . . . . . . . 150 Industry Canada Class A emission compliance statement . . . . . . . . 150 Avis de conformité à la réglementation d'Industrie Canada . . . . . . . 150 Australia and New Zealand Class A statement . . . . . . . . . . . . 150 European Union EMC Directive conformance statement . . . . . . . . 150 Germany Class A statement . . . . . . . . . . . . . . . . . . 151 Deutschland: Einhaltung des Gesetzes über die elektromagnetische Verträglichkeit von Geräten . . . . . . . . . . . . . . . . . 151 Zulassungsbescheinigung laut dem Deutschen Gesetz über die elektromagnetische Verträglichkeit von Geräten (EMVG) (bzw. der EMC EG Richtlinie 2004/108/EG) für Geräte der Klasse A . . . . . . . . 151 Japan VCCI Class A statement . . . . . . . . . . . . . . . . . 152 Japan Electronics and Information Technology Industries Association (JEITA) statement . . . . . . . . . . . . . . . . . . . . . . . . 152 Korea Communications Commission (KCC) statement . . . . . . . . . 152 Russia Electromagnetic Interference (EMI) Class A statement . . . . . . 152 People's Republic of China Class A electronic emission statement . . . . 153 Taiwan Class A compliance statement . . . . . . . . . . . . . . . 153 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 vi System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

. . . . 2 2. DS3512 storage subsystem and EXP3512 expansion enclosure hot-swap drive bays and bezels 7 3. DS3524 storage subsystem and EXP3524 expansion enclosure hot-swap drive bays and bezels 7 4. Left-side bezel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5. Right-side bezel (DS3512 and EXP3512) . . . . . . . . . . . . . . . . . . . . . . 8 6. Bottom shelf (DS3524 and EXP3524). . . . . . . . . . . . . . . . . . . . . . . . 8 7. Rear view, single-controller model (shown without an optional host port adapter) . . . . . . . 10 8. Rear view, single ESM model expansion enclosure . . . . . . . . . . . . . . . . . . 10 9. Power supply components . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 10. Battery unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 11. DS3512 storage subsystem and EXP3512 expansion enclosure dimensions . . . . . . . . . 13 12. DS3524 storage subsystem and EXP3524 expansion enclosure dimensions . . . . . . . . . 13 13. Example of cold aisle/hot aisle rack configuration . . . . . . . . . . . . . . . . . . . 16 14. Dual-controller DS3500 storage subsystem ports and controllers (with optional Fibre Channel host port adapter) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 15. Dual-controller DS3500 storage subsystem ports and controllers (with optional iSCSI host port adapter) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 16. Dual-controller DS3500 storage subsystem ports and controllers (with an optional SAS host port adapter) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 17. Mini-SAS cable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 18. Connecting a mini-SAS cable . . . . . . . . . . . . . . . . . . . . . . . . . . 26 19. Removing a mini-SAS cable. . . . . . . . . . . . . . . . . . . . . . . . . . . 26 20. SFP module and protective cap . . . . . . . . . . . . . . . . . . . . . . . . . 29 21. Installing an SFP module into the host port . . . . . . . . . . . . . . . . . . . . . 30 22. Unlocking the SFP module latch - plastic variety . . . . . . . . . . . . . . . . . . . 30 23. Unlocking the SFP module latch - wire variety . . . . . . . . . . . . . . . . . . . . 31 24. LC-LC Fibre Channel cable . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 25. Removing fiber optic cable protective caps . . . . . . . . . . . . . . . . . . . . . 32 26. Inserting an LC-LC Fibre Channel cable into an SFP module . . . . . . . . . . . . . . 33 27. LC-LC Fibre Channel cable lever and latches . . . . . . . . . . . . . . . . . . . . 33 28. Removing the LC-LC Fibre Channel cable . . . . . . . . . . . . . . . . . . . . . 34 29. LC-SC Fibre Channel cable adapter. . . . . . . . . . . . . . . . . . . . . . . . 34 30. Removing the LC-SC cable adapter protective caps . . . . . . . . . . . . . . . . . . 35 31. Connecting an LC-LC cable into the LC-SC cable adapter . . . . . . . . . . . . . . . 36 32. LC-LC Fibre Channel cable lever and latches . . . . . . . . . . . . . . . . . . . . 36 33. Removing the LC-LC Fibre Channel cable from an LC-SC Fibre Channel cable adapter. . . . . 37 34. ESM connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 35. Example of a redundant drive path . . . . . . . . . . . . . . . . . . . . . . . . 41 36. One single-controller DS3500 and multiple single-ESM expansion enclosures . . . . . . . . 42 37. One dual-controller DS3500 and one expansion enclosure . . . . . . . . . . . . . . . 43 38. One dual-controller DS3500 and two expansion enclosures . . . . . . . . . . . . . . . 43 39. One dual-controller DS3500 and three expansion enclosures . . . . . . . . . . . . . . 44 40. Ethernet port locations on a dual-controller DS3500 . . . . . . . . . . . . . . . . . . 47 41. Host-agent (in-band) managed storage subsystems . . . . . . . . . . . . . . . . . . 48 42. Direct (out-of-band) managed storage subsystems . . . . . . . . . . . . . . . . . . 49 43. Single-controller direct-attached SAS connection to a single-host HBA . . . . . . . . . . . 51 44. Single-controller direct-attached SAS connection to dual HBAs in a single-host (redundant host connection) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 45. Single-controller direct-attached SAS connection to single HBAs in multiple hosts . . . . . . . 52 46. Dual-controller direct-attached SAS connection to two HBAs in the same host (redundant host connection) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 47. Dual-controller direct-attached SAS connections to multiple HBAs in multiple hosts . . . . . . 53 48. Location of host connectors on RAID controllers . . . . . . . . . . . . . . . . . . . 54 © Copyright IBM Corp. 2010 vii

. . . . . . . 54 50. Single-controller direct-attached Fibre Channel connection to dual HBAs (redundant host connection) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 51. Single-controller direct-attached Fibre Channel connection to single HBAs in multiple hosts 55 52. Dual-controller direct-attached Fibre Channel connection to two HBAs in the same host (redundant host connection). . . . . . . . . . . . . . . . . . . . . . . . . . . 56 53. Dual-controller direct-attached Fibre Channel connections to multiple HBAs in multiple hosts 57 54. Example of a single Fibre Channel SAN fabric configuration . . . . . . . . . . . . . . . 58 55. Example of a dual Fibre Channel SAN fabric configuration . . . . . . . . . . . . . . . 58 56. Example of two storage subsystems in a dual Fibre Channel SAN environment. . . . . . . . 59 57. Single-host, single-controller iSCSI configuration . . . . . . . . . . . . . . . . . . . 60 58. Multiple-port, single-controller iSCSI configuration. . . . . . . . . . . . . . . . . . . 60 59. Multiple single-port hosts, single-controller iSCSI configuration . . . . . . . . . . . . . . 61 60. Multiple-host, multiple-port, single-controller, and redundant-path iSCSI configuration. . . . . . 61 61. Multiple-host, multiple-port, multiple-storage-subsystem iSCSI configuration . . . . . . . . . 62 62. Single-host, multiple-port, dual-controller iSCSI configuration. . . . . . . . . . . . . . . 62 63. Multiple-host, multiple-port, dual-controller iSCSI configuration . . . . . . . . . . . . . . 63 64. Example of a multiple-host, multiple-port, and multiple-fabric (Fibre Channel or iSCSI, and SAS) configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 65. Example of a DS3500 that is connected to one BladeCenter unit . . . . . . . . . . . . . 65 66. Power-supply switches and connectors for the DS3500. . . . . . . . . . . . . . . . . 69 67. Power-supply LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 68. DS3512 storage subsystem and EXP3512 expansion enclosure front LEDs and controls . . . . 74 69. DS3524 storage subsystem and EXP3524 expansion enclosure front LEDs . . . . . . . . . 75 70. Controller LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 71. iSCSI host port adapter LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . 77 72. Fibre Channel host port adapter LEDs . . . . . . . . . . . . . . . . . . . . . . . 77 73. SAS host port adapter LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . 78 74. Numeric display LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 75. ESM LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 76. Removing a controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 77. Removing the cover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 78. Installing a controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 79. Removing and replacing a controller. . . . . . . . . . . . . . . . . . . . . . . . 98 80. Removing the battery unit from the controller . . . . . . . . . . . . . . . . . . . . 98 81. Removing a controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 82. Removing the host port adapter filler panel. . . . . . . . . . . . . . . . . . . . . 102 83. Installing a host port adapter . . . . . . . . . . . . . . . . . . . . . . . . . . 102 84. DS3512 storage subsystem and EXP3512 expansion enclosure hard disk drive LEDs . . . . . 104 85. DS3524 storage subsystem and EXP3524 expansion enclosure hard disk drive LEDs . . . . . 104 86. Removing a drive from a DS3512 or EXP3512 . . . . . . . . . . . . . . . . . . . 105 87. Removing a drive from a DS3524 or EXP3524 . . . . . . . . . . . . . . . . . . . 106 88. Installing a drive in a DS3512 or EXP3512 . . . . . . . . . . . . . . . . . . . . . 107 89. Installing a drive in a DS3524 or EXP3524 . . . . . . . . . . . . . . . . . . . . . 107 90. Replacing a power supply . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 91. Removing a controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 92. Removing a battery unit from the controller . . . . . . . . . . . . . . . . . . . . . 119 93. Memory cache DIMM location . . . . . . . . . . . . . . . . . . . . . . . . . 120 94. Removing a controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 95. Removing the DIMM from the controller . . . . . . . . . . . . . . . . . . . . . . 121 96. Installing the DIMM in the controller . . . . . . . . . . . . . . . . . . . . . . . 122 97. Removing the bezels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 98. Removing an environmental service module . . . . . . . . . . . . . . . . . . . . 124 99. Replacing the midplane assembly in a DS3512 or EXP3512 . . . . . . . . . . . . . . 125 100. Replacing the midplane assembly in a DS3524 or EXP3524 . . . . . . . . . . . . . . 126 101. DS3512 storage subsystem and EXP3512 expansion enclosure storage subsystem parts 136 viii System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

. . . . . . . . . . 137 103. Serial number location on the DS3500 . . . . . . . . . . . . . . . . . . . . . . 141 Figures ix

. . . . . . . . . . . . . . . . . . . . . . . 4 2. DS3512 storage subsystem and EXP3512 expansion enclosure weights . . . . . . . . . . 13 3. DS3524 storage subsystem and EXP3524 expansion enclosure weights . . . . . . . . . . 14 4. DS3500 component weights. . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5. Maximum number of EXP3512 and EXP3524 expansion enclosures . . . . . . . . . . . . 40 6. Fibre Channel port LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 7. Controller numeric display diagnostic codes . . . . . . . . . . . . . . . . . . . . . 79 8. ESM numeric display diagnostic codes. . . . . . . . . . . . . . . . . . . . . . . 79 9. Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 10. Troubleshooting information . . . . . . . . . . . . . . . . . . . . . . . . . . 132 11. Parts listing for DS3500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 12. Parts listing for DS3500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 13. IBM power cords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 14. Product identification record . . . . . . . . . . . . . . . . . . . . . . . . . . 141 15. Drive location information record . . . . . . . . . . . . . . . . . . . . . . . . 142 16. Storage subsystem and controller information record . . . . . . . . . . . . . . . . . 143 17. Sample information record . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 18. Limits for particulates and gases . . . . . . . . . . . . . . . . . . . . . . . . 149 © Copyright IBM Corp. 2010 xi

Antes de instalar este produto, leia as Informações de Segurança. Pred instalací tohoto produktu si prectete prírucku bezpecnostních instrukcí. Læs sikkerhedsforskrifterne, før du installerer dette produkt. Lees voordat u dit product installeert eerst de veiligheidsvoorschriften. Ennen kuin asennat tämän tuotteen, lue turvaohjeet kohdasta Safety Information. Avant d'installer ce produit, lisez les consignes de sécurité. Vor der Installation dieses Produkts die Sicherheitshinweise lesen. Prima di installare questo prodotto, leggere le Informazioni sulla Sicurezza. Les sikkerhetsinformasjonen (Safety Information) før du installerer dette produktet. Antes de instalar este produto, leia as Informações sobre Segurança. Antes de instalar este producto, lea la información de seguridad. Läs säkerhetsinformationen innan du installerar den här produkten. © Copyright IBM Corp. 2010 xiii

number. This number is used to cross reference the English-language caution or danger statement with translated versions of the caution or danger statement in the IBM Systems Safety Notices document. For example, if a caution statement is labeled “D005a,” translations for that caution statement are in the IBM Systems Safety Notices document under “D005a.” Be sure to read all caution and danger statements in this document before you perform the procedures. Read any additional safety information that comes with the server or optional device before you install the device. DANGER Hazardous voltage, current, or energy levels are present inside any component that has this label attached. Do not open any cover or barrier that contains this label. (L001) xiv System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

(L002) DANGER Multiple power cords. The product might be equipped with multiple power cords. To remove all hazardous voltages, disconnect all power cords. (L003) 1 2 or ! 1 2 Safety xv

Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard: v Connect power to this unit only with the provided power cord. Do not use the provided power cord for any other product. v Do not open or service any power supply assembly. v Do not connect or disconnect any cables or perform installation, maintenance, or reconfiguration of this product during an electrical storm. v The product might be equipped with multiple power cords. To remove all hazardous voltages, disconnect all power cords. v Connect all power cords to a properly wired and grounded electrical outlet. Ensure that the outlet supplies proper voltage and phase rotation according to the system rating plate. v Connect any equipment that will be attached to this product to properly wired outlets. v When possible, use one hand only to connect or disconnect signal cables. v Never turn on any equipment when there is evidence of fire, water, or structural damage. v Disconnect the attached power cords, telecommunications systems, networks, and modems before you open the device covers, unless instructed otherwise in the installation and configuration procedures. v Connect and disconnect cables as described in the following procedures when installing, moving, or opening covers on this product or attached devices. To disconnect: 1. Turn off everything (unless instructed otherwise). 2. Remove the power cords from the outlets. 3. Remove the signal cables from the connectors. 4. Remove all cables from the devices. To connect: 1. Turn off everything (unless instructed otherwise). 2. Attach all cables to the devices. 3. Attach the signal cables to the connectors. 4. Attach the power cords to the outlets. 5. Turn on the devices. (D005a) xvi System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

or 18-32 kg (39.7-70.5 lb) The weight of this part or unit is between 18 and 32 kg (39.7 and 70.5 lb). It takes two persons to safely lift this part or unit. (C009) Safety xvii

This chapter describes the operating specifications, features, and components of the IBM® System Storage® DS3500 storage subsystem and the EXP3500 expansion enclosure. The DS3500 storage subsystem and EXP3500 expansion enclosure consist of the following models: v DS3512 storage subsystem v DS3524 storage subsystem v EXP3512 expansion enclosure v EXP3524 expansion enclosure This chapter also includes an inventory checklist and important information about best practices guidelines and product updates for your DS3500 storage subsystem and EXP3500 expansion enclosure. If firmware and documentation updates are available, you can download them from the IBM Web site. The DS3500 storage subsystem and EXP3500 expansion enclosure might have features that are not described in the documentation that comes with the device, and the documentation might be updated occasionally to include information about those features, or technical updates might be available to provide additional information that is not included in the DS3500 storage subsystem and EXP3500 expansion enclosure documentation. To check for updates, complete the following steps: 1. Go to http://www.ibm.com/servers/storage/support/. 2. On the “Support for System Storage and TotalStorage products” page, under Select your product, in the Product family field, select Disk systems. 3. In the Product field, select DS3500 or EXP3500. 4. Click Go. 5. Make the following selections: v For firmware updates, click the Download tab. v For documentation updates, click the Install and use tab. Note: Changes are made periodically to the IBM Web site. Procedures for locating firmware and documentation might vary slightly from what is described in this document. The DS3500 storage subsystem and EXP3500 expansion enclosure come with a limited warranty. For more information about the terms of your warranty, see the Warranty Information document that comes with the device. Record information about the DS3500 storage subsystem and EXP3500 expansion enclosure in Table 14 on page 141. You will need this information if you have to call for service. Figure 1 on page 2 shows the location of the serial number label, and the product name, and machine type, model, and serial number label. Note: The illustrations in this document might differ slightly from your hardware. © Copyright IBM Corp. 2010 1

in or attached to the DS3500 storage subsystem and EXP3500 expansion enclosure. This information can be helpful when you install additional hard disk drives or if you have to report a hardware problem. Make a copy of this table before you record information in it, in case you need extra space to write new values later, or when you update the DS3500 storage subsystem and EXP3500 expansion enclosure configuration. DS3500 storage subsystem overview With direct-attached Serial Attached SCSI (SAS) connectivity and support for RAID levels 0, 1, 3, 5, and 6, the DS3500 storage subsystem provides an internal physical storage capacity that is scalable up to 24 TB (terabytes) for the DS3512 and 12 TB for the DS3524 storage subsystem. Notes: 1. RAID 6 uses a P+Q design implementation. 2. When RAID level 1 is implemented and the number of hard disk drives increases to more than two, RAID level 10 is automatically implemented. The DS3500 is a 2U rack-mountable storage subsystem that supports up to two redundant, dual-active RAID controllers. The DS3500 storage subsystem comes standard with two 6 Gbps x4 SAS host ports and a single 6 Gbps x4 SAS expansion port per controller. Each RAID controller contains 1 GB of cache memory that is upgradable to 2 GB of cache. Also, each controller supports the installation of the following host port adapters: v 6 Gbps SAS v 8 Gbps FC v 1 Gbps iSCSI The DS3500 storage subsystem supports up to 96 hard disk drives when attached to EXP3500 expansion enclosures. The DS3500 storage subsystem and EXP3500 expansion enclosures support configurations of 6 Gbps SAS, Near Line (NL) SAS, 6 Gbps SAS Full Disk Encryption (FDE) disk drives, or a mix of disk drives. To use the FDE disk encryption functionality, you must purchase the Full Disk Encryption option. Serial number label Product name Machine type, model, and serial number label Figure 1. Serial number label, and product name, machine type, and model number label locations 2 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

advanced disaster recovery functions are available for the DS3500, including FlashCopy®, VolumeCopy, and Enhanced Remote Mirroring. The DS3500 storage subsystem comes with redundant 585-watt ac power supplies and fan units. The DS Storage Manager version 10 software is also available for the DS3500. This storage-management software is designed to help centralize storage management, simplify partitioning of the DS3500 series storage into as many as 64 virtual servers, and strategically allocate storage capacity to maximize storage space. The hot-swap features enable you to remove and replace SAS hard disk drives, power supplies, and controllers (in a dual-controller storage subsystem). EXP3500 expansion enclosure overview The EXP3500 is a 2U rack-mountable expansion enclosure with redundant 585-watt ac power supplies and fan units. The EXP3500 comes with a single environmental services module (ESMs) that is expandable to dual redundant ESMs when you install the optional ESM kit. The EXP3512 expansion enclosure supports up to 12 hard disk drives and the EXP3524 expansion enclosure supports up to 24 hard disk drives. Notices and statements in this document The caution and danger statements in this document are also in the multilingual IBM Systems Safety Notices document, which is in the Documentation folder on the IBM System Storage DS3500 Support DVD. Each statement is followed by a reference number that you can use to locate the corresponding statement in your language in the IBM Systems Safety Notices document. The following notices and statements are used in this document: v Note: These notices provide important tips, guidance, or advice. v Important: These notices provide information or advice that might help you avoid inconvenient or problem situations. v Attention: These notices indicate potential damage to programs, devices, or data. An attention notice is placed just before the instruction or situation in which damage might occur. v Caution: These statements indicate situations that can be potentially hazardous to you. A caution statement is placed just before the description of a potentially hazardous procedure step or situation. v Danger: These statements indicate situations that can be potentially lethal or extremely hazardous to you. A danger statement is placed just before the description of a potentially lethal or extremely hazardous procedure step or situation. Chapter 1. Introduction 3

Table 1 contains a summary of the features and operating specifications of the DS3500 storage subsystem and EXP3500 expansion enclosure. Depending on your model, some features might not be available, or some specifications might not apply. Table 1. Features and operating specifications General: v Modular components – High-capacity disk drives – (DS3500) RAID controller modules – (EXP3500) environmental services modules (ESMs) – Power supplies with built-in fans v Technology – Supports disk array technology – Redundant data storage, power and cooling system, (DS3500) SAS disk controllers, and (EXP3500) ESMs – Hot-swap technology for hard disk drives, power supplies, (EXP3500) ESMs, and (DS3500) controllers – (DS3500) Host port adapter. Each controller supports one host port adapter. Support for: - 6 Gbps SAS - 8 Gbps FC - 1 Gbps iSCSI v User interface – Built-in power, activity, and fault LEDs, identification labeling on components, rear LEDs, and connectors – Easy-to-replace hard disk drives, power supplies with built-in fans, and controllers Acoustical noise emissions: v DS3512 and EXP3512 fully populated with 12 hard disk drives – Sound power (idling): 6.2 bels – Sound power (operating): 6.2 bels – Sound pressure (idling): 62 dBA – Sound pressure (operating): 62 dBA v DS3524 and EXP3524 fully populated with 24 hard disk drives – Sound power (idling): 6.4 bels – Sound power (operating): 6.4 bels – Sound pressure (idling): 64 dBA – Sound pressure (operating): 64 dBA Power supply with built-in fans: v Two hot-swap 585-watt (100 - 240 V ac) standard v Provides redundant power Size: v DS3512 and EXP3512: – Height: 8.6 cm (3.4 in.) – Depth: 54.0 cm (21.3 in.) – Width: 44.9 cm (17.7 in.) – (DS3512) Weight: Single controller: 17.2 kg (38.0 lb) Dual controller: 18.5 kg (40.7 lb) – (EXP3512) Weight: 16.9 kg (37.3 lb) v DS3524 and EXP3524: – Height: 8.8 cm (3.5 in.) – Depth: 48.7 cm (19.2 in.) – Width: 44.9 cm (17.7 in.) – (DS3524) Weight: Single controller: 20.2 kg (44.5 lb) Dual controller: 21.4 kg (47.2 lb) – (EXP3524) Weight: 19.9 kg (43.8 lb) Environment: v Air temperature: – On: 10° to 35°C (50.0° to 95°F); altitude: 30.5 (100 ft) below to 3000 m (9840 ft) above sea level; temperature change: 10°C (18°F) per hour – Off: -10° to 50°C (14.0° to 120.0°F); maximum altitude: 3000 m (9840 ft); temperature change: 15°C (27.0°F) per hour v Humidity: – On: 20% to 80% – Off: 10% to 90% – Maximum dew point: 26°C (79°F) – Maximum humidity gradient: 10% per hour Hard disk drives: v DS3512 and EXP3512: 12 LFF 3.5–inch 6 Gbps v DS3524 and EXP3524: 24 SFF 2.5–inch 6 Gbps Drive type: SAS and NL SAS Controllers: (DS3500) SAS interface: Three 26-pin, mini-SAS connectors per controller ESMs: (EXP3500) SAS interface: Three 26-pin, mini-SAS connectors per ESM Heat output Approximate heat output in British thermal units (Btu) per hour: v Minimum configuration: 205 Btu (60 watts) v Maximum configuration 1235 Btu (361 watts) Electrical input: v Sine-wave input (50 - 60 Hz) required v Input voltage low range: – Minimum: 90 V ac – Maximum: 136 V ac v Input voltage high range: – Minimum: 198 V ac – Maximum: 264 V ac v Approximate input kilovolt-amperes (kVA): – Minimum: 0.06 kVA – Maximum: 0.38 kVA Notes: 1. Power consumption and heat output vary depending on the number and type of optional features that are installed and the power-management optional features that are in use. 2. These levels were measured in controlled acoustical environments according to the procedures specified by the American National Standards Institute (ANSI) S12.10 and ISO 7779 and are reported in accordance with ISO 9296. Actual sound-pressure levels in a given location might exceed the average stated values because of room reflections and other nearby noise sources. The declared sound-power levels indicate an upper limit, below which a large number of computers will operate. 4 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

The DS3500 storage subsystem RAID controller cache size, partitions, and other features vary depending the model and optional devices. Contact your IBM marketing representative or authorized reseller for more information about the models and options. Operating-system support The following operating systems are supported for host servers that have mapped LUNs that are created in the DS3500 storage subsystem with expansion enclosures: v IBM AIX v IBM Linux on POWER (LoP) v Microsoft® Windows® Server 2003 v Microsoft Windows Server 2008 v Red Hat® Enterprise Linux® v SuSE Linux Enterprise Server v VMware ESX Server For additional host operating-system support, see the latest DS Storage Manager software readme file and the IBM DS3000 series products interoperability matrix at http://www.ibm.com/systems/storage/disk/. Product updates Important: To keep your storage subsystem up-to-date with the latest firmware and other product updates, register the storage subsystem for technical support notifications. Go to http://www.ibm.com/servers/storage/support/disk/. From the menu at the top of the page, click My IBM and select My Technical Support. On the next page, click register now. Download the latest version of the DS Storage Manager software, DS3500 storage subsystem controller firmware, DS3000 series expansion enclosure firmware, and drive firmware when you initially install the storage subsystem and when product updates become available. To receive product updates, complete the following steps: 1. After you have registered, type your user ID and password to log in to the site. The “My notifications for technical support” page opens. 2. From the Subscribe tab, click Storage disk systems. 3. Click the box next to the product you want to receive information about. 4. Click Continue. 5. Under Notify me by, click e-mail. 6. Click the Submit button to complete the notifications process. Best practices guidelines To ensure optimal operation of your system, always follow these best practices guidelines: Chapter 1. Introduction 5

down. Never turn off the power if any amber LED is lit; be sure to resolve any error conditions before you shut down the storage subsystem. v Back up the data on your storage drives periodically. v To maintain power redundancy, connect the DS3500 storage subsystem and EXP3500 expansion enclosure right and left power supplies to two independent external power circuits through ac power distribution units (PDUs) inside a rack or directly into external receptacles. This ensures that all devices in the configuration have power in the event that only one power circuit is available. In addition, having all the right or all the left power cables connected to the same power circuit enables the devices in the configuration to power-on simultaneously during an unattended restoration of power. Note: Do not overload the circuits that power your storage subsystem and expansion enclosures. Use additional pairs of ac PDUs. See Table 1 on page 4 for information about storage subsystem power requirements. Contact your technical-support representative for additional information. v Before any planned system shutdown or after any system additions, removals, or modifications (including firmware updates, logical drive creations, storage partitioning definitions, hardware changes, and so on), save the storage subsystem profile as explained in the DS Storage Manager 10 Installation and Host Support Guide for your operating system. Save the profile in a location other than in the logical drives that are created for the DS3500 storage subsystem and EXP3500 expansion enclosure. v During any maintenance or attended power-on procedure, carefully follow the power-on sequence that is described in “Turning on the storage subsystem” on page 68. Make sure that each component of the storage subsystem is powered-on in the correct order during this entire power-on procedure to make sure that the controller will be able to optimally access all of the storage subsystems. v The storage subsystem supports simultaneous power-on to the system components; however, you should always follow the power-on sequence that is described in “Turning on the storage subsystem” on page 68 during any attended power-on procedure. v A storage subsystem in an optimal state should recover automatically from an unexpected shutdown and unattended simultaneous restoration of power to system components. After power is restored, call your IBM technical-support representative if any of the following conditions occur: – The storage subsystem logical drives and subsystems are not displayed in the DS Storage Manager graphical user interface. – The storage subsystem logical drives and subsystems do not come online. – The storage subsystem logical drives and subsystems seem to be degraded. DS3500 storage subsystem and EXP3500 expansion enclosure components The DS3500 storage subsystem and EXP3500 expansion enclosure have the following removable components. These components, called customer replaceable units (CRUs), are accessible from the front or rear of the storage subsystem. v (DS3512 and EXP3512) Up to 12 LFF 3.5–inch 6 Gbps SAS or NL SAS hard disk drives and drive filler panels. v (DS3524 and EXP3524) Up to 24 SFF 2.5–inch 6 Gbps SAS or NL SAS hard disk drives and drive filler panels. 6 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

installation one of the following optional host port adapters: – 6 Gbps SAS – 8 Gbps FC – 1 Gbps iSCSI v (EXP3512 and EXP3524) Up to two ESM modules. v Two power supplies. Disk drives and bezels The hot-swap drive bays that are accessible from the front of the DS3512 storage subsystem and EXP3512 expansion enclosure are shown in Figure 2. The hot-swap drive bays that are accessible from the front of the DS3524 storage subsystem and EXP3524 expansion enclosure are shown in Figure 3. Hot-swap hard disk drive You can install up to 12 hot-swap SAS or NL SAS hard disk drives in the DS3512 storage subsystem and EXP3512 expansion enclosure. You can install up to 24 hot-swap SAS or NL SAS hard disk drives in the DS3524 storage subsystem and EXP3524 expansion enclosure. Filler panel The DS3500 storage enclosure and EXP3500 expansion enclosure come with filler panels in the empty drive bays. Before you install a hard disk drive, remove the filler panel and save it for future use. Each of the drive bays must contain either a filler panel or a hard disk drive. Left-side bezel The left-side bezel contains the LEDs, as shown in the following illustration. Hot-swap hard disk drive Filler panel Bezels Figure 2. DS3512 storage subsystem and EXP3512 expansion enclosure hot-swap drive bays and bezels Figure 3. DS3524 storage subsystem and EXP3524 expansion enclosure hot-swap drive bays and bezels Chapter 1. Introduction 7

Right-side bezel and bottom shelf The right-side bezel and bottom shelf contain the hard disk drive identification information. Figure 5 shows the right-side bezel of the DS3512 storage subsystem and EXP3512 expansion enclosure. Figure 6 shows the bottom shelf of the DS3524 storage subsystem and EXP3524 expansion enclosure. The hard disk drives come preinstalled in drive trays. Install drives in the drive bays on the front of the storage subsystem or expansion enclosure. When a drive is installed, the drive and tray bay designation is set automatically. The hardware addresses are based on the enclosure ID setting on the controller and on the physical locations of the drives. Power-on LED System locator LED System error LED Overtemperature LED Standby power LED Figure 4. Left-side bezel Drives 1 - 4 Drives 5 - 8 Drives 9 - 12 Figure 5. Right-side bezel (DS3512 and EXP3512) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Drives 1 - 24 Figure 6. Bottom shelf (DS3524 and EXP3524) 8 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

its entirety (drive, bezel, and tray). When you replace a drive, be sure to order and install the correct drive. Using an unsupported drive causes the drive to be locked out by the controller firmware. Attention: 1. After you remove a drive from a bay, wait 70 seconds to allow the drive to spin down before you replace or reseat the drive. Failure to do so might cause unpredictable results. 2. Never hot-swap a drive when its associated green activity LED is flashing or its associated amber fault LED is flashing. Hot-swap a drive only when its associated amber fault LED is lit continuously or when the drive is inactive and its associated green activity LED is not flashing. Note: If the hard disk drive that you want to remove is not in a failed or bypass state, always use the DS Storage Manager software either to place the drive in a failed state or to place the array that is associated with the drive (or drives) in an offline state before you remove the drive from the enclosure. Controllers The DS3500 storage subsystem comes with one or two RAID controllers. When the DS3500 storage subsystem has two controllers, the controllers are hot-swappable and redundant. The controllers contain the storage subsystem control logic, interface ports, and LEDs. Each controller contains the following ports: v Two 6 Gbps SAS host ports v One SAS drive port to connect EXP3500 expansion enclosures v Two Ethernet ports for subsystem management v Support for one optional host port adapter: – Two-port 6 Gbps SAS – Four-port 8 Gbps FC – Four-port 1 Gbps iSCSI The Ethernet ports come with the following default IP addresses: Port 1 on controller A is 192.168.128.101 Port 2 on controller A is 192.168.129.101 Port 1 on controller B is 192.168.128.102 Port 2 on controller B is 192.168.129.102 The subnet mask for both Ethernet ports is 255.255.255.0. Attention: When a DS3500 storage subsystem has two RAID controllers, the controllers must be identical to each other in hardware (host port adapter and cache size) and firmware. If you install a host port adapter in one controller, you must install an identical host port adapter in the other controller. The storage management software automatically sets the enclosure ID for the controllers. You can change the enclosure ID setting through the DS Storage Manager software only. There are no switches on the DS3500 chassis to manually set the enclosure ID. Both controller enclosure IDs are identical under normal operating conditions. Figure 7 on page 10 shows a single-controller storage subsystem without an optional host port adapter installed. See “Cabling the DS3500 storage subsystem” on page 23 for illustrations of the storage subsystem with optional host port Chapter 1. Introduction 9

Environmental service modules The EXP3500 expansion enclosure comes with a single ESM. An optional ESM is available for redundant configurations. If the RAID controller connected to the ESM supports more than one EXP3500 expansion enclosure per physical port, you can connect two or more EXP3500 expansion enclosures by chaining them together. For information about connecting an EXP3500 expansion enclosure to a DS3500 storage subsystem, see “Connecting expansion enclosures to the DS3500” on page 40. Figure 8 shows an EXP3512 expansion enclosure containing one ESM and a filler panel over the blank ESM bay. Power supply and fans The DS3500 storage enclosure and EXP3500 expansion enclosure have two removable power supplies. Each power supply contains two fans. The four fans pull air through the drives from front to back across the drives. The fans provide redundant cooling, which means that if one fan fails, the remaining fans continue to provide sufficient cooling to operate the storage subsystem or expansion enclosure. If one power supply is turned off or malfunctions, the other power supply maintains electrical power to the storage subsystem or expansion enclosure. To preserve the optimal airflow, do not remove a failed power supply from the chassis until you are ready to replace it with a new power supply. Figure 9 on page 11 shows the power supply components. Hot-swap power supplies RAID controller A Filler panel Figure 7. Rear view, single-controller model (shown without an optional host port adapter) Hot-swap power supplies Filler panel ESM 1 2 ESM Figure 8. Rear view, single ESM model expansion enclosure 10 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

contains a sealed, rechargeable lithium ion battery that maintains data in the cache so that it can be transferred to flash memory in the event of a power failure. Figure 10 shows the locations of the battery and memory cache DIMM in the controller. The battery chargers in the power supplies perform a battery test when the storage subsystem is started and on a regularly scheduled interval thereafter. Data caching starts after the battery tests are completed. The condition of the battery is indicated by an LED on the rear of the controller (see “Controller LEDs” on page 75 for the location of the battery fault LED and conditions that the LED indicates). Software and hardware compatibility and upgrades The latest controller firmware, NVSRAM firmware, ESM firmware, and hard disk drive firmware must be installed to ensure optimal functionality, manageability, and reliability. Software and firmware support code upgrades To enable support for the DS3500, you must make sure that your system software and firmware are at the latest levels. To find the latest DS Storage Manager software, controller firmware, NVSRAM, ESM firmware, and hard disk drive firmware, check the firmware readme files. Power switch Power connector Power switch Power connector Figure 9. Power supply components Battery connector Captive fastener Memory cache battery Memory cache DIMM Host adapter or filler panel Figure 10. Battery unit Chapter 1. Introduction 11

operating system for instructions that describe how to install the DS3500 controller firmware, EXP3500 ESM firmware, and Storage Manager host software. Determining firmware levels To determine the firmware levels of the DS3500 storage subsystem, the connected expansion enclosures, and the installed hard disk drives, use the DS Storage Manager software that is used to manage the DS3500 storage subsystem. In the Subsystem Management window, click the Summary tab; then, click Storage Subsystem Profile in the Status section. When the Storage Subsystem Profile window opens, click the All tab and scroll through the Profile For Storage Subsystem to locate the following information. Note: The Profile For Storage Subsystem contains all the profile information for the entire subsystem. Therefore, you might have to scroll through a large amount of information to locate the firmware version numbers. DS3500 Storage Server v Firmware version v NVSRAM version Hard Disk Drives v Firmware version Drive Enclosure v ESM firmware version Specifications The specifications of the DS3500 storage subsystem and EXP3500 expansion enclosure are listed in Table 1 on page 4. This section provides additional site specifications for the DS3500 storage subsystem and EXP3500 expansion enclosure. Before you install the storage subsystem, you must either make sure that your planned installation site meets these requirements or prepare the site so that it does meet these requirements. Preparations might involve meeting area requirements, environmental requirements, and electrical requirements for DS3500 storage subsystem and EXP3500 expansion enclosure installation, service, and operation. Area requirements The floor space at the installation site must support the weight of the storage subsystem and associated equipment; provide sufficient space to install, operate, and service the storage subsystem; and provide sufficient ventilation to provide a free flow of air to the unit. Dimensions Figure 11 on page 13 shows the dimensions of the DS3512 storage subsystem and EXP3512 expansion enclosure, which conforms to the 19-inch rack standard. 12 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

expansion enclosure, which conforms to the 19-inch rack standard. Weight The total weight of the storage subsystem and expansion enclosure depends on the number of installed components. Table 2 lists the maximum and empty weights for the DS3512 storage subsystem and EXP3512 expansion enclosure in different configurations. Table 3 on page 14 lists the maximum and empty weights for the DS3524 storage subsystem and EXP3524 expansion enclosure in different configurations. Table 4 on page 14 lists the weight of each component. Table 2. DS3512 storage subsystem and EXP3512 expansion enclosure weights Model Maximum 1 Empty 2 DS3512 27.0 kg (59.6 lb) 8.4 kg (18.5 lb) EXP3512 26.3 kg (57.9 lb) 1 Chassis with dual controller or ESMs and 12 hard disk drives. 2 Chassis without power supplies, controllers or ESMs, and hard disk drive filler panels. 48.2 cm (19 in.) 44.9 cm (17.7 in.) 8.6 cm (3.4 in.) 54.0 cm (21.3 in.) Figure 11. DS3512 storage subsystem and EXP3512 expansion enclosure dimensions 48.2 cm (19 in.) 44.9 cm (17.7 in.) 8.8 cm (3.5 in.) 48.7 cm (19.2 in.) Figure 12. DS3524 storage subsystem and EXP3524 expansion enclosure dimensions Chapter 1. Introduction 13

Model Maximum 1 Empty 2 DS3524 25.9 kg (57.2 lb) 10.7 kg (23.6 lb) EXP3524 25.2 kg (55.5 lb) 1 Chassis with dual controller or ESMs and 24 hard disk drives. 2 Chassis without power supplies, controllers or ESMs, and hard disk drive filler panels. Table 4. DS3500 component weights Unit Weight 2.5–inch hard disk drive 0.25 kg (0.55 lb) 3.5–inch hard disk drive 0.78 kg (1.72 lb) Power supply with fan 2.50 kg (5.31 lb) Controller (including cache battery backup and no host port adapter) 2.05 kg (4.52 lb) Fibre channel host daughter card .08 kg (.18 lb) iSCSI host daughter card .05 kg (.11 lb) SAS host daughter card .07 kg (.15 lb) Battery .15 kg (.33 lb) ESM 1.75 kg (3.86 lb) Temperature and humidity Table 1 on page 4 lists the acceptable temperature and humidity ranges in which the storage subsystem is designed to operate. Notes: 1. The non-operating environment must not exceed the environment limits for longer than 60 days. 2. The storage environment must not exceed the environment limits for longer than 1 year. 3. Substantial deviations from the suggested operating range, in either direction, if sustained for extended periods of time, will expose the unit to greater risk of failure from external causes. Electrical requirements Consider the following information when you prepare the installation site: v Protective ground: Site wiring must include a protective ground connection to the ac power source. Note: Protective ground is also known as safety ground or chassis ground. v Circuit overloading: Power circuits and associated circuit breakers must provide sufficient power and overload protection. To prevent possible damage to the unit, isolate its power source from large switching loads (such as air conditioning motors, elevator motors, and factory loads). v Power failures: If a total power failure occurs, the unit automatically performs a power-on recovery sequence without operator intervention after power is restored. 14 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

The storage subsystem uses wide-ranging redundant power supplies that automatically accommodate voltages to the ac power source. The power supplies operate within the range of 90 V ac to 264 V ac, at a minimum frequency of 50 Hz and a maximum frequency of 60 Hz. The power supplies meet standard voltage requirements for operation both inside the U.S.A. and outside the U.S.A. They use standard industrial wiring with line-to-neutral or line-to-line power connections. The agency ratings for the DS3500 storage subsystem and EXP3500 expansion enclosure are 6 amps at 100 V ac and 2.5 amps at 240 V ac. These are the overall maximum operating currents for this system. AC power recovery After normal power is restored after a total ac power failure, the storage subsystem performs power-on recovery procedures automatically without operator intervention. Power cords and receptacles The storage subsystem comes with two jumper cords that are used to connect to the rack PDU. You must purchase the power cords that are applicable for use in a typical receptacle in your country. See “Power cords” on page 138 for more information. Heat output, airflow, and cooling Air enters at the front of the DS3500 storage subsystem and EXP3500 expansion enclosure and leaves at the back. Allow at least 30 inches in front of the DS3500 storage subsystem and EXP3500 expansion enclosure and at least 24 inches at the rear for service clearance, proper ventilation, and heat dissipation. When racks that contain many DS3500 storage subsystems and EXP3500 expansion enclosures are to be installed together, the following requirements must be met to ensure that the devices are adequately cooled: v Air enters at the front of the rack and leaves at the back. To prevent the air that is leaving the rack from entering the intake of another piece of equipment, you must position the racks in alternate rows, back-to-back and front-to-front. This arrangement is known as “cold aisle/hot aisle” and is shown in Figure 13 on page 16. v Where racks are in rows, each rack must touch the rack that is next to it to reduce the amount of hot air that can flow around from the back of the rack into the intakes of the expansion enclosures that are in that rack. Use Suite Attach Kits to completely seal any gaps that remain between the racks. For details about Suite Attach Kits, contact your IBM marketing representative or authorized reseller. v Where racks are in rows front-to-front or back-to-back, gaps of at least 122 cm (48 in.) must separate the rows across the cold aisle (see Figure 13 on page 16). v To correct airflow in each rack, the rack filler plates must be installed in unused positions. Also, all the gaps in the front of the racks must be sealed, including the gaps between the storage subsystems. Chapter 1. Introduction 15

cold aisle width 244 cm (96 in.) between center lines of hot and cold aisle Perforated tiles or gratings back back T42 racks T42 racks T42 racks Hot aisle Air conditioner back front front front Airflow Cold aisle Figure 13. Example of cold aisle/hot aisle rack configuration 16 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

enclosure This chapter provides instructions for installing the DS3500 storage subsystem and the EXP3500 expansion enclosure in a rack. To install the DS3500 storage subsystem in a rack, see “Installing the DS3500 storage subsystem.” To install the EXP3500 expansion enclosure in a rack, see “Installing the EXP3500 expansion enclosure” on page 21. Before you begin the installation, review the safety information in “Safety” on page xiii and “Handling static-sensitive devices” on page 19. Installing the DS3500 storage subsystem This section provides instructions for installing the DS3500 storage subsystem in a rack. Inventory checklist After you unpack the DS3500 storage subsystem, make sure that you have the following items. Depending on your order, your shipping box might contain additional items that are not on the list. v Hardware – Hard disk drives or drive filler panels: - DS3512 (12) - DS3524 (24) – RAID controllers (up to 2) – Power supplies (2) – Power cables (2 jumper line cords) – Rack installation hardware kit: - Support rails (2) (right and left assembly) - M5 black hex-head slotted screws (8) - M5 Washers (6) - Small diameter spacers (8) (these come installed, four in each rail) - Large diameter spacers (8) - M4 pan-head screws (2) Attention: The DS3500 storage subsystem does not come with region-specific power cords. You must obtain the IBM-approved power cords for your region. See “Power cords” on page 138 for the IBM-approved power cords for your region. v Software and documentation – IBM System Storage DS3500 Support DVD The support DVD contains the IBM DS Storage Manager host software. The DVD also includes firmware, online help, and the following documentation in Adobe Acrobat Portable Document Format (PDF): - IBM System Storage DS3500 and EXP3500 Installation, User's, and Maintenance Guide (this document) - DS Storage Manager 10 Installation and Host Support Guide for your operating system © Copyright IBM Corp. 2010 17

– IBM System Storage DS3500 and EXP3500 Rack Installation and Quick Start Guide Premium features activation or entitlement kits might also come inside the box, if you ordered them. If an item is missing or damaged, contact your IBM marketing representative or authorized reseller. DS3500 installation overview CAUTION: or >18 kg (39.7 lb) or 18-32 kg (39.7-70.5 lb) The weight of this part or unit is between 18 and 32 kg (39.7 and 70.5 lb). It takes two persons to safely lift this part or unit. (C009) Attention: A fully configured DS3500 storage subsystem weighs up to 27 kg (60 lb). At least two people should lift the DS3500 storage subsystem from the shipping box. You might want to open the sides of the shipping box and remove the components from the DS3500 storage subsystem before you lift it from the shipping box, to lighten the storage subsystem. The following steps summarize the installation process: 1. Review the preparation recommendations. See “Preparing for installation” on page 19. 2. Prepare the installation site. See “Preparing the site” on page 20. 3. Prepare the rack. See the DS3500 and EXP3500 Rack Installation and Quick Start Guide. 4. Record the serial number, machine type and model number, and RAID controller MAC addresses for your DS3500 storage subsystem in Appendix A, “Records,” on page 141. See Figure 1 on page 2 for the location of the serial number. The MAC addresses are labeled near the Ethernet port on each RAID controller. 5. Install and secure the DS3500 chassis and components in the rack. See the DS3500 and EXP3500 Rack Installation and Quick Start Guide. 6. Install the expansion enclosure in the rack, if you are installing one or more expansion enclosures. Follow the instructions in the DS3500 and EXP3500 Rack Installation and Quick Start Guide. 7. Cable the DS3500 storage subsystem to the expansion enclosures. See “Connecting expansion enclosures to the DS3500” on page 40. 8. Complete one of the following cabling tasks to enable management of the DS3500 configuration: v If you are using out-of-band management, cable the DS3500 Ethernet ports to either the management workstation or the host. 18 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

bus adapters (HBAs) in the hosts. See “Connecting iSCSI hosts to the DS3500” on page 59. 9. Connect the power cables. See “Cabling the DS3500 and EXP3500 power supplies” on page 65. 10. Power-on the expansion enclosure and the DS3500 storage subsystem, using the procedure in “Turning on the storage subsystem” on page 68. 11. Install the DS Storage Manager host software on the management workstation (for out-of-band management) or on the host (for in-band management). See the DS Storage Manager 10 Installation and Host Support Guide for the management-workstation or host operating system for installation instructions for installing the DS Storage Manager host software. 12. Use the DS Storage Manager host software to verify the configuration. 13. Review and perform the procedures in “Performing the health check process” on page 67. Handling static-sensitive devices Attention: Static electricity can damage the storage subsystem and other electronic devices. To avoid damage, keep static-sensitive devices in their static-protective packages until you are ready to install them. To reduce the possibility of electrostatic discharge, observe the following precautions: v Limit your movement. Movement can cause static electricity to build up around you. v Handle the device carefully, holding it by its edges or its frame. v Do not touch solder joints, pins, or exposed printed circuitry. v Do not leave the device where others can handle and damage it. v While the device is still in its static-protective package, touch it to an unpainted metal part of the system unit for at least 2 seconds. This drains static electricity from the package and from your body. v Remove the device from its package and install it directly into your system unit without setting it down. If it is necessary to set down the device, put it back into its static-protective package. Do not place the device on your system unit cover or on a metal surface. v Take additional care when you handle devices during cold weather. Heating reduces indoor humidity and increases static electricity. Preparing for installation Before you install the DS3500 storage subsystem, create a detailed plan of how the device will be used in your storage configuration. The plan should include determining RAID levels, failover requirements, operating systems that are to be used, and total storage capacity requirements. To prepare the DS3500 storage subsystem for installation in a rack, complete the following steps: 1. Prepare the site to meet all area, environmental, power, and site requirements. For more information, see “Specifications” on page 12. 2. Move the shipping box that contains the DS3500 to the site. Chapter 2. Installing the storage subsystem and expansion enclosure 19

or 18-32 kg (39.7-70.5 lb) The weight of this part or unit is between 18 and 32 kg (39.7 and 70.5 lb). It takes two persons to safely lift this part or unit. (C009) 3. Make sure that you have the correct host software for your operating system. The support DVD that comes with the DS3500 has the correct DS Storage Manager host software. The DVD also includes the storage subsystem controller firmware. For the latest controller firmware, see http://www.ibm.com/servers/storage/support/disk/. 4. Read the applicable readme files that are included in the DS Storage Manager host software or DS3500 controller firmware packages for any updated information about hardware, software, or firmware products. 5. Continue with “Required tools and hardware.” Required tools and hardware Have the following tools and equipment available: v Region-specific power cords v 5/16 (8 mm) hex nut driver v #0 and #1 Phillips screwdrivers v Electrostatic-discharge protection (such as a grounding wrist strap) v Rack power jumper cords v Rack-mounting hardware v Ethernet interface cables and cable straps (depending on configuration) v SAS cables (depending on configuration) v Fibre Channel cables, interface cables, and cable straps (depending on configuration) v SFP modules (depending on configuration) Preparing the site This section lists the floor space requirements and weight information for the DS3500. For information about interface cables and connections, see Chapter 3, “Cabling the storage subsystem and expansion enclosure,” on page 23. The floor area at the installation site must provide the following conditions: v Sufficient space to install the DS3500 storage subsystem v Enough stability to support the weight of the fully configured DS3500 storage subsystem and associated devices (see “Weight” on page 13). Make sure that all requirements, such as floor space, air conditioning, and electrical service, have been met. Other site preparation activities include the following tasks: v Make sure that there is enough room to move around the rack and install the modules. v Install uninterruptible-power-supply devices. 20 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

other devices. v Route interface cables from the HBA ports in the hosts or switches to the installation area. v Route main power cords to the installation area. Continue with “Installing the DS3500 storage subsystem in a rack.” Installing the DS3500 storage subsystem in a rack To install the DS3500 in a rack, follow the instructions in the DS3500 and EXP3500 Rack Installation and Quick Start Guide. Then, continue with Chapter 3, “Cabling the storage subsystem and expansion enclosure,” on page 23. Installing the EXP3500 expansion enclosure This section provides instructions for installing the EXP3500 expansion enclosure in a rack. The EXP3500 connects to a DS3500 storage subsystem. Inventory checklist After you unpack the EXP3500 expansion enclosure, make sure that you have the following items. Depending on your order, your shipping box might contain additional materials that are not in the following list. v Hardware: – Hard disk drives or drive filler panels: - EXP3512 (12) - EXP3524 (24) – ESMs (up to 2) – Power supplies (2) – Power cables (2 jumper line cords) – Rack installation hardware kit: - Support rails (2) (right and left assembly) - M5 black hex-head slotted screws (8) - M5 Washers (6) - Small diameter spacers (8) (these come installed, four in each rail) - Large diameter spacers (8) - M4 pan-head screws (2) v Printed documents: – IBM System Storage DS3500 and EXP3500 Rack Installation and Quick Start Guide v Online documents: – IBM System Storage DS3500 and EXP3500 Installation, User's, and Maintenance Guide (this document) – IBM Systems Safety Notices All documents are available on the IBM Documentation CD or from the IBM support Web site at http://www.ibm.com/servers/storage/support/. Chapter 2. Installing the storage subsystem and expansion enclosure 21

You can install the EXP3500 storage subsystem in an Electronic Industries Association (EIA) 310 standard rack. For complete rack installation instructions, see IBM System Storage DS3500 and EXP3500 Rack Installation and Quick Start Guide. 22 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

enclosure This chapter provides information and instructions for cabling the DS3500 storage subsystem and the EXP3500 expansion enclosure. To cable the DS3500 storage subsystem, see “Cabling the DS3500 storage subsystem.” To cable the EXP3500 expansion enclosure, see “Cabling the EXP3500 expansion enclosure” on page 37. Cabling the DS3500 storage subsystem The section provides information and instructions for cabling the DS3500 storage subsystem. After the storage subsystem is installed in its permanent location, you must cable it to hosts, drives, and other external devices, depending on your hardware configuration. Controller connectors (with Fibre Channel host port adapters) Figure 14 shows the optional Fibre Channel host port adapter installed in both controllers at the rear of the DS3500 storage subsystem. Fibre Channel 1, Fibre Channel 2 Each Fibre Channel host port supports a small-form-factor pluggable (SFP) transceiver and is capable of operating at 8 Gbps, 4 Gbps, or 2 Gbps. Drive expansion port The drive expansion port is a x4 multilane SAS port. Connect a SAS cable to this port and to a drive expansion enclosure. Controller connectors (with iSCSI host port adapters) Figure 15 on page 24 shows the optional iSCSI host port adapter installed in both controllers at the rear of the DS3500 storage subsystem. Serial port Ethernet ports SAS host port 1 SAS host port 2 Fibre channel host port 3 Controller A Fibre channel host port 4 Fibre channel host port 5 Fibre channel host port 6 Drive expansion port Seven-segment numeric display Controller B Figure 14. Dual-controller DS3500 storage subsystem ports and controllers (with optional Fibre Channel host port adapter) © Copyright IBM Corp. 2010 23

The iSCSI host ports are auto-negotiated 100/1000 Mbps Ethernet ports that support iSCSI off load. The host ports also provide support for both Internet Protocol version 4 (IPv4) and IPv6, and have the advanced networking functions of jumbo frames, IEEE 802.1p, and virtual local area network (VLAN) support. Drive expansion port The drive expansion port is a x4 multilane SAS port. Connect a SAS cable to this port and to a drive expansion enclosure. Controller connectors (with SAS host port adapters) Figure 16 shows the optional SAS host port adapter installed in both controllers at the rear of the DS3500 storage subsystem. Host port 1, Host port 2, and Host port 3 The DS3500 host ports are each a x4 multilane, 6 Gbps universal mini-SAS port. Connect a SAS cable from your host system SAS host bus adapter to a host port on each controller. Drive expansion port The drive expansion port is a x4 multilane SAS port. Connect a SAS cable to this port and to a drive expansion enclosure. Serial port Ethernet ports SAS host port 1 SAS host port 2 iSCSI host port 3 Controller A iSCSI host port 4 iSCSI host port 5 iSCSI host port 6 Drive expansion port Controller B Seven-segment numeric display Figure 15. Dual-controller DS3500 storage subsystem ports and controllers (with optional iSCSI host port adapter) Serial port Ethernet ports SAS host port 1 SAS host port 2 SAS host port 3 Controller A SAS host port 4 Drive expansion port Seven-segment numeric display Controller B Figure 16. Dual-controller DS3500 storage subsystem ports and controllers (with an optional SAS host port adapter) 24 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

The enclosure ID is a unique two-digit identifier for each enclosure in the storage subsystem configuration. In a dual controller or ESM configuration, both enclosure IDs are identical under normal operating conditions. Each EXP3500 storage expansion enclosure and DS3500 storage subsystem in the storage subsystem configuration must have a unique storage enclosure ID. The controller automatically sets the enclosure ID. You can change the setting through the DS Storage Manager software, if necessary. The range of supported enclosure ID settings is 0 through 99. The enclosure ID is normally set to a value of 00 at the factory. The enclosure ID is displayed on the seven-segment numeric display located on the back of each controller and ESM. Working with SAS cables Each DS3500 RAID controller has up to four x4 multilane SAS host ports and a single x4 multilane SAS port for drive channel connections. Use a 1M or 3M (1-meter or 3-meter) SAS cable with a mini-SAS 4x multilane SAS connector on each end to connect a controller host port to a host HBA and to connect the drive expansion port to an expansion enclosure. The following illustration shows the 1M and 3M mini-SAS cable. The 1M and 3M SAS cables have a universal key connector that enables the cable to be used in all mini-SAS ports. Mini-SAS connectors Figure 17. Mini-SAS cable Chapter 3. Cabling the storage subsystem and expansion enclosure 25

precautions: v When you route the cable along a folding cable-management arm, leave enough slack in the cable. v Route the cable away from places where it can be damaged by other devices in the rack. v Do not put excess weight on the cable at the connection point. Make sure that the cable is well supported. To connect a mini-SAS cable, insert the mini-SAS connector into a mini-SAS port. Make sure that it locks into place. To remove a mini-SAS cable, complete the following steps: 1. Put one finger into the hole on the blue plastic tab on the mini-SAS connector and gently pull on the tab to release the locking mechanism. 2. As you pull on the tab, pull out the connector to remove it from the port. Figure 18. Connecting a mini-SAS cable Blue plastic tab Figure 19. Removing a mini-SAS cable 26 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

Each RAID controller can have up to four Fibre Channel host ports. You use a small-form-factor pluggable (SFP) module to connect a host port to a host. The SFP module is inserted into the port, and then a fiber optic cable is inserted into the SFP module. The other end of the fiber optic cable connects to an optical interface connector in a Fibre Channel HBA on a host. SFP modules are laser products. CAUTION: This product might contain one or more of the following devices: CD-ROM drive, DVD-ROM drive, DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information: v Do not remove the covers. Removing the covers of the laser product could result in exposure to hazardous laser radiation. There are no serviceable parts inside the device. v Use of the controls or adjustments or performance of procedures other than those specified herein might result in hazardous radiation exposure. (C026) CAUTION: Data processing environments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical fiber cable or open receptacle. (C027) Handling fiber optic cables Attention: To avoid damage to the fiber optic cables, follow these guidelines: v Do not route the cable along a folding cable-management arm. v For devices on slide rails, leave enough slack in the cables so they do not bend to a diameter of less than 76 mm (3 in.), or a radius less than 38 mm (1.5 in.), when extended or become pinched when retracted. v Route the cable away from places where it can be damaged by other devices in the rack cabinet. v Do not use plastic cable ties in place of the provided cable straps. v Do not overtighten the cable straps or bend the cables to a diameter of less than 76 mm (3 in.), or a radius less than 38 mm (1.5 in.). v Do not put excess weight on the cable at the connection point. Be sure that the cable is well supported. v The following are the recommended maximum cable lengths. – 2 Gbps: 300 m (984 ft.) of 50/125 um fiber, 150 m (492 ft.) of 62.5/125 um fiber – 4 Gbps: 150 m (492 ft.) of 50/125 um fiber, 70 m (230 ft.) of 62.5/125 um fiber – 8 Gbps: 50 m (164 ft.) of 50/125 um fiber, 35 m (115 ft.) of 62.5/125 um fiber v The following are the recommended maximum lengths of OM2 category Fibre Channel cable. Chapter 3. Cabling the storage subsystem and expansion enclosure 27

fiber – 8 Gbps: 50 m (164 ft.) of 50/125 um fiber, 150 m (492 ft.) of 62.5/125 um fiber Installing SFP modules The storage subsystem requires SFP modules. SFP modules convert electrical signals to optical signals that are required for Fibre Channel transmission to and from RAID controllers. After you install the SFP modules, you use fiber optic cables to connect the storage subsystem to other Fibre Channel devices. Before installing SFP modules and fiber optic cables, read the following information: v Do not mix long-wave SFP modules and short-wave SFP modules on a single storage subsystem. Use only short-wave SFP modules. You can use the DS Storage Manager Client to view the Storage Subsystem Profile to make sure that you are not mixing long-wave and short-wave SFP modules. Attention: Do not use long-wave SFP modules or Gigabit Interface Converters (GBICs) in any of the Fibre Channel ports in the storage subsystem controllers. (Long-wave SFP modules and GBICs are not supported for use in the storage subsystem or any of the attached expansion enclosures. Long-wave SFP modules and GBICs are supported only in the Fibre Channel switch ports to which the storage subsystem connects.) v The SFP module housing has an integral guide key that is designed to prevent you from inserting the SFP module incorrectly. v Use minimal pressure when inserting an SFP module into a Fibre Channel port. Forcing the SFP module into a port could cause damage to the SFP module or to the port. v You can insert or remove the SFP module while the port is powered-on. v The operational or redundant loop performance is not affected when you install or remove an SFP module. v You must insert the SFP module into a port before you connect the fiber optic cable. v You must remove the fiber optic cable from the SFP module before you remove the SFP module from the port. See “Removing SFP modules” on page 30 for more information. v The speed of the SFP module determines the maximum operating speed of the Fibre Channel port in which the SFP module is installed. For example, a 2-Gbps SFP module that is connected to a 4-Gbps-capable port will limit the speed of that port to a maximum of 2 Gbps. Attention: Carefully check the IBM part number, option number, and FRU part number of the SFP module to identify its speed. There are no physical features that distinguish an 8 Gbps, 4 Gbps, or 2 Gbps SFP module from each other. 28 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

drive, DVD-ROM drive, DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information: v Do not remove the covers. Removing the covers of the laser product could result in exposure to hazardous laser radiation. There are no serviceable parts inside the device. v Use of the controls or adjustments or performance of procedures other than those specified herein might result in hazardous radiation exposure. (C026) CAUTION: Data processing environments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical fiber cable or open receptacle. (C027) Attention: When you handle static-sensitive devices, take precautions to avoid damage from static electricity. For details about handling static-sensitive devices, see “Handling static-sensitive devices” on page 19. To install an SFP module, complete the following steps: 1. Remove the SFP module from its static-protective package. 2. Remove the protective cap from the SFP module, as shown in Figure 20. Save the protective cap for future use. 3. Remove the protective cap from the SFP port. Save the protective cap for future use. 4. Insert the SFP module into the host port until it clicks into place. See Figure 21 on page 30. Protective cap SFP module Plastic tab Figure 20. SFP module and protective cap Chapter 3. Cabling the storage subsystem and expansion enclosure 29

see “Using LC-LC Fibre Channel cables” on page 31. Removing SFP modules To remove the SFP module from the host port, complete the following steps: Attention: To avoid damage to the cable or to the SFP module, make sure that you disconnect the LC-LC Fibre Channel cable before you remove the SFP module. 1. Remove the LC-LC Fibre Channel cable from the SFP module. For more information, see “Removing an LC-LC Fibre Channel cable” on page 33. 2. Unlock the SFP module latch: v For SFP modules that contain plastic tabs, unlock the SFP module latch by pulling the plastic tab outward 10°, as shown in Figure 22. v For SFP modules that contain wire tabs, unlock the SFP module latch by pulling the wire latch outward 90°, as shown in Figure 23 on page 31. SFP module Fiber optic cable Figure 21. Installing an SFP module into the host port Protective cap SFP module 10 Plastic tab Figure 22. Unlocking the SFP module latch - plastic variety 30 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

v For SFP modules that contain plastic tabs, slide the SFP module out of the port. v For SFP modules that contain wire tabs, grasp the wire latch and pull the SFP module out of the port. 4. Replace the protective cap on the SFP module. 5. Place the SFP module into a static-protective package. 6. Replace the protective cap on the host port. Using LC-LC Fibre Channel cables The LC-LC Fibre Channel cable is a fiber optic cable that you use to connect Fibre Channel ports to one of the following devices: v An SFP module installed in a Fibre Channel switch port v A Fibre Channel host bus adapter port (optical interface connector) See Figure 24 for an illustration of the LC-LC Fibre Channel cable. For more information, see the documentation that comes with the LC-LC Fibre Channel cable. Connecting an LC-LC cable to an SFP module: To connect an LC-LC Fibre Channel cable to an SFP module, complete the following steps. Protective cap SFP module 90 Wire tab Figure 23. Unlocking the SFP module latch - wire variety Figure 24. LC-LC Fibre Channel cable Chapter 3. Cabling the storage subsystem and expansion enclosure 31

drive, DVD-ROM drive, DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information: v Do not remove the covers. Removing the covers of the laser product could result in exposure to hazardous laser radiation. There are no serviceable parts inside the device. v Use of the controls or adjustments or performance of procedures other than those specified herein might result in hazardous radiation exposure. (C026) CAUTION: Data processing environments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical fiber cable or open receptacle. (C027) 1. Read the information in “Handling fiber optic cables” on page 27. 2. If necessary, remove the protective cap from the SFP module, as shown in Figure 20 on page 29. Save the protective cap for future use. 3. Remove the two protective caps from one end of the LC-LC cable, as shown in Figure 25. Save the protective caps for future use. 4. Carefully insert this end of the LC-LC cable into an SFP module that is installed in the storage subsystem. The cable connector is keyed for correct installation. Holding the connector, push in the cable until it clicks into place, as shown in Figure 26 on page 33. Protective cap Fiber-optic cable Figure 25. Removing fiber optic cable protective caps 32 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

the protective caps for future use. 6. Connect this end of the LC-LC cable to one of the following devices: v An SFP module that is installed in a Fibre Channel switch port v A Fibre Channel host bus adapter port Removing an LC-LC Fibre Channel cable: To remove an LC-LC Fibre Channel cable, complete the following steps. Attention: To avoid damaging the LC-LC cable or SFP module, make sure that you observe the following precautions: v Press and hold the lever to release the latches before you remove the cable from the SFP module. v Make sure that the levers are in the released position when you remove the cable. v Do not grasp the SFP module plastic tab when you remove the cable. 1. On the end of the LC-LC cable that connects into the SFP module or host bus adapter, press down and hold the lever to release the latches, as shown in Figure 27. Lever Latches 2. While pressing down the cable lever, carefully pull on the connector to remove the cable from the SFP module, as shown in Figure 28 on page 34. Figure 26. Inserting an LC-LC Fibre Channel cable into an SFP module Figure 27. LC-LC Fibre Channel cable lever and latches Chapter 3. Cabling the storage subsystem and expansion enclosure 33

4. Replace the protective cap on the SFP module. Using LC-SC Fibre Channel cable adapters The LC-SC Fibre Channel cable adapter is a fiber optic cable that is used to connect an LC connector to one of the following devices that require SC connectors: v 1-Gbps Fibre Channel switch v Fibre Channel host bus adapter For more information, see the documentation that comes with the LC-SC Fibre Channel cable adapter. The following sections provide the procedures for correctly connecting and removing an LC-SC Fibre Channel cable. Connecting an LC-SC cable adapter to a device: To connect an LC-SC Fibre Channel cable adapter to a device, complete the following steps. Figure 28. Removing the LC-LC Fibre Channel cable LC connector SC connector Figure 29. LC-SC Fibre Channel cable adapter 34 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

drive, DVD-ROM drive, DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information: v Do not remove the covers. Removing the covers of the laser product could result in exposure to hazardous laser radiation. There are no serviceable parts inside the device. v Use of the controls or adjustments or performance of procedures other than those specified herein might result in hazardous radiation exposure. (C026) CAUTION: Data processing environments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical fiber cable or open receptacle. (C027) 1. Read the information in “Handling fiber optic cables” on page 27. 2. Connect one end of an LC-LC cable to an SFP module in the DS3500. For instructions, see “Using LC-LC Fibre Channel cables” on page 31. 3. Remove the two protective caps from the LC connector end of the LC-SC cable adapter as shown in Figure 30. Save the protective caps for future use. 4. Carefully insert the other end of the LC-LC cable into the LC connector end of the LC-SC cable adapter as shown in Figure 31 on page 36. Push in the connector until it clicks into place. Figure 30. Removing the LC-SC cable adapter protective caps Chapter 3. Cabling the storage subsystem and expansion enclosure 35

or Fibre Channel host bus adapter, connect the SC connector end of the LC-SC cable adapter to a Gigabit Interface Converter (GBIC) that is installed in the 1-Gbps Fibre Channel switch or Fibre Channel host bus adapter. For more information about connecting to these devices, see the documentation that comes with the device. Removing an LC-LC cable from an LC-SC cable adapter: To remove an LC-LC cable from an LC-SC cable adapter, complete the following steps. Attention: To avoid damaging the LC-LC cable, make sure you press and hold the lever to release the latches before you remove the cable from an LC-SC cable adapter. that both levers are in the released position when removing the cable. When removing the cable from the SFP module, make sure you do not grasp the SFP module plastic tab. 1. On the end of the cable that connects into the LC connector end of the LC-SC cable adapter, press down and hold the lever to release the latches. Figure 32 shows the location of the lever and latches. 2. Carefully pull on the connector to remove it. Make sure you grasp the connector and not the cable when removing the LC-LC cable from the LC-SC cable adapter as shown in Figure 33 on page 37. Figure 31. Connecting an LC-LC cable into the LC-SC cable adapter Lever Latches Figure 32. LC-LC Fibre Channel cable lever and latches 36 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

Cabling the EXP3500 expansion enclosure The EXP3500 expansion enclosure comes with one ESM, which enables you to connect the EXP3500 expansion enclosure to a DS3500 storage subsystem. If you are attaching the EXP3500 to a dual controller DS3500 storage subsystem, you must add a second ESM to the EXP3500 expansion enclosure to provide a redundant drive path. ESM connectors Figure 34 shows the connectors on the ESM. SAS In 1 Connect a SAS cable to this connector and to either an IBM SAS RAID controller or the SAS Out (↓) connector of another EXP3500. SAS In 2 Connect a SAS cable to this connector and to either an IBM SAS RAID controller or the SAS Out (↓) connector of another EXP3500. SAS Out Connect a SAS cable to this connector and to the SAS In (↑) connector of another EXP3500. Note: Each ESM contains two SAS In connectors. You can use either SAS In connector but only one SAS In connector can be used at a time. Single-ESM configuration The DS3500 RAID controller supports more than one EXP3500 expansion enclosure per drive expansion port. You can connect two or more EXP3500 Figure 33. Removing the LC-LC Fibre Channel cable from an LC-SC Fibre Channel cable adapter 1 2 ESM SAS in connector 1 SAS in connector 2 SAS out connector Figure 34. ESM connectors Chapter 3. Cabling the storage subsystem and expansion enclosure 37

one or more EXP3500 expansion enclosures that have one ESM each, complete the following steps: 1. Connect one EXP3500 expansion enclosure to the RAID controller: a. Connect one end of a SAS cable to the drive expansion port on the RAID controller. b. Connect the other end to one of the In (↑) SAS connectors on the ESM in the EXP3500 expansion enclosure. 2. Connect a second EXP3500 expansion enclosure to the first EXP3500 expansion enclosure: a. Connect one end of a SAS cable to the Out (↓) SAS connector on the ESM of the EXP3500 expansion enclosure that you just connected. b. Connect the other end of the SAS cable to one of the In (↑) SAS connectors on the ESM on the next EXP3500 expansion enclosure. c. Repeat steps 2a and 2b for each EXP3500 expansion enclosure that you add. Dual-ESM configuration The EXP3500 expansion enclosure comes with one ESM. If your DS3500 storage subsystem is a dual RAID controller configuration, it supports redundant drive paths. You must install a second ESM. See “Installing an additional ESM” on page 123 before attaching any EXP3500 expansion enclosures. Dual-ESM configuration with a single EXP3500 expansion enclosure: To connect RAID controllers A and B to one EXP3500 expansion enclosure that has two ESMs, complete the following steps: 1. Connect the EXP3500 expansion enclosure to RAID controller A: 38 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

b. Connect the other end of the SAS cable to one of the In (↑) SAS connectors on the left ESM in the EXP3500 expansion enclosure. 2. Connect the EXP3500 expansion enclosure to RAID controller B: a. Connect one end of a SAS cable to RAID controller B. b. Connect the other end of the SAS cable to one of the In (↑) SAS connectors on the right ESM in the EXP3500 expansion enclosure. Dual-ESM configuration with two or more EXP3500 expansion enclosures: The DS3500 controllers support more than one EXP3500 expansion enclosure per physical port, so multiple EXP3500 expansion enclosures can be connected by chaining them together. To connect DS3500 RAID controllers A and B to multiple EXP3500 expansion enclosures that have two ESMs each, complete the following steps: 1. Connect one EXP3500 expansion enclosure to RAID controller A: a. Connect one end of a SAS cable to the drive expansion port on the DS3500 RAID controller A. b. Connect the other end of the SAS cable to one of the In (↑) SAS connectors on the left ESM in the EXP3500 expansion enclosure. 2. Connect the left ESM of the EXP3500 expansion enclosure to the next EXP3500 expansion enclosure in the chain: a. Connect one end of a SAS cable to the Out (↓) SAS connector on the left ESM of the EXP3500 expansion enclosure that you just connected. b. Connect the other end of the SAS cable to one of the In (↑) SAS connectors on the left ESM in the next EXP3500 expansion enclosure in the chain. c. Repeat steps 2a and 2b for each EXP3500 expansion enclosure that you add. 3. Connect the last EXP3500 expansion enclosure in the chain to RAID controller B: a. Connect one end of a SAS cable to the drive expansion port on the DS3500 RAID controller B. Chapter 3. Cabling the storage subsystem and expansion enclosure 39

on the right ESM in the last EXP3500 expansion enclosure of the chain that you created. 4. In reverse order (last in chain to first in chain), connect the right ESMs in the EXP3500 expansion enclosures in the chain: a. Connect one end of a SAS cable to the Out (↓) SAS connector on the right ESM of the EXP3500 expansion enclosure that you just connected. b. Connect the other end of the SAS cable to one of the In (↑) SAS connectors on the right ESM of the previous EXP3500 expansion enclosure in the chain. c. Repeat steps 4a and 4b until you connect the right ESM of the first EXP3500 expansion enclosure in step 2 on page 39 to the right ESM of the EXP3500 expansion enclosure in step 1 on page 39. Connecting expansion enclosures to the DS3500 The DS3500 supports the attachment of either the EXP3512 or EXP3524 expansion enclosure. The DS3500 supports a total of 96 hard disk drives, so the total number of expansion enclosures supported varies depending on the model of DS3500 storage subsystem and EXP3500 expansion enclosure. Table 5 shows the maximum number of EXP3512 and EXP3524 expansion enclosures that can be attached to either a DS3512 or a DS3524 storage subsystem and not exceed the 96 total disk drives. Any combination of EXP3512 or EXP3524 expansion enclosures can be attached to a DS3512 or a DS3524 as long as the total number of disk drives does not exceed 96 disk drives. See the Hard Disk Drive and Storage Expansion Enclosure Installation and Migration Guide for more information. Table 5. Maximum number of EXP3512 and EXP3524 expansion enclosures DS3512 DS3524 Expansion units Quantity Expansion units Quantity EXP3512 7 EXP3524 6 EXP3524 0 EXP3512 0 or or EXP3512 5 EXP3524 4 EXP3524 1 EXP3512 1 or or EXP3512 3 EXP3524 2 EXP3524 2 EXP3512 2 or or EXP3512 1 EXP3524 0 EXP3524 3 EXP3512 3 Redundant drive channel pair Each RAID controller on the DS3500 has a drive expansion channel that contains an x4 SAS port. The expansion enclosures that are connected to this connector form a drive channel. The maximum number of hard disk drives that can be installed in one drive channel is 96. In a dual-controller DS3500, one drive channel from each controller combines to form a redundant drive-channel pair. 40 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

the drive channel fails, the RAID controllers can still access the expansion enclosures in the redundant drive-channel pair. Note: In expansion enclosures that are connected to a dual-controller storage subsystem, you must install the optional second ESM to support dual redundant drive paths. Overview of steps to connect expansion enclosures to a storage subsystem To connect expansion enclosures to the storage subsystem, complete the following steps: 1. Follow the instructions in the IBM System Storage DS3500 and EXP3500 Rack Installation and Quick Start Guide for your expansion enclosure and the rack installation instructions to set up and mount the expansion enclosures. 2. Select a cabling topology applicable to the number of ESMs in the expansion enclosures that you will connect to the DS3500. “DS3500 storage subsystem drive cabling topologies” on page 42 describes the recommended schemes for cabling expansion enclosures with one ESM or two ESMs to the DS3500 and to each other (if you are connecting more than one expansion enclosure). 3. Follow the cabling diagram for your chosen topology. 4. If required, set unique enclosure IDs for all expansion enclosures that are cabled to the DS3500. See the DS Storage Manager 10 Installation and Host Support Guide for information about setting the enclosure ID. The DS3500 storage subsystem locates the drives in the expansion enclosures after you power-on the configuration. Always turn on the expansion enclosures first and then turn on the DS3500. After you have powered-on the configuration, use the DS Storage Manager software to check the status of the new drives, correct any errors, and configure the new drives. Figure 35. Example of a redundant drive path Chapter 3. Cabling the storage subsystem and expansion enclosure 41

This section describes the following preferred cabling topologies for cabling expansion enclosures to the DS3500 storage subsystem: v “One single-controller DS3500 and one or more expansion enclosures” v “One dual-controller DS3500 and one expansion enclosure” v “One dual-controller DS3500 and two expansion enclosures” on page 43 v “One dual-controller DS3500 and up to eight expansion enclosures” on page 43 If one of these examples is suitable for your hardware and application, complete the cabling connections as shown in the illustrations. If you have hardware other than what is shown in these examples to include in your topology, use these examples as a starting point for creating your specific topology. Note: In expansion enclosures that are connected to a dual-controller storage subsystem, you must install the optional second ESM to support dual redundant drive paths. One single-controller DS3500 and one or more expansion enclosures: To cable one single-controller DS3500 to one or more single-ESM expansion enclosures, connect them as shown in Figure 36. Note: In expansion enclosures that are connected to a dual-controller storage subsystem, you must install the optional second ESM to support dual redundant drive paths, and must use one of the dual-controller topologies that are described in this section. One dual-controller DS3500 and one expansion enclosure: To cable one dual-controller DS3500 to one expansion enclosure, connect them as shown in Figure 36. One single-controller DS3500 and multiple single-ESM expansion enclosures 42 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

To cable one dual-controller DS3500 and two expansion enclosures, connect them as shown in Figure 38. One dual-controller DS3500 and up to eight expansion enclosures: To cable one dual-controller DS3500 and up to eight expansion enclosures, connect them as shown in Figure 39 on page 44. Figure 37. One dual-controller DS3500 and one expansion enclosure Figure 38. One dual-controller DS3500 and two expansion enclosures Chapter 3. Cabling the storage subsystem and expansion enclosure 43

configuration The following procedures provide instructions for adding expansion enclosures to a running dual-controller DS3500 storage subsystem. Note: Make sure that each expansion enclosure has two ESMs before you connect it to a dual-controller DS3500 storage subsystem. The second ESM must be purchased separately. Contact your IBM marketing representative or authorized reseller for more information about purchasing a second ESM. For more information about installing a second ESM, see the Installation Guide that comes with the expansion enclosure. To add an expansion enclosure to a dual-controller DS3500 storage subsystem that is running, complete the following steps: Figure 39. One dual-controller DS3500 and three expansion enclosures 44 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

Storage Manager software. 2. Use the expansion enclosure Rack Installation Instructions document to install the new expansion enclosure in the rack. 3. Connect the power cables to the new expansion enclosure. 4. Power-on the expansion enclosure. 5. Connect one end of a Serial Attached SCSI (SAS) cable to the Drive expansion port on Controller A in the DS3500 storage subsystem. 6. Connect the other end of the SAS cable to either of the In ports on the left ESM in the expansion enclosure. 7. Connect one end of a second SAS cable to the Drive expansion port on Controller B in the DS3500 storage subsystem. 8. Connect the other end of the second SAS cable to the In port on the right ESM in the expansion enclosure. 9. Wait until the storage subsystem status is Optimal in the DS Storage Manager software before proceeding to step 10. 10. Add the hard disk drives to the expansion enclosure. Make sure that you wait at least 30 seconds for the DS Storage Manager software to recognize each newly added drive before you install another drive. To add additional expansion enclosures after the first expansion unit is connected to a dual-controller DS3500 storage subsystem that is running, complete the following steps: 1. Make sure that the storage subsystem status is Optimal in the DS Storage Manager software. 2. Use the expansion enclosure Rack Installation Instructions document to install the new expansion enclosure in the rack. 3. Connect the power cables to the new expansion enclosure. 4. Power on the expansion enclosure. 5. Connect one end of a Serial Attached SCSI (SAS) cable to the In port on the left ESM in the expansion enclosure that is being added to the configuration. 6. Connect the other end of the SAS cable to the Out port on the left ESM in the last expansion enclosure in the configuration (this expansion enclosure is directly connected to the Drive expansion port on controller A of the DS3500). 7. On the last expansion enclosure in the configuration, disconnect the SAS cable from the In port on the right ESM and connect it to the In port on the right ESM on the new expansion enclosure that is being added to the configuration. Note: When the SAS cable is disconnected in step 7, a loss of path redundancy occurs and an error message is displayed in the DS Storage Manager Recovery Guru. Disregard the error message. The redundant path is restored in step 8 after the SAS cable is connected. 8. Connect a SAS cable to the Out port on the right ESM of the new expansion enclosure and connect the other end to either of the In port on the right ESM in the expansion enclosure that was previously connected to controller B of the DS3500. 9. Wait until the storage subsystem status is Optimal in the DS Storage Manager software before proceeding to step 10. 10. Add the hard disk drives to the expansion enclosure. Make sure that you wait at least 30 seconds for the DS Storage Manager software to recognize each newly added drive before you install another drive. Chapter 3. Cabling the storage subsystem and expansion enclosure 45

attached EXP3500 enclosure If you are upgrading a single-controller DS3500 storage subsystem to a dual-controller DS3500 storage subsystem and there are no EXP3500 expansion enclosures attached to the DS3500 storage subsystem, see “Installing a controller” on page 95. Upgrading a single-controller to a dual-controller when there is one or more EXP3500 enclosures attached to the storage subsystem Before you upgrade the DS3500 storage subsystem from a single-controller to a dual-controller, make sure that: v You purchase a second environmental services module (ESM) for each EXP3500 expansion enclosure that is attached to the DS3500 storage subsystem. You install the second ESM during this procedure. v You purchase the SAS cables that are required to create a redundant drive enclosure path. To upgrade a single-controller DS3500 storage subsystem to a dual-controller DS3500 storage subsystem when there are EXP3500 expansion enclosures attached to the DS3500, complete the following steps: 1. Turn off the DS3500 storage subsystem and all attached EXP3500 expansion enclosures. 2. Install a second RAID controller in the DS3500 storage subsystem by using the instructions in “Installing a controller” on page 95. 3. Install a second ESM in each EXP3500 that is connected to the DS3500 storage subsystem: a. Read the safety information that begins on page xiii and “Best practices guidelines” on page 5. b. Remove the ESM filler panel from the rightmost ESM bay of the EXP3500: 1) On the left side of the ESM filler panel, press the orange release tab to the right just enough to release the handle (no more than 6 mm [0.25 in.]) as you rotate the handle upward. 2) Using the handle, gently slide the ESM filler panel out of the EXP3500. Save the ESM filler panel for future use. c. Hold the new ESM so that the handle is fully extended. d. Gently slide the ESM into the bay until it stops. Rotate the handle downward into the closed position until it clicks. 4. Using the information provided in “DS3500 storage subsystem drive cabling topologies” on page 42, use SAS cables to create a redundant drive-channel path from the DS3500 RAID controllers to the EXP3500 ESMs. Connecting secondary interface cables This section applies to direct (out-of-band) management configurations only. If your configuration uses host-agent (in-band) management, skip this section. Use the Ethernet management port on the rear of the storage subsystem to connect the controllers for direct management of the storage subsystems (see “Direct (out-of-band) management method” on page 48). 46 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

subnet. Use a local private network for the DS3500 and the management station Ethernet connectors. 2. For proper EMI shielding, always use quality braided and shielded serial cables. Connect an Ethernet cable from the management station to the Ethernet connector on controller A on the rear of the storage subsystem. For a dual-controller storage subsystem, you can connect a second Ethernet cable from the management station to the Ethernet connector on controller B. Figure 40 shows the locations of the Ethernet management connectors on the DS3500 storage subsystem. Configuring the storage subsystem You must configure the storage subsystem after you install the storage subsystem in a rack. Use the information in the following sections to configure your storage subsystem. Storage subsystem management methods Before you configure the storage subsystem, determine which method of storage subsystem management you want to use. You can manage the storage subsystems in either of two ways: host-agent (in-band) management or direct (out-of-band) management. Note: For information about any in-band limitations or restrictions that are associated with particular combinations of controller and host bus adapter, see the DS Storage Manager readme file. For more information about setting up in-band or out-of-band management connections, see the DS Storage Manager 10 Installation and Host Support Guide for the operating system of the host server that you will use to manage the DS3500 storage subsystem. The document is in the Documentation folder on the IBM System Storage DS3500 Support DVD. Important: If the operating system of the host server that the storage subsystem logical drives are mapped to is not Microsoft Windows Server 2003 or Windows Server 2008, you must make a direct (out-of-band) management connection to the storage subsystem to set the correct host type first. Then, the server will be able to recognize the storage subsystem correctly for host-agent (in-band) management. Host-agent (in-band) management method: This method requires that host-agent software be installed on the host server. The host-agent software enables the DS Storage Manager software client program to manage the storage subsystem by using the same connections between the host server and the storage Ethernet ports Controller A Ethernet ports Controller B Figure 40. Ethernet port locations on a dual-controller DS3500 Chapter 3. Cabling the storage subsystem and expansion enclosure 47

host. The management station can be the host or a workstation on the Ethernet network. The client software is installed on the management station. Figure 41 shows the host-agent (in-band) management method. Direct (out-of-band) management method: This method uses Ethernet connections from a management station to each controller in the storage subsystem. You must install at least one management station. The management station can be the host or a workstation on the Ethernet network. The client software is installed on the management station. Attach Ethernet cables to each management station (one pair per storage subsystem). You will connect the cables to each storage subsystem controller later when you install the storage subsystem. Figure 42 on page 49 shows the direct (out-of-band) management method. Note: Do not connect the DS3500 storage subsystem Ethernet ports to a public network or public subnet. To minimize security risks, create a private network between the DS3500 storage subsystem and the management station. Figure 41. Host-agent (in-band) managed storage subsystems 48 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

Before attempting to attach hosts to the storage subsystem, make sure that the host systems and host bus adapters (HBAs) are installed correctly and updated with the latest firmware and drivers. Notes: 1. See the documentation that comes with the HBA for installation requirements and procedures. 2. Use the correct HBA firmware and device driver. For the latest supported HBAs and device drivers, consult the readme that comes with the DS3500 controller firmware. For a SAS attached host, attach a SAS cable to each HBA. Continue with “Connecting SAS hosts to the DS3500” to connect the other end of each cable to a controller. For a Fibre Channel attached host, attach a fibre channel cable to each HBA. Continue with “Connecting Fibre Channel hosts to the DS3500” on page 53 to connect the other end of each cable to the controller. For an iSCSI attached host, attach an Ethernet cable to each host. Continue with “Connecting iSCSI hosts to the DS3500” on page 59 to connect the other end of each cable to the controller. Connecting SAS hosts to the DS3500 The DS3500 can support up to 28 hosts when attached to the SAS Connectivity Module for IBM BladeCenter and up to four hosts in a direct attached host environment. The number of supported hosts depends on the number of host ports Figure 42. Direct (out-of-band) managed storage subsystems Chapter 3. Cabling the storage subsystem and expansion enclosure 49

the host servers to the DS3500 storage subsystem, use redundant host connections. Notes: 1. In a single-controller storage subsystem, the use of redundant host connections is supported. Redundant host connections provide path failover but limit the number of redundant host connections to one. 2. The DS3500 ships with four storage partitions by default. For additional storage partitions, purchase an optional Storage Partition Premium Feature upgrade. Contact your IBM reseller or marketing representative for more information. To connect a SAS host bus adapter (HBA) to the storage subsystem, complete the following steps: 1. Connect a SAS cable from the RAID controller host port to the SAS HBA that is in the host. Figure 16 on page 24 shows the location of the host ports. 2. Create a redundant host connection: v If the storage subsystem is a single-controller storage subsystem, connect a SAS cable from the second SAS HBA to another host port on the RAID controller. You can create a maximum of one redundant host connection to a single-controller storage subsystem. v If the storage subsystem is a dual-controller storage subsystem, connect a SAS cable from the second SAS HBA to a host port on the other RAID controller. You can create a maximum of three redundant host connections to a dual-controller storage subsystem. See “Direct-attached single-controller connections” and “Direct-attached dual-controller connections” on page 52 for illustrations of host connections. Direct-attached single-controller connections: The following illustration shows a direct-attached SAS connection to a single host HBA. 50 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

single-host (redundant host connection). The following illustration shows a direct-attached SAS connection to single HBAs in multiple hosts. Host system with single host adapter DS3500 Figure 43. Single-controller direct-attached SAS connection to a single-host HBA Host system with dual host adapters DS3500 Figure 44. Single-controller direct-attached SAS connection to dual HBAs in a single-host (redundant host connection) Chapter 3. Cabling the storage subsystem and expansion enclosure 51

The following illustration shows a direct-attached SAS connection from each controller to a single HBA in the same host (redundant host connection). The following illustration shows multiple redundant direct-attached SAS host connections. This configuration is also the configuration to use for a dual-node cluster configuration. Host system with single host adapter DS3500 Host system with single host adapter Figure 45. Single-controller direct-attached SAS connection to single HBAs in multiple hosts Host system with dual host adapters DS3500 Figure 46. Dual-controller direct-attached SAS connection to two HBAs in the same host (redundant host connection) 52 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

Using Fibre Channel switches, up to 64 hosts can be redundantly attached to the DS3500 storage subsystem in a Fibre Channel storage area network (SAN) fabric. Note: The DS3500 ships with four storage partitions by default. For additional storage partitions, purchase an optional Storage Partition Premium Feature upgrade. Contact your IBM reseller or marketing representative for more information. To connect a host to the RAID controllers, complete the following steps: 1. Install SFP modules in the host ports on RAID controllers A and B. 2. Connect the Fibre Channel cables to the SFP modules in the RAID controller host ports and to either the SFP modules in the Fibre Channel switch or the optical interface connector on the host bus adapters. Figure 48 on page 54 shows the location on the RAID controllers of the storage subsystem ports where the host-system cables connect. Host system with dual host adapters DS3500 Host system with dual host adapters Figure 47. Dual-controller direct-attached SAS connections to multiple HBAs in multiple hosts Chapter 3. Cabling the storage subsystem and expansion enclosure 53

connection. In a direct connection configuration, you can create a maximum of two redundant host connections for the storage subsystem. See “Direct-attached single-controller connections” and “Direct-attached dual-controller connections” on page 56 for illustrations of host connections. For examples of redundant, partially-redundant, and non-redundant host and drive loop configurations, see “Fibre Channel host loop configurations” on page 57. Direct-attached single-controller connections: The following illustration shows a direct-attached connection to a single host HBA. The following illustration shows a direct-attached connection to dual HBAs in a single host (redundant host connection). Serial port Ethernet ports SAS host port 1 SAS host port 2 Fibre channel host port 3 Controller A Fibre channel host port 4 Fibre channel host port 5 Fibre channel host port 6 Drive expansion port Seven-segment numeric display Controller B Figure 48. Location of host connectors on RAID controllers Host system with single host adapter DS3500 Figure 49. Single-controller direct-attached Fibre Channel connection to a single host HBA 54 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

multiple hosts. Host system with dual host adapters DS3500 Figure 50. Single-controller direct-attached Fibre Channel connection to dual HBAs (redundant host connection) Host system with single host adapter DS3500 Host system with single host adapter Figure 51. Single-controller direct-attached Fibre Channel connection to single HBAs in multiple hosts Chapter 3. Cabling the storage subsystem and expansion enclosure 55

The following illustration shows a direct-attached connection from each controller to a single HBA in the same host (redundant host connection). The following illustration shows multiple redundant direct-attached host connections. Host system with dual host adapters DS3500 Figure 52. Dual-controller direct-attached Fibre Channel connection to two HBAs in the same host (redundant host connection) 56 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

The storage subsystem Fibre Channel connection consists of up to two host Fibre Channel loops. The host Fibre Channel loops provide the fibre path for host attachment. They can consist of Fibre Channel cables, SFP modules, host bus adapters, Fibre Channel switches, and RAID controllers. Fibre Channel host loop configurations: You must determine how the host systems will connect to the storage subsystem. You can connect up to two host systems directly to the storage subsystem. The illustrations in the following sections show common host system configurations. Redundant host loops: This section provides examples of host Fibre Channel configurations. Host Fibre Channel configurations: This section provides the following examples of host Fibre Channel configurations. v Single SAN fabric configuration, as shown in Figure 54 on page 58 v Dual SAN fabric configuration, as shown in Figure 55 on page 58 v Two storage subsystems in a dual SAN fabric configuration, as shown in Figure 56 on page 59 Note: These configurations have host and drive path failover protection and are recommended for high availability. In Figure 54 on page 58, the Fibre Channel switches are connected together through the Inter-Switch Link (ISL) to form a single SAN fabric. Host system with dual host adapters DS3500 Host system with dual host adapters Figure 53. Dual-controller direct-attached Fibre Channel connections to multiple HBAs in multiple hosts Chapter 3. Cabling the storage subsystem and expansion enclosure 57

ISL. Each switch forms its own SAN fabric. This configuration is also the configuration to use for a dual-node cluster configuration. In Figure 56 on page 59, the Fibre Channel switches are not connected together. Each switch forms its own SAN fabric. Host system with two host adapters Host system with two host adapters Fibre Channel switch Fibre Channel switch Inter-Switch Link DS3500 Figure 54. Example of a single Fibre Channel SAN fabric configuration Host system with two host adapters Host system with two host adapters Fibre Channel switch Fibre Channel switch DS3500 Figure 55. Example of a dual Fibre Channel SAN fabric configuration 58 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

The DS3500 can support up to 64 hosts, depending on the number of host ports and host partitions that are used on the RAID controller. To protect against the loss of any one path from the host servers to the storage subsystem, use redundant host connections. To connect a hardware or software iSCSI initiator to the storage subsystem, complete the following steps: 1. Connect a Category 5e or Category 6 Ethernet cable from the RAID controller host port to a gigabit Ethernet switch, and then connect an Ethernet cable from the switch to the host iSCSI initiator port. Figure 15 on page 24 shows the host ports. 2. Create a redundant host connection: v If the storage subsystem is a single-controller storage subsystem, connect an Ethernet cable from the second host port on the RAID controller to a gigabit Ethernet switch. v If the storage subsystem is a dual-controller storage subsystem, connect an Ethernet cable from the host port on the other RAID controller on the storage subsystem to a gigabit Ethernet switch. Notes: 1. Do not configure any two interfaces on the same subnet. 2. If you are using DHCP on more than one interface, multiple interfaces can end up on the same subnet. 3. Direct iSCSI connections from the host to the storage subsystem are not supported. For illustrations of host connections, see “Single-controller iSCSI connections” on page 60 and “Dual-controller iSCSI configurations” on page 62. Host system with two host adapters Host system with two host adapters Fibre Channel switch Fibre Channel switch DS3500 DS3500 Figure 56. Example of two storage subsystems in a dual Fibre Channel SAN environment Chapter 3. Cabling the storage subsystem and expansion enclosure 59

The following illustration shows a single controller storage subsystem that is attached to a single host. This configuration has no redundancy. A system with dual iSCSI ports can provide enhanced performance compared to a system that uses only one iSCSI port. The following illustration is an example of a host that is using multiple ports that are attached to a single-controller configuration. This configuration provides some path redundancy. The following illustration is an example of multiple single-port host systems that are connected to a single-controller DS3500 using multiple connections. Gigabit Ethernet switch Host system with one iSCSI connector DS3500 Figure 57. Single-host, single-controller iSCSI configuration Gigabit Ethernet switch DS3500 Host system with two iSCSI connectors Figure 58. Multiple-port, single-controller iSCSI configuration 60 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

multiple-port, single-controller, and redundant-path configuration. You can replace the two gigabit Ethernet switches that are shown in Figure 60 and Figure 61 on page 62 with one large gigabit Ethernet switch that has the required number of ports. Virtual local area network (VLAN) can also be used to isolate the two iSCSI networks, instead of a single switch. Gigabit Ethernet switch DS3500 Host system with one iSCSI connector Host system with one iSCSI connector Figure 59. Multiple single-port hosts, single-controller iSCSI configuration Gigabit Ethernet switch Gigabit Ethernet switch DS3500 Host system with two iSCSI connectors Host system with two iSCSI connectors Figure 60. Multiple-host, multiple-port, single-controller, and redundant-path iSCSI configuration Chapter 3. Cabling the storage subsystem and expansion enclosure 61

The following illustration is an example of a multiple port, single-host configuration. A system that uses multiple iSCSI ports can provide enhanced performance compared to a single-port iSCSI connection. For cluster configurations, the following illustration is an example of a multiple-host, multiple-port, and redundant path and controller configuration. You can replace the two gigabit Ethernet switches that are shown in Figure 63 on page 63 with one large gigabit Ethernet switch that has the required number of ports. Virtual local area network (VLAN) can also be used to isolate the two iSCSI networks, instead of a single switch. Gigabit Ethernet switch Gigabit Ethernet switch DS3500 Host system with two iSCSI connectors Host system with two iSCSI connectors DS3500 Figure 61. Multiple-host, multiple-port, multiple-storage-subsystem iSCSI configuration Gigabit Ethernet switch Host system with two iSCSI connectors DS3500 Figure 62. Single-host, multiple-port, dual-controller iSCSI configuration 62 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

(Fibre Channel or iSCSI, and SAS) configuration. Gigabit Ethernet switch Gigabit Ethernet switch DS3500 Host system with two iSCSI connectors Host system with two iSCSI connectors Figure 63. Multiple-host, multiple-port, dual-controller iSCSI configuration Figure 64. Example of a multiple-host, multiple-port, and multiple-fabric (Fibre Channel or iSCSI, and SAS) configuration Chapter 3. Cabling the storage subsystem and expansion enclosure 63

a DS3500 This section gives an overview and guidelines for installing an IBM BladeCenter configuration to connect to a DS3500. Before you install the BladeCenter configuration, consider the following information: v Make sure that the BladeCenter chassis, blade server, SAS connectivity module, and SAS expansion card are supported. Go to http://www.ibm.com/servers/ storage/support/disk/, select the link for the DS3500, and see the interoperability matrix. v Make sure that an Advanced Management Module is installed in the BladeCenter chassis. The latest level of management-module firmware is available at http://www.ibm.com/bladecenter/. v For the latest SAS connectivity module firmware, SAS expansion card firmware, blade server BIOS code, and device drivers, see http://www.ibm.com/systems/ support/. Installing the BladeCenter configuration To install the IBM BladeCenter SAS Expansion Card (CFFv), IBM BladeCenter SAS Connectivity Module, and blade server, complete the following general steps: 1. Install an IBM BladeCenter SAS Expansion Card (CFFv) in each blade server that you want to connect to the DS3500. See the documentation that comes with the SAS expansion card for installation requirements and procedures. 2. Install the blade servers in the BladeCenter chassis. See the documentation that comes with the blade server for installation requirements and procedures. 3. Install the IBM BladeCenter SAS Connectivity Modules in the I/O bays in BladeCenter unit. See the documentation that comes with the SAS connectivity modules for installation requirements and procedures. 4. Go to “Connecting BladeCenter hosts to the DS3500.” Connecting BladeCenter hosts to the DS3500 The dual-controller DS3500 comes with two SAS host connectors on each controller, enabling up to 28 blade servers in one BladeCenter unit to be redundantly connected to the storage subsystem. To protect against the loss of any one path from the host blade servers to the DS3500, use redundant host connections. To connect a SAS expansion card that is installed in a blade server to the DS3500, complete the following steps: 1. Connect a SAS cable from the SAS host connector on the DS3500 RAID controller to any SAS port on an IBM BladeCenter SAS Connectivity Module that is installed in a BladeCenter unit. 2. To create a redundant host connection, connect a SAS cable from a second SAS connectivity module to a host connector on the other RAID controller in the DS3500. You can create a maximum of two redundant host connections in a dual-controller DS3500. Attention: You can connect only one DS3500 to the BladeCenter SAS Connectivity Module. No other external devices can be connected to the SAS connectivity module. 3. (Optional) For additional storage, you can connect EXP3500 expansion enclosures to the DS3500. For an illustration showing how to configure the expansion enclosures, see Figure 35 on page 41. 64 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

unit. Cabling the DS3500 and EXP3500 power supplies To connect the DS3500 and EXP3500 power-supply cords, complete the following steps: 1. Connect a power cord to a power supply in the DS3500 or EXP3500. 2. Attach the power cord to the strain relief guide. 3. Repeat steps 1 and 2 for the second power supply. 4. Connect the other end of each power cord to a correctly grounded electrical receptacle. For maximum protection against power loss, connect each of the two power supplies to a different power source. Figure 65. Example of a DS3500 that is connected to one BladeCenter unit Chapter 3. Cabling the storage subsystem and expansion enclosure 65

enclosure This chapter describes operating procedures for the storage subsystem and expansion enclosure. To ensure optimal operation of the storage subsystem and expansion enclosure, see “Best practices guidelines” on page 5. Performing the health check process Use the health check process to help you verify and maintain the optimal performance of the storage configuration. The information that you collect in these steps also helps provide IBM Service with important information that is needed during a service call. Perform the following health check tasks after the initial configuration of the storage subsystem and after all configuration sessions. Set a regular schedule of periodic health check evaluations to keep the firmware levels current and to preserve optimal data access and performance. 1. Monitor the Recovery Guru in the DS Storage Manager software for any obvious storage subsystem errors or problem conditions. 2. Gather and save the following storage subsystem event logs for review by IBM Service. These event logs should be gathered periodically for regular health check monitoring regardless of the state of the Recovery Guru. (You can collect all these logs at once and compress them into a single file by clicking the Support tab and then Gather Support Information in the Subsystem Management window.) v DS3500 storage subsystem management event log (MEL) v Storage Subsystem Profile or DS3500 Profile v SAS PHY error log In addition, you should also collect event logs for the host servers that have mapped logical drives from the storage subsystem. Attention: Save these event-log files to a server disk that will remain accessible in the event of a DS3500 storage configuration failure. Do not save these event log files only to a LUN in the DS3500 storage subsystem. 3. Use the Storage Subsystem Profile or DS3500 Profile to make sure that the following firmware levels are at the latest versions: v Controller firmware v ESM firmware v Drive firmware If the firmware is not up-to-date, upgrade the firmware and software to the latest level that is applicable to the storage configuration. See “Software and hardware compatibility and upgrades” on page 11 for information about where to find the latest firmware and software. Attention: You must resolve Recovery Guru errors or problems before you upgrade firmware. Save the storage subsystem profile before you perform any controller or ESM firmware upgrades. Save the storage subsystem profile and all .cfg files to a server disk that will remain accessible in the event of a DS3500 storage configuration failure. © Copyright IBM Corp. 2010 67

following functions are enabled: v For all DS3500 models, enable Media Scan both at the controller level and at the LUN level. v For all DS3500 models, enable the read/write cache. In addition, use the Storage Subsystem Profile to make sure that cache is matched between the controllers. Hardware inspection In addition to the health check process, regular hardware inspection and maintenance helps to support the optimal performance of the storage configuration. Periodically inspect the components of the storage configuration. For best results, follow these guidelines: v Maintain an up-to-date storage subsystem profile for the storage configuration. Save the profile to a server disk that will remain accessible in case of a DS3500 storage configuration failure. Do not save the profile only to a LUN in the storage subsystem. v Develop a change-management plan. The plan should include schedules for updating subsystem firmware and server host software. Note: Some updates might require storage subsystem downtime. v Use applicable IBM-approved cables for all situations. Note in the configuration documentation whether any cables are not IBM-approved. v Create and maintain a cabling diagram of the current configuration. Keep this diagram updated as you make configuration changes, and keep the cabling diagram available for review. v Create and maintain a list of other components that are being used within the cabling diagram (such as the host system and other attached devices). v Make sure that all controllers and ESMs are correctly seated. v Make sure that all drives are correctly seated. v Make sure that cables are managed correctly. v Ensure proper air flow and temperature for all components in the storage configuration. You can find details about many of these inspection and maintenance responsibilities in other sections of this document. In addition to these inspection and maintenance responsibilities, conduct training for staff that support storage configurations. Although training is not a part of the health check process, training reduces the potential for configuration problems and contributes to the correct operation of the system. Turning on the storage subsystem This section contains instructions for turning on the storage subsystem under normal situations. “Turning off the storage subsystem” on page 81 contains instructions for turning off the storage subsystem under normal and emergency situations. When you turn on and turn off the storage subsystem, be sure to use the startup sequence that is described in this section. If you are turning on the storage subsystem after an emergency shutdown or power outage, see “Restoring power after an unexpected shutdown” on page 83. 68 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

v The entire storage subsystem has been shut down (the main circuit breakers for the rack are off). v Some expansion enclosures are being powered-on while others remain online (the main circuit breakers for the rack are on). You might encounter this if you are adding an expansion enclosure to increase storage capacity. Attention: 1. Repeatedly turning the power off and on without waiting for the drives to spin down can damage the drives. Always wait at least 70 seconds after you turn off the power before you turn it on again. 2. If you are connecting a power cord to a storage subsystem or expansion enclosure, turn off both of its power switches first. If the main circuit breaker is off, make sure that both power switches are off on each expansion enclosure in the rack before you turn on the main circuit breakers. 1. Are the main circuit breakers turned on? v Yes: Turn off both power switches on each enclosure that you intend to connect to the power. v No: Turn off both power switches on all enclosures in the storage subsystem. 2. Make sure that all power cords are connected. Note: If the power cords are not connected, turn off both power switches on all modules in the configuration before you connect power cords or turn on the main circuit breakers. 3. If the main circuit breakers are turned off, turn them on. Attention: You must turn on power to each attached expansion enclosure before you turn on power to the storage subsystem so that the controllers recognize all drives in the configuration during the startup process. 4. Turn on both power switches on the rear of each expansion enclosure that is attached to the storage subsystem. While each expansion enclosure powers up, the green and amber LEDs on the front and rear of the expansion enclosure turn on and off intermittently. Depending on the configuration, it can take several minutes for each expansion enclosure to power-on. Check the LEDs on the front and rear of all the expansion enclosures. Make sure that no amber LEDs are lit on any of the expansion enclosures. 5. Turn on both ac power switches on the rear of the storage subsystem. Figure 66 shows the locations of the ac power switches. Depending on the number of expansion enclosures in the configuration, the storage subsystem might take up to 10 minutes to fully power-on. While each storage enclosure powers-on, the green and amber LEDs on the front and rear of the expansion enclosure turn on and off intermittently. The cache battery Power switch Power connector Power switch Power connector Figure 66. Power-supply switches and connectors for the DS3500 Chapter 4. Operating the storage subsystem and expansion enclosure 69

During this time, the LEDs on the front and rear of the storage subsystem might flash intermittently. 6. Determine the status of all storage subsystems and components in the configuration by completing the following steps: a. Check all LEDs on each component in the expansion enclosures. Make sure that all the LEDs show normal status. For more information about LED status for expansion enclosures, see “Checking the LEDs” on page 73. b. Check all LEDs on each component in the storage subsystem. Make sure that all the LEDs show normal status. For information about LED status, see Chapter 6, “Solving problems,” on page 127. c. Open the Subsystem Management window, and display the status of the storage subsystem. 7. Are the LEDs indicating normal operation, and is the status Optimal on all the configuration components? v Yes: The procedure is complete. v No: Go to step 8. 8. Diagnose and correct the fault by completing the following steps: a. Run the DS Storage Manager Recovery Guru by clicking Recovery Guru in the Subsystem Management window toolbar. b. Complete the recovery procedure. If the Recovery Guru directs you to replace a failed component, use the individual LEDs on the storage subsystem to locate the specific failed component. For troubleshooting procedures, see Chapter 6, “Solving problems,” on page 127. c. When the recovery procedure is completed, select Recheck in the Recovery Guru. This action runs the Recovery Guru again to make sure that the problem is corrected. d. If the problem remains, contact your IBM technical-support representative. Installing the DS Storage Manager Client See the DS Storage Manager 10 Installation and Host Support Guide for the applicable operating system for instructions for installing the DS Storage Manager software. The document is in the Documentation folder on the IBM System Storage DS3500 Support DVD. Use that document and the online help to configure the logical drives, partitions, and so on for the RAID controllers. Follow the instructions in the operating-system documentation to make the new logical drives accessible to the operating system. Do not proceed with the configuration setup until you have completed the DS Storage Manager installation. Assemble any additional items in preparation for software installation. These items might include the following items: v HBA device drivers v Controller firmware v IP addresses for RAID controllers (for out-of-band management only) v Additional documentation for hosts, HBAs, and expansion enclosures Before any planned system shutdown or after any system additions, removals, or modifications (including firmware updates, logical drive creations, storage partitioning definitions, hardware changes, and so on), save the storage subsystem profile as explained in the DS Storage Manager 10 Installation and Host Support 70 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

drives that are created for the storage subsystem. Always see the readme file that is included with the DS3500 Storage Subsystem firmware package (whether the firmware is accessed by the Web or a CD) for any special requirements or restrictions that apply to that firmware version. Note: Be sure to install the DS Storage Manager event monitoring service to enable continuous monitoring of the status of the storage subsystem. For more information about the importance of this information, see “Monitoring status through software.” Monitoring status through software Use the DS Storage Manager software to monitor the status of the storage subsystem. Run the software constantly, and check it frequently. Notes: 1. You can monitor only storage subsystems that are within the management domain of the storage-management software. 2. If you have not installed the DS Storage Manager Event Monitor service as part of the storage-management software installation, the DS Storage Manager Enterprise Management window must remain open. (If you close the window, you will not receive any alert notifications from the managed storage subsystems.) For more information, see the Enterprise Management online help. Important: See the DS Storage Manager 10 Installation and Host Support Guide for the applicable operating system for instructions for installing the DS Storage Manager software. The document is in the Documentation folder on the IBM System Storage DS3500 Support DVD. To download the latest version of the DS Storage Manager software, controller firmware, NVSRAM firmware, and the latest ESM firmware, go to http://www.ibm.com/servers/storage/support/disk/. The DS Storage Manager software provides the best way to diagnose and repair storage-subsystem failures. The software can help you: v Determine the nature of the failure v Locate the failed component v Determine the recovery procedures to repair the failure Amber (needs attention) LEDs do not necessarily indicate which component has failed or must be replaced or which type of recovery procedure you must perform. In some cases (such as when a hard disk drive exceeds its Predictive Failure Analysis [PFA] threshold), the amber LED is not lit. Only the DS Storage Manager software can detect the failure. For example, the recovery procedure for a PFA flag (impending drive failure) on a drive varies depending on the drive status (hot spare, unassigned, RAID level, current logical drive status, and so on). Depending on the circumstances, a PFA flag on a drive can indicate a high risk of data loss (if the drive is in a RAID 0 volume) or a minimal risk (if the drive is unassigned). Only the DS Storage Manager software can identify the risk level and provide the necessary recovery procedures. Chapter 4. Operating the storage subsystem and expansion enclosure 71

lit, so checking the LEDs will not notify you of the failure, even if the risk of data loss is high. Recovering from a storage-subsystem failure might require you to perform procedures other than replacing the component (such as backing up the logical drive). The DS Storage Manager software provides these procedures. Attention: Not following the software-recovery procedures can result in data loss. In addition, always replace a failing component as soon as possible to minimize additional failures that might occur, causing loss of data access. Firmware updates Attention: Save the storage subsystem profile before you perform any controller or ESM firmware upgrades. Save the profile and all configuration (.cfg) files to a server disk that will remain accessible in the event of a DS3500 storage configuration failure. Do not save these files only to a LUN in the storage subsystem. To ensure the optimal operation of the storage subsystem and its attached expansion enclosures, the ESM firmware, controller firmware, hard disk drive firmware, and the NVSRAM firmware must be up-to-date. Go to http://www.ibm.com/servers/storage/support/disk/ to get the latest updates. Always see the readme files that are included with the firmware packages for the most up-to-date information about firmware prerequisites, firmware update instructions, download-sequence information, and host I/O restrictions, if any. Apply the necessary updates before you configure the storage subsystem arrays and logical drives. Subscribe to My Support for automatic notifications of firmware or DS Storage Manager software updates or any important information about the DS3500 storage subsystems (see “Product updates” on page 5). Attention: Failure to observe the limitations, prerequisites, sequences, and dependencies in the readme file might result in a loss of data access. Unless the readme file contains special requirements for the sequence in which you upgrade firmware, perform firmware upgrades in the following sequence: 1. ESM firmware for the expansion enclosures 2. Controller firmware 3. Controller NVSRAM 4. Drive firmware Troubleshooting the storage subsystem The DS Storage Manager software is the best way to monitor the storage subsystem, diagnose a problem, and recover from a hardware failure. Run the DS Storage Manager software continuously, and frequently check the configuration status. To check the status of and identify a problem with the storage subsystem, complete the following steps. If a problem has occurred, use the DS Storage Manager software and the LEDs on the storage subsystem to help locate a failed component. 1. Open the Subsystem Management window. 2. Click the Summary tab, and view the status of the storage subsystem. 72 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

v Yes: Go to step 4. v No: All components are Optimal. Go to step 5. 4. Click Recovery Guru on the toolbar. Perform the procedure in the Recovery Guru to correct the problem. The Recovery Guru might direct you to replace the failed component. If so, go to step 5. Attention: If the fault requires you to power-off an attached expansion enclosure, you might have to cycle the power on the DS3500 storage subsystem and all remaining expansion enclosures that are connected to the storage subsystem. Contact your IBM technical-support representative before you power-off any attached expansion enclosures. 5. Check the LEDs on the front and the rear of the expansion enclosure. A green LED indicates a normal status; an amber LED indicates a hardware fault. 6. Is an amber LED lit? v Yes: Locate and troubleshoot the failed components. See “Checking the LEDs.” v No: You are finished with this procedure. If you are still experiencing a problem with the storage subsystem, create, save, and print a storage subsystem profile, and contact your IBM technical-support representative for assistance. When the recovery procedure is completed, select Recheck in the Recovery Guru to run the Recovery Guru again to make sure that the problem is corrected. Checking the LEDs The LEDs indicate the status of the storage subsystem and components. Green LEDs indicate a normal operating status; amber LEDs indicate a possible failure; a blue LED on a component indicates that it is safe to remove the component. The DS3500 also has a blue system locator LED that is lit when the menu function is selected in the Subsystem Management window, causing the Locate command to be sent to the DS3500. Check all the LEDs on the front and rear of the storage subsystem when you turn on the power. During power-on, the LEDs flash intermittently as the storage subsystem and components complete the power-on process. In addition to checking for faults, you can use the LEDs on the front of the storage subsystem to determine whether the drives are responding to I/O transmissions from the host. Power-supply LEDs This section describes the primary LEDs on the DS3500 power supplies. Chapter 4. Operating the storage subsystem and expansion enclosure 73

Future use only. DC power LED (green) When this green LED is lit, it indicates that the DS3500 is turned on and is supplying both 5-volt and 12-volt dc power. OK to remove LED (blue) When this blue LED is lit, it indicates that it is safe to remove the power supply. Fault LED (amber) When this amber LED is lit, it indicates that a power supply or fan has failed or that a redundant power supply is not turned on. AC power LED (green) When this green LED is lit, it indicates that the storage subsystem is receiving ac power. Front LEDs This section describes the primary LEDs and controls on the front of the storage subsystem. The front LEDs and controls on the DS3512 storage subsystem and EXP3512 expansion enclosure are shown in Figure 68. The front LEDs and controls on the front of the DS3524 storage subsystem and EXP3524 expansion enclosure are shown in Figure 69 on page 75. Fault LED (amber) AC power LED (green) OK to remove LED (blue) DC power LED (green) Standby power LED (green) Figure 67. Power-supply LEDs Hard disk drive activity LED Power-on LED System locator LED System error LED Overtemperature LED Standby power LED Hard disk drive status LED Figure 68. DS3512 storage subsystem and EXP3512 expansion enclosure front LEDs and controls 74 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

When this green LED is lit, it indicates that the power supply is turned on and is supplying both 5-volt and 12-volt dc power. System locator LED (blue) This blue LED can be lit by the DS Storage Manager software to aid in visually locating the storage subsystem. Overtemperature LED (amber) When this amber LED is lit, it indicates that the storage subsystem is in an over temperature condition. System error LED (amber) When this amber LED is lit, it indicates that the unit has a fault, such as in a power supply, controller, or hard disk drive. Standby power LED (green) Future use only. Hard disk drive activity LED (green) Each hard disk drive has an activity LED. When this green LED is flashing, it indicates drive activity. Hard disk drive status LED (amber) Each hard disk drive has a status LED. When this amber LED is lit continuously, it indicates a drive failure. When it is flashing, it indicates that the controller is identifying or building a drive. Controller LEDs This section describes the controller LEDs on the rear of the storage subsystem. Power-on LED System locator LED System error LED Overtemperature LED Standby power LED Hard disk drive status LED (amber) Hard disk drive activity LED (green) Figure 69. DS3524 storage subsystem and EXP3524 expansion enclosure front LEDs Chapter 4. Operating the storage subsystem and expansion enclosure 75

When this LED is lit, it indicates that the Ethernet speed between the controller and the management workstation is 1000 Mbps. When this LED is off, it indicates that the Ethernet speed is 100 Mbps. Ethernet activity LED When this LED is lit, it indicates that the link is established between the controller and the management workstation. When this LED is flashing, there is activity between the controller and the management workstation. When this LED is off, no link is established between the controller and the management workstation. SAS link service action LED (amber) When this LED is lit, it indicates a problem with the SAS link that requires service action. The normal condition of this LED is off. SAS link up LED (green) When this LED is lit, it indicates that the link is established between the controller and a host. When this LED is flashing, it indicates activity on the link. When this LED is off, no link is established. Cache active LED (green) When this LED is lit, it indicates that battery backup has been enabled and data is in the memory cache. When this LED is blinking, cache offload is in process. When this LED is off, caching is turned off and no data is in the memory cache. Fault LED (amber) When this LED is lit, it indicates that a fault has occurred in the controller and the controller must be replaced. The normal condition of this LED is off. OK to remove LED (blue) When this LED is lit, it indicates that it is safe to remove the controller from the DS3500; no activity is taking place, and no data remains in the memory cache. The normal condition of this LED is off. Battery charging LED (green) When this LED is lit, the battery is fully charged. When this LED is blinking, the battery is charging. When this LED is off, the battery either failed or has been removed from the controller. Battery fault LED (amber) When this LED is lit, it indicates that the battery fails to hold a charge and should be replaced. The normal condition of this LED is off. Figure 70. Controller LEDs 76 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

When this LED is lit, it indicates that the link is established with the controller. When this LED is flashing, there is activity on this port. When this LED is off, no link is established with this port. iSCSI speed LED (amber) When this LED is lit, it indicates that the port is operating at 1000 Mbps. When this LED is off, it indicates that the speed of the port is 100 Mbps. If the optional Fibre Channel host port adapter is installed in the RAID controller, the host port adapter contains four additional Fibre Channel host connectors with LEDs. Fibre Channel speed LEDs 2 Gbps and 4 Gbps These LEDs in combination indicate the speed of the Fibre Channel host port. Table 6. Fibre Channel port LEDs 2 Gbps LED 4 Gbps LED Host port speed On Off 2 Gbps Off On 4 Gbps On On 8 Gbps Off Off No SFP module is present or the SFP module is faulty iSCSI activity LED iSCSI speed LED Figure 71. iSCSI host port adapter LEDs Figure 72. Fibre Channel host port adapter LEDs Chapter 4. Operating the storage subsystem and expansion enclosure 77

When this LED is lit, it indicates a problem with the SAS link that requires service action. The normal condition of this LED is off. SAS link up LED (green) When this LED is lit, it indicates that the link is established between the controller and a host. When this LED is flashing, it indicates activity on the link. When this LED is off, no link is established. Seven-segment numeric display LEDs The seven-segment numeric display LEDs provide information about enclosure identification and diagnostics. Figure 74 shows the numeric display LEDs. When you turn on the power to the storage or expansion enclosure, the numeric display LEDs cycle though various codes as the controller or ESM firmware starts to boot. When the storage subsystem is operating normally, the numeric display shows the enclosure identification (enclosure ID) of the storage subsystem. If an error has occurred and the Needs Attention LED is on, the numeric display shows diagnostic information. The numeric display indicates the information is diagnostic by illuminating an LED that appears as a decimal point between the display numbers. The diagnostic LED tuns off when the numeric display shows the enclosure ID. The numeric display shows various diagnostic codes as the enclosures perform the startup process after each power cycle or reset. After diagnostics are complete, the current storage subsystem enclosure ID is displayed. Diagnostic codes in the form of Lx or Hx, where x is a hexadecimal digit, indicate storage subsystem state information. In general, these codes are displayed only when the storage subsystem is in a non-operational state. The storage subsystem might be non-operational due to a configuration problem (such as mismatched types), or it might be non-operational due to hardware faults. If the storage subsystem is non-operational due to system configuration, the storage subsystem Needs Attention LED is off. If the storage subsystem is non-operational due to a SAS link service action LED (amber) SAS link up LED Figure 73. SAS host port adapter LEDs Figure 74. Numeric display LEDs 78 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide

for Lx diagnostic codes are listed in Table 7. Table 7. Controller numeric display diagnostic codes Value Description L0 Mismatched controller board ID. L1 Missing interconnect canister. L2 Persistent memory errors. L3 Persistent hardware errors. L4 Persistent data protection errors. L5 ACS failure. L6 Unsupported host card. L7 Submodel identifier not set or mismatched. L8 Memory configuration error. L9 Link speed mismatch. LA Reserved. LB Host card configuration error. LC Persistent cache backup configuration error. LD Mixed cache memory DIMMs. LE Uncertified cache memory DIMM sizes. LF Lockdown with limited SYMbol support. LH Controller firmware mismatch. Table 8. ESM numeric display diagnostic codes Value Description FF ESM Boot Diagnostic executing. 88 This ESM is being held in Reset by the other ESM. AA ESM-A application is booting up. bb ESM-B application is booting up. L0 Mismatched ESM types. L2 Persistent memory errors. L3 Persistent hardware errors. L9 Over Temperature. LL Cannot read midplane EEPROMs. LP Missing Drive Port Mapping table. Ln ESM not valid for enclosure. H2 Invalid/Incomplete Configuration. LF Lockdown with limited SYMbol support. LH Controller firmware mismatch. H3 Maximum Reboot Attempts Exceeded. H4 Cannot Communicate with Other ESM. H5 Midplane Harness Failure. H6 Firmware Failure. Chapter 4. Operating the storage subsystem and expansion enclosure 79

Value Description H9 Non-Catastrophic HW. ESM operational, but in a degraded mode. J0 Incompatible module. ESM LEDs This section describes the ESM LEDS. Figure 75 shows the LEDs on the ESM. Link fault LED (amber) When this amber LED is lit, it indicates that the link through the SAS cable is not successful. Link up LED (green) When this green LED is lit, it indicates that the link through the SAS cable is successful. OK to remove LED (blue) This LED is supported only when the EXP3500 is attached to a DS3500 controller. Do not remove the ESM unless this blue LED is lit. Fault LED (amber) When this amber LED is lit, it indicates that the ESM has failed. Power-on LED (green) When this green LED is lit, it indicates that the ESM is receiving power. 1 2 ESM Link fault LED (amber) Link up LED (green) OK to remove LED (blue) Fault LED (amber) Power-on LED (green) Link fault LED (amber) Link up LED (green) Seven-segment numeric display Figure 75. ESM LEDs 80 System Storage DS3500 and EXP3500: Installation, User’s, and Maintenance Guide