Cisco UCS 6248 UP Installation Manual

First Published: 2012-07-19 Last Modified: 2018-03-06 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883 Text Part Number: OL-24475-03

INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS. THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY. The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense. The following information is for FCC compliance of Class B devices: This equipment has been tested and found to comply with the limits for a Class B digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses and can radiate radio frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference will not occur in a particular installation. If the equipment causes interference to radio or television reception, which can be determined by turning the equipment off and on, users are encouraged to try to correct the interference by using one or more of the following measures: • Reorient or relocate the receiving antenna. • Increase the separation between the equipment and receiver. • Connect the equipment into an outlet on a circuit different from that to which the receiver is connected. • Consult the dealer or an experienced radio/TV technician for help. Modifications to this product not authorized by Cisco could void the FCC approval and negate your authority to operate the product The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California. NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED "AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental. Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: https://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R) © 2011–2016 Cisco Systems, Inc. All rights reserved.

Preface vii P R E F A C E Audience vii Conventions vii Related Cisco UCS Documentation ix Obtaining Documentation and Submitting a Service Request ix Product Overview 1 C H A P T E R 1 Interconnect Features 1 Cisco UCS 6248 UP Fabric Interconnect Chassis 2 Cisco UCS 6296 UP Fabric Interconnect Chassis 4 UCS Unified Port Expansion Module 6 Cisco UCS 6248 UP Port Numbering 6 Cisco UCS 6296 UP Port Numbering 7 Cabling Considerations for Fabric Port Channels 7 Power Supplies 8 Fan Module 11 LED Descriptions 12 Port Level LEDs 13 Supported Transceivers 14 SFP+ Transceivers 14 SFP Transceivers 15 SFP+ Copper Cables 15 SFP Fibre Channel Transceivers 16 Installing the Cisco UCS 6200 Series Fabric Interconnect 19 C H A P T E R 2 Preparing for Installation 19 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide iii OL-24475-03

19 Installation Options 20 Airflow Considerations 20 Chassis Weight 20 Installation Guidelines 21 Required Equipment 21 Cabinet and Rack Requirements 22 General Requirements for Cabinets and Racks 22 Requirements Specific to Perforated Cabinets 23 Requirements Specific to Standard Open Racks 23 Cable Management Guidelines 24 Required Equipment 24 Unpacking and Inspecting the Cisco UCS 6200 Series Fabric Interconnect 24 Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack 25 Grounding the System 28 Installing the Cisco UCS 6296 Chassis in a Cabinet or Rack 28 Proper Grounding Practices 30 Preventing Electrostatic Discharge Damage 32 Establishing the System Ground 34 Required Tools and Equipment 34 Grounding the Interconnect 35 Starting the System 37 Replacing or Installing Components 39 Replacing a Management Daughter Card for a Cisco UCS 6248 39 Removing a Management Daughter Card 39 Installing a Management Daughter Card 41 Replacing or Installing Expansion Modules 42 Removing an Expansion Module 42 Installing an Expansion Module 43 Replacing or Installing Power Supplies 44 Removing a Power Supply 44 Installing a Power Supply 44 Wiring a DC Power Connector 45 Fan Modules 46 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 iv Contents

46 Preparing a Fabric Interconnect for Removal 47 Removing a Cisco UCS 6248 UP 48 Removing a Cisco UCS 6296 48 Replacing a Fabric Interconnect 48 Upgrading from a Cisco UCS 6248 UP Fabric Interconnect to a Cisco UCS 6296 UP Fabric Interconnect 52 Repacking the Cisco UCS Fabric Interconnect for Return Shipment 53 Connecting the Cisco UCS 6200 Series Fabric Interconnect 55 C H A P T E R 3 Preparing for Network Connections 55 Connecting to the Console Port 56 Connecting the Management Port 58 Connecting to an SFP+ Ethernet or Fibre Channel Port 59 Installing or Removing SFP Transceivers 59 Installing a Transceiver 59 Removing a Transceiver 59 Installing or Removing Cables into SFP or SFP+ Transceivers 60 Installing a Cable into a Transceiver 60 Removing a Cable from a Transceiver 61 Connecting to a Fibre Channel Port 62 Installing or Removing SFP+ Transceivers 62 Installing an SFP Transceiver 62 Removing an SFP Transceiver 62 Installing or Removing Cables into SFP Transceivers 64 Installing a Cable into an SFP Transceiver 64 Removing a Cable from an SFP Transceiver 64 Maintaining SFP Transceivers and Fiber-Optic Cables 65 Technical Specifications 67 A P P E N D I X A System Specifications 67 Power Specifications 68 Transceiver Specifications 69 Environmental Conditions and Power Requirement Specifications for SFP+ Transceivers 69 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide v OL-24475-03 Contents

70 Environmental Conditions and Power Requirements Specification for SFP Transceivers 70 Cable and Port Specifications 71 A P P E N D I X B Accessory Kit for the Cisco UCS Fabric Interconnect 71 Console Cable 72 Console Port 72 Supported AC Power Cords and Plugs 73 Argentina 73 Australia and New Zealand 74 Peoples Republic of China 74 Taiwan 75 Europe 75 India, South Africa, and United Arab Emirates 75 Israel 76 Italy 76 North America 77 Switzerland 78 United Kingdom 78 Cabinet Jumper Power Cords 79 Site Planning and Maintenance Records 81 A P P E N D I X C Site Preparation Checklist 81 Contact and Site Information 83 Chassis and Module Information 84 Troubleshooting Hardware Components 87 A P P E N D I X D Overview 87 SNMP Traps 87 Server Port Link State Transitions 87 System Hardware Best Practices 88 Installation Best Practices 88 Initialization Best Practices 88 System Operation Best Practices 88 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 vi Contents

• Conventions, on page vii • Related Cisco UCS Documentation, on page ix • Obtaining Documentation and Submitting a Service Request, on page ix Audience To use this installation guide, you must be familiar with electronic circuitry and wiring practices and preferably be an electronic or electromechanical technician who has experience with electronic and electromechanical equipment. Only trained and qualified service personnel (as defined in IEC 60950-1 and AS/NZS60950) should install, replace, or service the equipment. Install the system in accordance with the U.S. National Electric Code if you are in the United States. Conventions Indication Text Type GUI elements such as tab titles, area names, and field labels appear in this font. Main titles such as window, dialog box, and wizard titles appear in this font. GUI elements Document titles appear in this font. Document titles In a Text-based User Interface, text the system displays appears in this font. TUI elements Terminal sessions and information that the system displays appear in this font. System output CLI command keywords appear in this font. Variables in a CLI command appear in this font. CLI commands Elements in square brackets are optional. [ ] Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide vii OL-24475-03

bars. {x | y | z} Optional alternative keywords are grouped in brackets and separated by vertical bars. [x | y | z] A nonquoted set of characters. Do not use quotation marks around the string or the string will include the quotation marks. string Nonprinting characters such as passwords are in angle brackets. < > Default responses to system prompts are in square brackets. [ ] An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line. !, # Means reader take note. Notes contain helpful suggestions or references to material not covered in the document. Note Means the following information will help you solve a problem. The tips information might not be troubleshooting or even an action, but could be useful information, similar to a Timesaver. Tip Means the described action saves time. You can save time by performing the action described in the paragraph. Timesaver Means reader be careful. In this situation, you might perform an action that could result in equipment damage or loss of data. Caution IMPORTANT SAFETY INSTRUCTIONS This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device. SAVE THESE INSTRUCTIONS Warning Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 viii Preface Preface

Documentation Roadmaps For a complete list of all B-Series documentation, see the Cisco UCS B-Series Servers Documentation Roadmap available at the following URL: http://www.cisco.com/go/unifiedcomputing/b-series-doc. For a complete list of all C-Series documentation, see the Cisco UCS C-Series Servers Documentation Roadmap available at the following URL: http://www.cisco.com/go/unifiedcomputing/c-series-doc. For information on supported firmware versions and supported UCS Manager versions for the rack servers that are integrated with the UCS Manager for management, refer to Release Bundle Contents for Cisco UCS Software. Other Documentation Resources Follow Cisco UCS Docs on Twitter to receive document update notifications. Obtaining Documentation and Submitting a Service Request For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation. Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0. Follow Cisco UCS Docs on Twitter to receive document update notifications. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide ix OL-24475-03 Preface Related Cisco UCS Documentation

OL-24475-03 x Preface Obtaining Documentation and Submitting a Service Request

Product Overview This chapter includes the following sections: • Interconnect Features, on page 1 • Cisco UCS 6248 UP Fabric Interconnect Chassis, on page 2 • Cisco UCS 6296 UP Fabric Interconnect Chassis, on page 4 • Power Supplies, on page 8 • Fan Module, on page 11 • LED Descriptions, on page 12 • Supported Transceivers, on page 14 Interconnect Features A Cisco UCS 6200 series fabric interconnect is a top-of-rack fabric interconnect that provides Ethernet and Fibre Channel to all servers in the UCS system. Servers connect to the fabric interconnect, and it connects to the LAN or SAN. This family of fabric interconnects connect UCS servers to 10 Gigabit Ethernet 1, 2, 4, and 8 Gbps Fibre Channel networks, and provides consolidated I/O connectivity to both production Ethernet LANs and Fibre Channel SANs in a cost-effective, high-performance, low-latency environment. The Cisco UCS 6200 series has the following characteristics: • The UCS 6248 UP fabric interconnect is a one-rack-unit (1 RU), 10-Gigabit Ethernet and FCoE device that offers up to 960-Gbps throughput and up to 48 ports. It has 32 1- or 10-Gbps fixed small form-factor pluggable plus (SFP+) ports and one expansion slot. The Cisco UCS 6248 UP has 32 ports on the base system and can be upgraded with one expansion module providing an additional 16 ports. • The UCS 6296 UP fabric interconnect is a two-rack-unit (2 RU), 10-Gigabit Ethernet and FCoE device that offers up to 1920-Gbps throughput and up to 96 ports. It has 48 1- or 10-Gbps fixed SFP+ ports and three expansion slots. The Cisco UCS 6248 UP has 48 ports on the base system and can be upgraded with three expansion modules providing an additional 48 ports. • Universal port functionality allows Ethernet or Fibre Channel over Ethernet (FCoE) physical port types to be selected in software. • Slots on the back of the switch for an expansion module. You can use the UCS EIGUP module. • Slots on the front of the chassis for hot swap-capable power supplies. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 1 OL-24475-03

• One USB port at the front of the switch. Cisco UCS 6248 UP Fabric Interconnect Chassis The Cisco UCS 6248 UP chassis is 1 RU, 1.72 inches tall, 17.3 inches wide and 30.0 inches deep. It mounts in a standard 19-inch rack (the Cisco R Series rack is an ideal choice). The chassis has two power supplies and two fan modules on the front of the chassis, and it has network ports on the rear of the chassis and has one USB port (usb1:) at the front. Thirty-two fixed 10-Gigabit Ethernet ports and an expansion modules slot supporting 16 ports are at the rear of the switch. The airflow is front to back. Figure 1: Cisco UCS 6248 UP Front View 236703 1 3 2 Two power supplies (AC shown) 3 Management (10/100/1000) port, console port, and USB port 1 Two fan modules 2 The rear of the Cisco UCS 6248 UP chassis has 32 fixed 10-Gigabit, Fiber Channel over Ethernet-capable SFP+ Ethernet ports and 1 slot for an optional expansion module. Figure 2: Cisco UCS 6248 UP Rear View Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 2 Product Overview Cisco UCS 6248 UP Fabric Interconnect Chassis

ports 2 32 fixed 10-Gigabit Ethernet ports 1 Figure 3: Cisco UCS 6248 UP Switch Rear View Close-up 16 port Expansion module 3 System status LED and beaconing LED/button 1 32 fixed 10-Gigabit Ethernet ports 2 The front connector ports are in a 2x2 stacked RJ-45 jack. Figure 4: Connector Ports and LED Closeup Network management port 5 Beaconing LED/button 1 Console port 6 System Status LED 2 USB port 7 UCS cross connect port L1 3 UCS cross connect port L2 4 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 3 OL-24475-03 Product Overview Cisco UCS 6248 UP Fabric Interconnect Chassis

Description Status LED No link Off Left Physical link Solid green No activity Off Right Activity Blinking green Cisco UCS 6296 UP Fabric Interconnect Chassis The Cisco UCS 6296 UP chassis is 2 RU or 3.47 inches (8.8 cm) tall, 17.3 inches (43.9 cm) wide, and 29.5 inches (74.9 cm) deep. It is designed to be mounted in a standard 19-inch wide rack. The front of the switch has a USB port, four Ethernet and ports (two cross-connect ports, one management port, and one console port), two power supplies, and four fan modules. Figure 5: Front View of the Cisco UCS 6296 UP System status LED 4 USB port 1 Two power supplies Management and console 5 ports (two RJ-45 Ethernet connector ports on the left, a RJ-45 network management connector on the upper right, and a console connector on the lower right) 2 Four fan modules 6 Identifier LED 3 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 4 Product Overview Cisco UCS 6296 UP Fabric Interconnect Chassis

Figure 6: Management and Console Ports Closeup Network management port 5 Beaconing LED/button 1 Console port 6 System Status LED 2 USB port 7 UCS cross connect port L1 3 UCS cross connect port L2 4 The rear of the Cisco UCS 6296 UP chassis has 48 fixed 10-Gigabit Ethernet data ports on the bottom and three slots for optional expansion modules on top. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 5 OL-24475-03 Product Overview Cisco UCS 6296 UP Fabric Interconnect Chassis

System status LED 3 Expansion modules, shown here with three 16-port Universal GEM2 modules 1 48 fixed 1- and 10-Gigabit Ethernet ports 4 Identifier LED 2 UCS Unified Port Expansion Module Expansion modules allow Cisco UCS 6200 Series Fabric Interconnect to be configured as cost-effective 10-Gigabit Ethernet fabric interconnects and as I/O consolidation platforms with native Fibre Channel connectivity. The Cisco UCS 6248 UP has one slot for an optional expansion module. The Cisco UCS 6296 has three slots for an optional expansion module. Only the 16-port UCS E16UP hot-pluggable expansion module using the small form factor pluggable plus (SFP+) interface is supported. Figure 8: UCS E16UP GEM Cisco UCS 6248 UP Port Numbering Each individual port is numbered, and groups of ports are numbered based on their function. The ports are numbered top to bottom and left to right. There are 32 to 48 ports on the Cisco UCS 6248 UP, depending on whether an expansion module is installed. Each individual port is numbered, and groups of ports are numbered based on their function. The ports are numbered from top to bottom and left to right. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 6 Product Overview UCS Unified Port Expansion Module

Slot 2, 16 port Expansion module (UCS E16UP) 2 Slot 1, 32 fixed ports 1 Cisco UCS 6296 UP Port Numbering Each port on the Cisco UCS 6296 UP is numbered, and groups of ports are numbered based on their function. The ports are numbered top to bottom and left to right. The 48 fixed ports support 8-, 4-, 2-, or 1-Gbps Fibre Channel transceivers and 1- or 10-Gigabit Ethernet transceivers. Figure 10: Port Numbering of Fixed Ports and Universal Port Expansion Modules Port numbering for the 16-port GEM in slot 3 3 Port numbering for the 48 fixed ports in slot 1 1 Port numbering for the 16-port GEM in slot 4 4 Port numbering for the 16-port GEM in slot 2 2 Cabling Considerations for Fabric Port Channels When you configure the links between the Cisco UCS 2200 Series FEX and a Cisco UCS 6200 series fabric interconnect in fabric port channel mode, the available virtual interface namespace (VIF) on the adapter varies depending on where the FEX uplinks are connected to the fabric interconnect ports. Inside the 6248 fabric interconnect there are six sets of eight contiguous ports, with each set of ports managed by a single chip. When all uplinks from an FEX are connected to a set of ports managed by a single chip, Cisco UCS Manager maximizes the number of VIFs used in service profiles deployed on the blades in the chassis. If uplink connections from an IOM are distributed across ports managed by separate chips, the VIF count is decreased. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 7 OL-24475-03 Product Overview Cisco UCS 6296 UP Port Numbering

Adding a second link to a fabric-port-channel port group is disruptive and will automatically increase the available amount of VIF namespace from 63 to 118. Adding further links is not disruptive and the VIF namespace stays at 118. Caution Linking a chassis to two fabric-port-channel port groups does not affect the VIF namespace unless it is manually acknowledged. The VIF namespace is then automatically set to the smaller size fabric port-channel port group usage (either 63 or 118 VIFs) of the two groups. Caution For high availability cluster-mode applications, we strongly recommend symmetric cabling configurations. If the cabling is asymmetric, the maximum number of VIFs available is the smaller of the two cabling configurations. For more information on the maximum number of VIFs for your Cisco UCS environment, see the Configuration Limits document for your hardware and software configuration. Power Supplies Table 2: Power Supply Models Wattage Source Fabric Interconnect Cisco PID 390 110 VAC Cisco UCS 6248 UP UCS-PSU-6248UP-AC 390 -48 VDC Cisco UCS 6248 UP UCS-PSU-6248UP-DC 680 110 VAC Cisco UCS 6296 UP UCS-PSU-6296UP-AC The fabric interconnect uses a front-end power supply. The chassis has slots for two power supplies. Two power supplies can be used for redundancy, but the fabric interconnect is fully functional with one power supply. Power supplies have two LEDs: one for power status and one for failure condition. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 8 Product Overview Power Supplies

Power LED 2 Fault LED 1 Figure 13: DC Power Supply for the Cisco UCS 6248 UP Fabric Interconnect Green LED indicates the input power status. 2 Amber LED indicates a failure condition. 1 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 9 OL-24475-03 Product Overview Power Supplies

Release lever 3 Failure (top) and Power (bottom) LEDs 1 Handle 2 Table 3: Power Supply LED Descriptions Fail LED Status Power LED Status Power Supply Condition Off Off No AC power to all power supplies. On Off Power supply failure, including over voltage, over current, over temperature, and fan failure. 1 Hz Blinking Off Power supply warning events where the power supply continues to operate. These events include high temperature, high power, and slow fan. Off 1 Hz blinking AC present, 3.3 voltage standby (VSB) on, and the power supply unit is off. Off On Power supply on and OK. If one power supply is installed in the chassis, but the other power supply slot is empty, a blank filler panel (N10-S1BLKP=) should be used to cover the empty slot. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 10 Product Overview Power Supplies

module houses four fans. The combination of four fans per module and two modules provides the chassis with eight fans. Figure 15: Cisco UCS 6248 UP Fan Module (UCS-FAN-6248UP=) 330342 1 Fan Tray Status LED 1 The Cisco UCS 6296 UP switch has four fan modules with one fan each. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 11 OL-24475-03 Product Overview Fan Module

2 1 3 239022 Handle 3 Captive screw 1 Status LED 2 The bicolor fan module LED indicates fan tray health. Green indicates normal operation, while amber indicates a fan failure. LED Descriptions The fabric interconnect has three chassis status LEDs. Table 4: LEDs for the Cisco UCS Fabric Interconnect Description Status Color Function Location LED System is on and operating normally Solid On Green Power/Health Front System Status LED Switch is powered off Off Fault condition On Amber Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 12 Product Overview LED Descriptions

operating normally On Green Fan tray health indicator (multi color) Fan trays (front) Fan Tray Status Fan failure has occurred within the fan tray On Amber No AC power to power supply OFF Green PSU Health (multi color) Power supply (front) PSU Status Power supply on and OK Solid On Power supply failures, overvoltage, overcurrent, overtemperature Solid On Amber AC present, 3.3 VSB on, PSU is off 1 Hz blinking Operating normally OFF Chassis is selected Solid on Identify selected Blue chassis Front of chassis Beaconing Chassis is not selected Off Port Level LEDs There are port activity LEDs on the chassis and on the expansion modules. Table 5: Port LEDs Description Color Link is up, enabled, and active. Green (blinking) Link is administratively disabled. Amber POST or operational error. Amber (blinking) Link is down. Off Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 13 OL-24475-03 Product Overview Port Level LEDs

transceivers. Specifications for these transceivers are at this URL: http://www.cisco.com/en/US/docs/interfaces_modules/transceiver_modules/installation/note/78_15160.html SFP+ Transceivers The high bandwidth of 10-Gigabit Ethernet poses challenges to transmissions that are met by the transceiver and cabling options supported by the Cisco UCS 6200 platform. The enhanced SFP+ 10-Gigabit Ethernet transceiver module is a bidirectional device with a transmitter and receiver in the same physical package. It has a 20-pin connector on the electrical interface and duplex LC connector on the optical interface. Table 6: Supported Transceivers Description Cisco SFP 10G SFP+ Fabric extender transceiver module. The FET-10G SFP is only supported for the 6200 Fabric Interconnect when the port is setup as a server port. When it is connected to an uplink port, the FET-10G SFP transciever module is not supported. FET-10G 10GBASE-SR SFP+ module (multimode fiber [MMF]) SFP-10G-SR 10GBASE-LR SFP+ module (single-mode fiber [SMF]) SFP-10G-LR 10GBASE-CU SFP+ cable 1 m (Twinax cable) SFP-H10GB-CU1M 10GBASE-CU SFP+ cable 3 m (Twinax cable) SFP-H10GB-CU3M 10GBASE-CU SFP+ cable 5 m (Twinax cable) SFP-H10GB-CU5M 10GBASE-CU SFP+ cable 7 m (Twinax cable) SFP-H10GB-ACU7M 10GBASE-CU SFP+ cable 10 m (Twinax cable) SFP-H10GB-ACU10M 1000BASE-T SFP GLC-T GE SFP, LC connector SX transceiver (MMF) GLC-SX-MM GE SFP, LC connector LX/LH transceiver (SMF) GLC-LH-SM 1000BASE-T SFP, extended temperature range SFP-GE-T GE SFP, LC connector SX transceiver (MMF), extended temperature range and digital optical monitoring (DOM) SFP-GE-S Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 14 Product Overview Supported Transceivers

extended temperature range and DOM SFP-GE-L 4-Gbps Fibre Channel SW SFP, LC (for FC configured Unified ports) DS-SFP-FC4G-SW 4-Gbps Fibre Channel LW SFP, LC (for FC configured Unified ports) DS-SFP-FC4G-LW 8-Gbps Fibre Channel SW SFP+, LC(for FC configured Unified ports) DS-SFP-FC8G-SW 8-Gbps Fibre Channel LW SFP+, LC (for FC configured Unified ports) DS-SFP-FC8G-LW The maximum length of fiber optic runs is limited to 300 meters. This is imposed by our use of 802.3X/802.1Qbb Priority Pauses. SFP-10G-LR is supported between fabric interconnect and FEX, but the 300 m limit still applies. Note SFP Transceivers The SFP 1 -Gigabit Ethernet transceiver module is a bidirectional device with a transmitter and receiver in the same physical package. Table 7: Supported SFP Optical Transceivers Description Model 1-Gigabit Ethernet copper SFP module GLC-T 1-Gigabit Ethernet—short range (550m max) SFP module GLC-SX-MM 1-Gigabit Ethernet—long range (10km) SFP module GLC-LH-SM The maximum length of fiber optic runs is limited to 300 meters. This is imposed by our use of 802.3X/802.1Qbb Priority Pauses. SFP-10G-LR is supported between fabric interconnect and I/O Module, but the 300 m limit still applies. Note SFP+ Copper Cables Copper cables are available for use with the 10-Gigabit Ethernet SFP+ module. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 15 OL-24475-03 Product Overview SFP Transceivers

Description Model 10GBASE-CU SFP+ 1-meter cable SFP-H10GB-CU1M 10GBASE-CU SFP+ 3-meter cable SFP-H10GB-CU3M 10GBASE-CU SFP+ 5-meter cable SFP-H10GB-CU5M 10GBASE-CU SFP+ 7-meter cable SFP-H10GB-ACU7M= 10GBASE-CU SFP+ 10-meter cable SFP-H10GB-ACU10M= On the Cisco UCS 6200 platforms, you can use an innovative Twinax copper cable that connects to standard SFP+ connectors for in-rack use, and on optical cable for longer cable runs. For in-rack or adjacent-rack cabling, the Cisco UCS 6200 platform supports SFP+ direct-attach 10-Gigabit Ethernet copper, which integrates transceivers with Twinax cables into an energy efficient, low-cost, and low-latency solution. SFP+ direct-attach 10-Gigabit Twinax copper cables use only 0.1 watts of power per transceiver and introduce only approximately 0.25 microsecond of latency per link. For longer cable runs, the Cisco UCS 6200 platform supports multimode, short-reach optical SFP+ transceivers. These optical transceivers use approximately 1 W per transceiver and have a latency of less than 0.1 microsecond. Table 9: Supported Cables Standard Transceiver Latency (Link) Power (each side) Distance Cable Connector (Media) SFF 8431 ~ 0.1 microseconds ~ 0.1 W 5 m Twinax SFP+ CU copper SFF 8461 ~ 6.8 nanoseconds ~ 0.5 W 7 m/ 10 m Active Twinax SFP+ ACU copper IEEE 802.3ae ~ 0 microseconds 1 W 82 m/ 300 m MM OM2 MM OM3 SFP+ SR MMF and SR SFP Fibre Channel Transceivers The Cisco UCS 6200 series fabric interconnects support multimode 850 nm 4 Gbps and 8 Gbps SFPs with 150 m reach. Table 10: SFP Fiber Channel Transceivers Description Model 4 Gbps Fibre Channel-SW SFP, LC (for FC configured Unified ports) Cisco DS-SFP-FC4G-SW Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 16 Product Overview SFP Fibre Channel Transceivers

(for FC configured Unified ports) Cisco DS-SFP-FC4G-LW 8-Gbps Fibre Channel SW SFP+, LC (for FC configured Unified ports) Cisco DS-SFP-FC8G-SW 8-Gbps Fibre Channel LW SFP+, LC (for FC configured Unified ports) Cisco DS-SFP-FC8G-LW The maximum length of fiber optic runs from the fabric interconnect to a chassis is limited to 300 meters. This restriction is imposed by the use of 802.3X/802.1Qbb Priority Pauses. Note Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 17 OL-24475-03 Product Overview SFP Fibre Channel Transceivers

OL-24475-03 18 Product Overview SFP Fibre Channel Transceivers

Installing the Cisco UCS 6200 Series Fabric Interconnect This chapter includes the following sections: • Preparing for Installation, on page 19 • Cabinet and Rack Requirements, on page 22 • Cable Management Guidelines, on page 24 • Required Equipment, on page 24 • Unpacking and Inspecting the Cisco UCS 6200 Series Fabric Interconnect, on page 24 • Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack, on page 25 • Grounding the System, on page 28 • Starting the System, on page 37 • Replacing or Installing Components, on page 39 • Repacking the Cisco UCS Fabric Interconnect for Return Shipment, on page 53 Preparing for Installation Considerations and Warnings Before you install, operate, or service the system, read the Regulatory Compliance and Safety Information for Cisco UCS for important safety information. Note IMPORTANT SAFETY INSTRUCTIONS This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device. Statement 1071 Warning SAVE THESE INSTRUCTIONS Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 19 OL-24475-03

through the use of a special tool, lock and key, or other means of security. Statement 1017 Warning Only trained and qualified personnel must be allowed to install, replace, or service this equipment. Statement 1030 Warning Each new fabric interconnect requires a license. For information on licensing, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html Note Installation Options You can install the Cisco UCS 6200 Series Fabric Interconnect chassis in a perforated or solid-walled EIA cabinet or an open EIA rack (the Cisco R Series Rack is an ideal choice), using the rack-mount kit shipped with the chassis (UCS-ACC-6248UP=), for instructions on installing the chassis using the rack-mount kit shipped with the chassis, see Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack, on page 25. Airflow Considerations Airflow through the Cisco UCS 6200 Series Fabric Interconnect is from front to back. Air enters the chassis through the fan trays and power supplies mounted at the front of the chassis and exits the chassis through perforations on the rear of the chassis. To ensure proper airflow, follow these guidelines: • Maintain ambient airflow throughout the data center to ensure normal operation. • Consider the heat dissipation of all equipment when determining air conditioning requirements. When evaluating airflow requirements, take into consideration that hot air generated by equipment at the bottom of the rack can be drawn in the intake ports of the equipment above. • Be sure to allow unobstructed exhaust airflow. Chassis Weight When lifting the system, follow these guidelines: • Disconnect all power and external cables before lifting the system. • Have two people to lift the system. The Cisco UCS 6248 UP weighs 35 pounds (15.9 kg). The Cisco UCS 6296 weighs 35 pounds (15.88 kg). The Cisco UCS 6296 UP weighs 50 pounds (22.68 kg). • Ensure that your footing is solid and the weight of the system is evenly distributed between your feet. • Lift the system slowly, keeping your back straight. Lift with your legs, not with your back. Bend at the knees, not at the waist. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 20 Installing the Cisco UCS 6200 Series Fabric Interconnect Installation Options

• Prepare the site as described in the Cisco UCS Site Preparation Guide. • Plan your site configuration and prepare the site before installing the chassis. Site Planning and Maintenance Records, on page 81, lists the recommended site planning tasks. • Record the information listed in Site Planning and Maintenance Records, on page 81, as you install and configure the fabric interconnect. • Ensure that there is adequate space around the chassis to allow for servicing and for adequate airflow Technical Specifications, on page 67, lists airflow requirements). • Ensure that the air conditioning meets the heat dissipation requirements listed in Technical Specifications, on page 67 Jumper power cords are available for use in a cabinet. See Cabinet Jumper Power Cords, on page 79. Note • Ensure that the chassis is adequately grounded. If the chassis is not mounted in a grounded rack, we recommend connecting both the system ground on the chassis and the power supply ground to an earth ground. • Ensure that the site power meets the power requirements listed in Technical Specifications, on page 67 If available, you can use an uninterruptible power supply (UPS) to protect against power failures. Avoid UPS types that use ferroresonant technology. These UPS types can become unstable with systems such as the Cisco UCS 6200 Series Fabric Interconnect, which can have substantial current draw fluctuations because of fluctuating data traffic patterns. • Ensure that circuits are sized according to local and national codes. For North America, the power supply requires a 15-A or 20-A circuit. To prevent loss of input power, ensure the total maximum loads on the circuits supplying power to the fabric interconnect are within the current ratings for the wiring and breakers. • Use the following screw torques (listed in Newton-metres) when installing the chassis: • Captive screws: 4 in-lb (0.45 N m) • M3 screws: 4 in-lb (0.45 N m) • M4 screws: 12 in-lb (1.36 N m) • 10-32 screws: 20 in-lb (2.26 N m) • 12-24 screws: 30 in-lb (3.39 N m) Required Equipment Before beginning the installation, ensure that the following items are ready: Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 21 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Installation Guidelines

• 3/16-inch flat-blade screwdriver • Tape measure and level • ESD wrist strap or other grounding device • Antistatic mat or antistatic foam The following additional items (not found in the accessory kit) are required to ground the chassis: • Grounding cable (6 AWG recommended), sized according to local and national installation requirements; the required length depends on the proximity of the Cisco UCS 6200 Series Fabric Interconnect to proper grounding facilities • Crimping tool large enough to accommodate girth of lug • Wire-stripping tool Cabinet and Rack Requirements This section provides the requirements for the following types of cabinets and racks, assuming an external ambient air temperature range of 0 to 104°F (0 to 40°C): • Standard perforated cabinets (60 percent or greater perforation front and back is required, the Cisco R Series rack is an ideal choice) • Standard open racks If you are selecting an enclosed cabinet, we recommend one of the thermally validated types: standard perforated or solid-walled with a fan tray. Note Do not use racks that have obstructions (such as power strips), because the obstructions could impair access to field-replaceable units (FRUs). The Cisco RP series PDUs when mounted in a Cisco R Series Rack should not obstruct FRU replacement. Note General Requirements for Cabinets and Racks The cabinet or rack must be one of the following types: • Standard 19 in. (48.3 cm) (four-post EIA cabinet or rack, with mounting rails that conform to English universal hole spacing per section 1 of ANSI/EIA-310-D-1992. See the Requirements Specific to Perforated Cabinets, on page 23. (The Cisco R Series Rack is an ideal choice.) The cabinet or rack must also meet the following requirements: Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 22 Installing the Cisco UCS 6200 Series Fabric Interconnect Cabinet and Rack Requirements

1.75 in. (4.4 cm). • The minimum vertical rack space per Cisco UCS 6296 chassis must be two RU (rack unit), equal to 3.5 in. (8.8 cm). • The width between the rack-mounting rails must be at least 17.72 in. (45.0 cm) if the rear of the chassis is not attached to the rack. For four-post EIA racks, this is the distance between the two front rails. • For four-post EIA cabinets (perforated): • The minimum spacing for the bend radius for fiber-optic cables should have the front-mounting rails of the cabinet offset from the front door by a minimum of 3 in. (7.6 cm), and a minimum of 5 in. (12.7 cm) if cable management brackets are installed on the front of the chassis. • The distance between the outside face of the front mounting rail and the outside face of the back mounting rail should be 23.5 to 34.0 in. (59.7 to 86.4 cm) to allow for rear-bracket installation. • A minimum of 2.5 in. (6.4 cm) of clear space should exist between the side edge of the chassis and the side wall of the cabinet. No sizeable flow obstructions should be immediately in the way of chassis air intake or exhaust vents. Optional jumper power cords are available for use in a cabinet. Note Requirements Specific to Perforated Cabinets A perforated cabinet is defined here as a cabinet with perforated front and rear doors and solid side walls. In addition to the requirements listed in the General Requirements for Cabinets and Racks, on page 22, perforated cabinets must meet the following requirements: • The front and rear doors must have at least a 60 percent open area perforation pattern, with at least 15 square inches of open area per rack unit of door height. • The roof should be perforated with at least a 20 percent open area. • The cabinet floor should be open or perforated to enhance cooling. The Cisco R Series racks meet or exceed all these requirements. Requirements Specific to Standard Open Racks In addition to the requirements listed in the General Requirements for Cabinets and Racks, on page 22, if mounting the chassis in an open rack (no side panels or doors), the minimum vertical rack space per chassis must be one RU (rack unit), equal to 1.75 in. (4.4 cm). The Cisco R Series racks meet or exceed all these requirements. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 23 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Requirements Specific to Perforated Cabinets

To help with cable management, you might want to allow additional space in the rack above and below the chassis to make it easier to route as many as 56 fiber or copper cables through the rack. Required Equipment Before beginning the installation, ensure that the following items are ready: • Number 1 and number 2 Phillips screwdrivers with torque capability • 3/16-in. flat-blade screwdriver • Tape measure and level • ESD wrist strap or other grounding device • Antistatic mat or antistatic foam The following additional items (not found in the accessory kit) are required to ground the chassis: • Grounding cable (6 AWG recommended), sized according to local and national installation requirements; the required length depends on the proximity of the Cisco UCS 6200 Series Fabric Interconnect to proper grounding facilities • Crimping tool large enough to accommodate girth of lug • Wire-stripping tool Unpacking and Inspecting the Cisco UCS 6200 Series Fabric Interconnect When handling chassis components, wear an ESD strap and handle modules by the carrier edges only. An ESD socket is provided on the chassis. For the ESD socket to be effective, the chassis must be grounded through the power cable, the chassis ground, or the metal-to-metal contact with a grounded rack. Caution Tip Keep the shipping container in case the chassis requires shipping in the future. Tip The interconnect is thoroughly inspected before shipment. If any damage occurred during transportation or any items are missing, contact your customer service representative immediately. Note Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 24 Installing the Cisco UCS 6200 Series Fabric Interconnect Cable Management Guidelines

you have received all items, including the following: • Grounding lug kit • Rack-mount kit • ESD wrist strap • Cables with connectors • Any optional items ordered Step 2 Check for damage and report any discrepancies or damage to your customer service representative. Have the following information ready: • Invoice number of shipper (see packing slip) • Model and serial number of the damaged unit • Description of damage • Effect of damage on the installation Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack This procedure describes how to use the rack-mount kit provided with the chassis to install Cisco UCS 6248 UP into a cabinet or rack. All Cisco UCS 6200 Series Fabric Interconnects use the same basic installation procedure. If the rack is on wheels, ensure that the brakes are engaged or that the rack is otherwise stabilized. Caution Table 11: Cisco UCS 6248 UP Rack-Mount Kit Part Description Quantity Rack-mount brackets 2 M4x0.7 x 8-mm Phillips countersink screws 12 Rack-mount guides 2 10-32 rack nuts 10 10-32 x 3/4-inch Phillips pan-head screws 10 Slider rails (20 inch minimum to a 36 inch maximum) 2 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 25 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack

a) Position a front rack-mount bracket against the chassis and align the screw holes as shown. Then attach the front rack-mount bracket to the chassis with four M4 screws. b) Repeat with the other front rack-mount bracket on the other side of the chassis. Figure 17: Attaching Front Rack-Mount Bracket to the Cisco UCS 6248 UP 1 2 3 4 310515 Rack-mount guides Front rack-mount bracket 3 1 Phillips countersink screws 4 Phillips countersink screws 2 Step 2 Install the rack-mount guides on the chassis as follows: a) Position one of the rack-mount brackets against the side of the chassis and align the screw holes. b) Attach the bracket to the chassis with two of the flat-head M4 screws. c) Repeat with the other rack-mount bracket on the other side of the chassis. Step 3 Attach the slider rails to the rack. Use 2 12-24 screws or 2 10-32 screws, depending on the rack rail thread type. For racks with square holes, insert the 12-24 cage nuts in position behind the mounting holes in the slider rails. a) Repeat with the other slider rail on the other side of the rack. b) Use the tape measure and level to verify that the rails are horizontal and at the same height. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 26 Installing the Cisco UCS 6200 Series Fabric Interconnect Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack

186412 Step 4 Insert the chassis into the rack: a) Using both hands, position the chassis with the back of the chassis between the front posts of the rack. b) Align the two rack-mount guides on either side of the chassis with the slider rails installed in the rack. Slide the rack-mount glides onto the slider rails, and then gently slide the chassis all the way into the rack. If the chassis does not slide easily, try realigning the rack-mount glides on the slider rails. Step 5 Stabilize the chassis in the rack by attaching the front rack-mount brackets to the front rack-mounting rails: a) Insert 2 screws (12-24 or 10-32, depending on rack type) through the cage nuts and the holes in one of the front rack-mount brackets and into the threaded holes in the rack-mounting rail. b) Repeat for the front rack-mount bracket on the other side of the chassis. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 27 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack

Installing the Cisco UCS 6296 Chassis in a Cabinet or Rack This section describes how to use the rack-mount kit provided with the chassis to install Cisco UCS 6296 into a cabinet or rack that meets the requirements. All Cisco UCS 6200 Series Fabric Interconnects use the same installation procedure. If the rack is on wheels, ensure that the brakes are engaged or that the rack is otherwise stabilized. Caution This table lists the items contained in the rack-mount kit provided with the chassis. Table 12: Cisco UCS 6296 Rack-Mount Kit Part Description Quantity Rack-mount brackets 2 M4x0.7 x 8-mm Phillips countersink screws 12 Rack-mount guides 2 10-32 rack nuts 10 10-32 x 3/4-inch Phillips pan-head screws 10 Slider rails 2 Procedure Step 1 Install the front rack-mount brackets as follows: a) Position a front rack-mount bracket against the chassis and align the screw holes as shown below. Then attach the front rack-mount bracket to the chassis with four M4 screws. b) Repeat with the other front rack-mount bracket on the other side of the chassis. Figure 19: Attaching Front Rack-Mount Bracket to a Cisco UCS 6296 2 3 1 186363 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 28 Installing the Cisco UCS 6200 Series Fabric Interconnect Grounding the System

a) Position one of the rack-mount brackets against the side of the chassis and align the screw holes. b) Attach the bracket to the chassis with two of the flat-head M4 screws. c) Repeat with the other rack-mount bracket on the other side of the chassis. Step 3 Attach the slider rails to the rack. Use 2 12-24 screws or 2 10-32 screws, depending on the rack rail thread type. For racks with square holes, insert the 12-24 cage nuts in position behind the mounting holes in the slider rails. a) Repeat with the other slider rail on the other side of the rack. b) Use the tape measure and level to verify that the rails are horizontal and at the same height. Figure 20: Installing the Slider Rails 186412 Step 4 Insert the chassis into the rack: a) Using both hands, position the chassis with the back of the chassis between the front posts of the rack. b) Align the two rack-mount guides on either side of the chassis with the slider rails installed in the rack. Slide the rack-mount glides onto the slider rails, and then gently slide the chassis all the way into the rack. If the chassis does not slide easily, try realigning the rack-mount glides on the slider rails. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 29 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Installing the Cisco UCS 6296 Chassis in a Cabinet or Rack

186413 Step 5 Stabilize the chassis in the rack by attaching the front rack-mount brackets to the front rack-mounting rails: a) Insert 2 screws (12-24 or 10-32, depending on rack type) through the cage nuts and the holes in one of the front rack-mount brackets and into the threaded holes in the rack-mounting rail. b) Repeat for the front rack-mount bracket on the other side of the chassis. Figure 22: Attaching the Switch to the Rack 186414 Proper Grounding Practices Grounding is one of the most important parts of equipment installation. Proper grounding practices ensure that the buildings and the installed equipment within them have low-impedance connections and low-voltage differentials between chassis. When you properly ground systems during installation, you reduce or prevent shock hazards, equipment damage due to transients, and data corruption. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 30 Installing the Cisco UCS 6200 Series Fabric Interconnect Proper Grounding Practices

Grounding Recommendations Electromagnetic Noise Severity Level Environment All lightning protection devices must be installed in strict accordance with manufacturer recommendations. Conductors carrying lightning current should be spaced away from power and data lines in accordance with applicable recommendations and codes. Best grounding recommendations must be closely followed. High Commercial building is subjected to direct lightning strikes. For example, some places in the United States, such as Florida, are subject to more lightning strikes than other areas. Best grounding recommendations must be closely followed. High Commercial building is located in an area where lightning storms frequently occur but is not subject to direct lightning strikes. Best grounding recommendations must be closely followed. Medium to high Commercial building contains a mix of information technology equipment and industrial equipment, such as welding. Determine source and cause of noise if possible, and mitigate as closely as possible at the noise source or reduce coupling from the noise source to the affected equipment. Best grounding recommendations must be closely followed. Existing commercial building is not Medium subject to natural environmental noise or man made industrial noise. This building contains a standard office environment. This installation has a history of malfunction due to electromagnetic noise. Electromagnetic noise problems are not anticipated, but installing a grounding system in a new building is often the least expensive route and the best way to plan for the future. Best grounding recommendations should be followed as closely as possible. Low New commercial building is not subject to natural environmental noise or man-made industrial noise. This building contains a standard office environment. Electromagnetic noise problems are not anticipated, but installing a grounding system is always recommended. Best grounding recommendations should be followed as much as possible. Existing commercial building is not Low subject to natural environmental noise or man-made industrial noise. This building contains a standard office environment. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 31 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Proper Grounding Practices

local laws and regulations. Note Always ensure that all of the modules are completely installed and that the captive installation screws are fully tightened. In addition, ensure that all I/O cables and power cords are properly seated. These practices are normal installation practices and must be followed in all installations. Note Preventing Electrostatic Discharge Damage Electrostatic discharge (ESD) damage, which can occur when modules or other devices are improperly handled, results in intermittent or complete failures. Modules consist of printed circuit boards that are fixed in metal carriers. Electromagnetic interference (EMI) shielding and connectors are integral components of the carrier. Although the metal carrier helps to protect the board from ESD, always use an ESD grounding strap when handling modules. For preventing ESD damage, follow these guidelines: • Always use an ESD wrist strap and ensure that it makes maximum contact with bare skin. • ESD grounding straps are available with banana plugs, metal spring clips, or alligator clips. All Cisco UCS 6200 Series Fabric Interconnect chassis are equipped with a banana plug connector (identified by the ground symbol next to the connector) somewhere on the front panel. We recommend that you use a personal ESD grounding strap equipped with a banana plug. • If you choose to use the disposable ESD wrist strap supplied with most field-replaceable units or an ESD wrist strap equipped with an alligator clip, you must attach the system ground lug to the chassis in order to provide a proper grounding point for the ESD wrist strap. This system ground is also referred to as the network equipment building system (NEBS) ground. Note • If your chassis does not have the system ground attached, you must install the system ground lug. See the Establishing the System Ground, on page 34 for installation instructions and location of the chassis system ground pads. You do not need to attach a supplemental system ground wire to the system ground lug; the lug provides a direct path to the bare metal of the chassis. Note Before you install the system ground lug, you must correctly attach the ESD wrist strap. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 32 Installing the Cisco UCS 6200 Series Fabric Interconnect Preventing Electrostatic Discharge Damage

a) If you are using the ESD wrist strap supplied with the FRUs, open the wrist strap package and unwrap the ESD wrist strap. Place the black conductive loop over your wrist and tighten the strap so that it makes good contact with your bare skin. b) Open the package and remove the ESD wrist strap. Locate the end of the wrist strap that attaches to your body and secure it to your bare skin. Step 2 Grasp the spring or alligator clip and momentarily touch the clip to a bare metal spot (unpainted surface) on the rack. We recommend that you touch the clip to an unpainted rack rail so that any built-up static charge is then safely dissipated to the entire rack. Step 3 Attach the ESD strap to the system ground in one of the following ways: • If you are using a wrist strap that is equipped with a plug, insert the plug into an open screw hole used for the grounding lug. • If you are using a wrist strap with spring or alligator clips, attach either the spring clip or the alligator clip to the ground lug screw: Figure 23: Attaching the ESD Wrist Strap to the Cisco UCS 6248 UP System Ground Lug Screw 2 1 3 4 330337 Side view of the clip being attached to one of the screws holding the grounding lug to the chassis. Open the clip by pressing 2 its handles together. 1 Side view of the clip attached to the grounding screw. 4 Back view of the clip attached to the clip attached to the grounding screw. 3 If you are using the ESD wrist strap that is supplied with the FRUs, squeeze the spring clip jaws open, position the spring clip to one side of the system ground lug screw head, and slide the spring clip over the lug screw head so that the spring clip jaws close behind the lug screw head. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 33 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Preventing Electrostatic Discharge Damage

lug barrel. If you are using an ESD wrist strap that is equipped with an alligator clip, attach the alligator clip directly over the head of the system ground lug screw or to the system ground lug barrel. Note In addition, follow these guidelines when handling modules: • Handle carriers by available handles or edges only; avoid touching the printed circuit boards or connectors. • Place a removed component board-side-up on an antistatic surface or in a static-shielding container. If you plan to return the component to the factory, immediately place it in a static-shielding container. • Never attempt to remove the printed circuit board from the metal carrier. For safety, periodically check the resistance value of the antistatic strap. The measurement should be between 1 and 10 megohm (Mohm). Caution Establishing the System Ground This system ground is referred to as the network equipment building system (NEBS) ground. Note You must use the NEBS ground on AC-powered systems if you are installing this equipment in a U.S. or European Central Office. The NEBS ground provides additional grounding for EMI shielding requirements and grounding for the low-voltage supplies (DC-DC converters) on the modules, and is intended to satisfy the Telcordia Technologies NEBS requirements for supplemental bonding and grounding connections. You must observe the following system grounding guidelines for your chassis: • You must install the NEBS ground connection with any other rack or system power ground connections that you make. The system ground connection is required if this equipment is installed in a U.S. or European Central Office. • You must connect both the NEBS ground connection and the power supply ground connection to an earth ground. The NEBS ground connection is required if this equipment is installed in a U.S. or European Central Office. • You do not need to power down the chassis because this device is equipped with AC-input power supplies. Required Tools and Equipment To connect the system ground, you need the following tools and materials: • Grounding lug—A two-hole standard barrel lug. Supports up to 6 AWG wire. Supplied as part of accessory kit. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 34 Installing the Cisco UCS 6200 Series Fabric Interconnect Establishing the System Ground

• Grounding wire—Not supplied as part of accessory kit. The grounding wire should be sized according to local and national installation requirements. Depending on the power supply and system, a 12 AWG to 6 AWG copper conductor is required for U.S. installations. Commercially available 6 AWG wire is recommended. The length of the grounding wire depends on the proximity of the chassis to proper grounding facilities. • No. 1 Phillips head screwdriver. • Crimping tool to crimp the grounding wire to the grounding lug. • Wire-stripping tool to remove the insulation from the grounding wire. Grounding the Interconnect The chassis has a grounding pad with two threaded M4 holes for attaching a grounding lug. When installing or replacing the unit, the ground connection must always be made first and disconnected last. Statement 1046 Warning We recommend grounding the chassis, even if the rack is already grounded. Caution All power supplies must be grounded. The receptacles of the AC power cables used to provide power to the chassis must be the grounding type, and the grounding conductors should connect to protective earth ground at the service equipment. Caution Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 35 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Grounding the Interconnect

186483 2 3 4 5 6 1 Screws, M4, with square cone washers 4 ESD socket (on switch) 1 NRTL listed grounding lug 5 ESD plug 2 Close-up of grounding pad on switch 6 Grounding cable 3 When installing or replacing the unit, the ground connection must always be made first and disconnected last. Statement 1046 Warning Grounding the chassis is required if you are using DC power supplies, even if the rack is already grounded. A grounding pad with two threaded M4 holes is provided on the chassis for attaching a grounding lug. The ground lug must be NRTL listed. In addition, the copper conductor (wires) must be used and the copper conductor must comply with NEC code. Caution Procedure Step 1 Use a wire-stripping tool to remove approximately 0.75 inches (19 mm) of the covering from the end of the grounding cable. Step 2 Insert the stripped end of the grounding cable into the open end of the grounding lug. Step 3 Use the crimping tool to secure the grounding cable in the grounding lug. Step 4 Remove the adhesive label from the grounding pad on the chassis. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 36 Installing the Cisco UCS 6200 Series Fabric Interconnect Grounding the Interconnect

330337 2 3 4 1 Step 5 Place the grounding lug against the grounding pad so that there is solid metal-to-metal contact, and insert the two M4 screws with washers through the holes in the grounding lug and into the grounding pad. Step 6 Ensure that the lug and cable do not interfere with other equipment. Step 7 Prepare the other end of the grounding cable and connect it to an appropriate grounding point in your site to ensure adequate earth ground. Starting the System Do not connect the Ethernet port to the LAN until the initial system configuration has been performed. For instructions on configuring the system, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html. For instructions on connecting to the console port, see Connecting to the Console Port, on page 56. Note When installing or replacing the unit, the ground connection must always be made first and disconnected last. Statement 1046 Warning The plug-socket combination must be accessible at all times, because it serves as the main disconnecting device. Statement 1019 Warning Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 37 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Starting the System

the front of the chassis, and the captive screws of the power supplies, fan module, and all expansion modules are tight. Step 2 Verify that the power supply and the fan modules are installed. (See Replacing or Installing Power Supplies, on page 44 if necessary.) Depending on the outlet receptacle on your power distribution unit, you may need the optional jumper power cord to connect the Cisco UCS 6200 Series Fabric Interconnect to your outlet receptacle. See Cabinet Jumper Power Cords, on page 79. Note Step 3 Ensure that the chassis is adequately grounded, and that the AC or DC power available has the required power voltages (see Power Specifications, on page 68). For a DC installation, see Wiring a DC Power Connector, on page 45 to correctly wire the DC connector before applying a DC cable. Step 4 For a first-time installation, you will need to work with your network manager to determine the following parameters: • System name • Password for the admin account. Choose a strong password that meets the guidelines for Cisco UCS Manager passwords. This password can not be blank. • Management port IP address and subnet mask • Default gateway IP address • DNS server IP address (optional) • Domain name for the system (optional) Step 5 Connect a PC or laptop directly to the console port of the primary or standalone fabric interconnect. In a cluster configuration, the primary will be the fabric interconnect that powers up first. The console port on the terminal should be set to 9600 baud, 8 data bits, no parity, 1 stop bit. Step 6 If the fabric interconnect will be running in a cluster with another fabric interconnect, you will need to connect ethernet cables between the L1 and L2 ports. Port L1 on fabric interconnect A connects to L1 on fabric interconnect B, and Port L2 on fabric interconnect A connects to L2 on fabric interconnect B. If the fabric interconnect and the UCS instance will be in standalone mode this will not be necessary. Step 7 Connect the power cable to a power source. The system should power on as soon as you connect the AC power cable or DC power connector. Step 8 Listen for the fans; they should begin operating when you plug in the power cable. Step 9 After the system boots, verify that the LED operation is as follows: • Fan module—Status LED is green. • Power supply—Status LED is green. • After initialization, the system status LED is green, indicating that all chassis environmental monitors are reporting that the system is operational. If this LED is orange or red, one or more environmental monitor is reporting a problem. • The Link LEDs for the Ethernet connector should not be on unless the cable is connected. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 38 Installing the Cisco UCS 6200 Series Fabric Interconnect Starting the System

for the Ethernet connector port remains off until the port is connected. Note Step 10 Try removing and reinstalling a component that is not operating correctly. If it still does not operate correctly, contact your customer service representative for a replacement. If you purchased this product through a Cisco reseller, contact the reseller directly for technical support. If you purchased this product directly from Cisco, contact Cisco Technical Support at this URL: http://www.cisco.com/en/US/support/tsd_cisco_worldwide_contacts.html. Note Step 11 Verify that the system software has booted and that the system has initialized without error messages. If any problems occur, see Troubleshooting Hardware Components, on page 87. If you cannot resolve an issue, contact your customer service representative. Step 12 Complete the worksheets provided in Site Planning and Maintenance Records, on page 81 for future reference. Step 13 Configure the primary fabric interconnect as described inthe Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html Step 14 Power up, connect the terminal to the console port, and configure the secondary fabric interconnect as described in the Cisco UCS Manager Configuration Guide. Replacing or Installing Components To prevent ESD damage, wear grounding wrist straps during these procedures and handle modules by the carrier edges only. Caution Replacing a Management Daughter Card for a Cisco UCS 6248 This procedure will be necessary if you need to return a Cisco UCS 6248 and receive a replacement bare chassis, which will not have the management daughter card (UCS-FI-DL2), fans, power supplies, or expansion modules. If you need to install or replace a Cisco UCS 6296 fabric interconnect, you do not need to remove and install the management daughter card. The Cisco UCS 6296 has a nonremovable management daughter card. Note Removing a Management Daughter Card Procedure Step 1 Prepare an antistatic surface for placing removed components. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 39 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Replacing or Installing Components

1 below). Figure 26: Removing the Right Fan Tray from the Chassis Pull the fan tray out of the management daughter card and set on an antistatic surface. Loosen the captive screw on the right most fan tray 2 so that the screw is free of the management daughter card. 1 Step 3 Grasping the fan module handle, pull the fan module out of the chassis (see Callout 2 above) and place it on an antistatic surface. Step 4 Loosen the two captive screws on the management daughter card so that they are no longer secured to the chassis (see Callout 1 below). Figure 27: Detaching the Management Daughter Card from the Chassis Pull the management daughter card part way out of the chassis. 2 Loosen captive screws until they are no longer attached to the chassis. 1 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 40 Installing the Cisco UCS 6200 Series Fabric Interconnect Removing a Management Daughter Card

part way out of the chassis (see Callout 2 above). Step 6 Place your other hand under the management daughter card to support its weight, and fully remove the assembly from the chassis (see Callout 1 below). Figure 28: Removing the Management Daughter Card from the Chassis Pull the handle on the left fan module to completely remove the management daughter card from the chassis. 1 Step 7 Place the removed management daughter card on the antistatic surface. Step 8 On the remaining fan tray in the management daughter card, loosen its captive screw so that it is no longer attached to the management daughter card. Step 9 Pull the fan tray out of the management daughter card. What to do next You are now ready to install a replacement management daughter card. Installing a Management Daughter Card Procedure Step 1 Align the replacement management daughter card to the management daughter card slot in the chassis and push it in all the way into the slot so that its two captive screws are aligned to their holes in the chassis. Step 2 Tighten both captive screws to the chassis. Step 3 Install both fan modules into the replacement management daughter card. (See Replacing a Fan Module, on page 46.) Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 41 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Installing a Management Daughter Card

You can either replace existing expansion modules or install new ones where expansion modules are not installed. If you are replacing a module, you need to follow the procedures that explain how to remove and install expansion modules. If you are installing an expansion module, you need to follow only the installation procedure. To prevent ESD damage, wear grounding wrist straps during these procedures. When handling the expansion modules, handle them only on their carrier edges. Caution Install the Cisco UCS 6200 Series Fabric Interconnect chassis in the rack before installing expansion modules. For information about installing the chassis, see Installing the Cisco UCS 6248 UP Chassis in a Cabinet or Rack, on page 25. Note The expansion module is hot-pluggable, which means that it can be removed or installed without powering down a Cisco UCS 6200 Series Fabric Interconnect. Note Removing an Expansion Module Procedure Step 1 Ensure that the system (earth) ground connection has been made. Step 2 Disconnect any network interface cables attached to the module. Step 3 Loosen the captive screw on the module. Step 4 Remove the module from the chassis by grasping the handle on the right side of the module and sliding the module part of the way out of the slot. Place your other hand under the module to support the weight of it. Do not touch the module circuitry. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 42 Installing the Cisco UCS 6200 Series Fabric Interconnect Replacing or Installing Expansion Modules

Figure 30: Removing an Expansion Module From the Cisco UCS 6296 186598 Step 5 Place the module on an antistatic mat or antistatic foam if not immediately reinstalling it in another slot. What to do next If you are not installing another expansion module in its place, install a filler panel (UCS-BLKE-6200=) to keep dust out of the chassis and to maintain the designed airflow through the chassis. If you are installing a replacement expansion module, proceed to the next section. Installing an Expansion Module Procedure Step 1 Ensure that the system (earth) ground connection has been made. For ground connection instructions, see Establishing the System Ground, on page 34. Step 2 Grasp the handle of the module and place your other hand under the module to support it. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 43 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Installing an Expansion Module

Step 4 Tighten the captive screw on the front of the module. Replacing or Installing Power Supplies The Cisco UCS 6300 Series Fabric Interconnect supports two AC or DC power supplies, but may be used with one power supply. Mixing of AC and DC power supplies is not supported. If you need to replace an existing power supply, follow the procedures that explain how to remove and install power supplies. If you are installing a new power supply where one did not exist before, follow the installation procedure. You can replace a faulty power supply while the system is operating provided the other power supply is functioning. Note Removing a Power Supply If you are using the Cisco UCS 6300 Series Fabric Interconnect with one power supply, removing the power supply causes the system to shut down. If you are using two power supplies and you remove one of them, the system continues to operate. Caution Procedure Step 1 Ensure that the system (earth) ground connection has been made. Step 2 Remove the AC power cord or DC wiring connector. Step 3 Grasp the power supply handle with your left hand. Step 4 Push against the release latch with your left thumb and slide the power supply out of the chassis. Step 5 Place your right hand under the power supply to support it while you slide it out of the chassis. Step 6 If you are not replacing the power supply, install a blank power supply filler panel. If you are replacing the power supply, proceed to Installing a Power Supply, on page 44. Installing a Power Supply Procedure Step 1 Ensure that the system (earth) ground connection has been made. Step 2 If the power supply bay has a filler panel, press the latches on the sides of the filler panel, and then slide it out of the power supply bay. Step 3 Hold the power supply by the handle and position it so that the release latch is on the right, and then slide it into the power supply bay, ensuring that the power supply is fully seated in the bay. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 44 Installing the Cisco UCS 6200 Series Fabric Interconnect Replacing or Installing Power Supplies

installation, you should secure the plug to the power supply by tightening both captive screws on the plug. Depending on the outlet receptacle on your power distribution unit, you may need the optional jumper power cord to connect the Cisco UCS 6300 Series Fabric Interconnect to your outlet receptacle. See Cabinet Jumper Power Cords, on page 79. Note Step 5 Connect the other end of the power cable or cables to a power source. DC sources should connect negative (black wire) and then positive (red wire) connections. In a system with dual power supplies, connect each power supply to a separate power source. In case of a power source failure, the second source will most likely still be available. Step 6 Verify power supply operation by checking that the power supply LED is green. Wiring a DC Power Connector A readily accessible two-poled disconnect device must be incorporated in the fixed wiring. Statement 1022 Warning This product requires short-circuit (overcurrent) protection, to be provided as part of the building installation. Install only in accordance with national and local wiring regulations. Statement 1045 Warning When installing or replacing the unit, the ground connection must always be made first and disconnected last. Statement 1046 Warning Installation of the equipment must comply with local and national electrical codes. Statement 1074 Warning Hazardous voltage or energy may be present on DC power terminals. Always replace cover when terminals are not in service. Be sure uninsulated conductors are not accessible when cover is in place. Statement 1075 Warning Before installing a DC power supply to the fabric interconnect, you must attach DC connection wires that you provide (10 GA recommended) to the DC power connector included in the DC power supply’s accessory kit. Procedure Step 1 Using a 1/8-inch flat head screwdriver or No. 1 Phillips head screwdriver, loosen the set screws on the connector to freely accept the power wires. The connector will accept 8-24 AWG wires, use what your local electrical code calls for. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 45 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Wiring a DC Power Connector

Step 3 Insert the black (DC negative) wire into the right aperture on the connector and tighten down the connection set screw. Finger tight or about 3 ft./lbs should be sufficient. Step 4 Insert the red (DC positive) wire into the left aperture on the connector and tighten down the connection set screw. Do not tighten over 0.7 Nm. Figure 31: Wiring the DC Power Connector 330338 + 2 1 2 1 330338 Fan Modules The fan module is designed to be removed and replaced while the system is operating without presenting an electrical or thermal hazard or damage to the system, provided that the replacement is performed promptly. Replacing a Fan Module When removing the fan tray, keep your hands and fingers away from the spinning fan blades. Let the fan blades completely stop before you remove the fan tray. Statement 258 Warning Procedure Step 1 Ensure that the system (earth) ground connection has been made. Step 2 Loosen the captive screws on the fan module by turning them counterclockwise, using a flat-blade or number 2 Phillips head screwdriver if required. Step 3 Grasp the handle of fan module and pull it outward. Step 4 Pull the fan module clear of the chassis and set it down on antistatic foam or place it in an antistatic bag. Step 5 Hold the replacement fan module with the LED at the bottom. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 46 Installing the Cisco UCS 6200 Series Fabric Interconnect Fan Modules

the chassis as far as it can go and the captive screw makes contact with the chassis, and tighten the captive screw. Step 7 Listen for the fans if the system is powered on. You should immediately hear them operating. If you do not hear them, ensure that the fan module is inserted completely in the chassis and the faceplate is flush with the outside surface of the chassis. Step 8 Verify that the LED is green. If the LED is not green, one or more fans are faulty. If this occurs, contact your customer service representative for a replacement part. If you purchased this product through a Cisco reseller, contact the reseller directly for technical support. If you purchased this product directly from Cisco, contact Cisco Technical Support at this URL: http://www.cisco.com/en/US/support/tsd_cisco_worldwide_contacts.html. Note Preparing a Fabric Interconnect for Removal Removing a standby fabric interconnect is non-disruptive in a redundant Cisco UCS configuration. Removing an active fabric interconnect will cause the standby fabric interconnect to become active with minimal or no disruption. Be aware that if you remove both the active and standby fabric interconnect, or the sole fabric interconnect from a standalone system, you are shutting down the entire Cisco UCS domain. Procedure Step 1 Use Cisco UCS Manager to perform the following tasks: a) Back up your Cisco UCS Manager configuration. b) Shut down the OS on all servers in the Cisco UCS domain. c) Disable the Smart Call Home feature in the Cisco UCS domain. d) Decommission every attached chassis in the Cisco UCS domain. For details, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html Step 2 Power down every attached chassis as described in the Cisco UCS 5108 Server Chassis Hardware Installation Guide. When powering down and removing clustered fabric interconnects, remove the secondary first, and then remove the primary. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 47 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Preparing a Fabric Interconnect for Removal

The slider rail and front rack-mount brackets do not have a stop mechanism when sliding in and out. If the front of the chassis is unfastened from the rack and the chassis slides forward on the slider rails, it may slip off the end of the rails and fall out of the rack. Caution Procedure Step 1 Ensure that the weight of the Cisco UCS 6248 UP is fully supported and that the chassis is being held by another person. Step 2 Remove the two screws holding the grounding cable to the chassis. Step 3 Disconnect the power cord and the console cables. Step 4 Disconnect all cables that are connected to SFP+ transceivers. Step 5 Remove the screws fastening the front rack-mount brackets to the mounting rails. Step 6 Gently slide the Cisco UCS 6248 UP toward you, off of the slider rails and out of the rack. Removing a Cisco UCS 6296 The slider rail and front rack-mount brackets do not have a stop mechanism when sliding in and out. If the front of the chassis is unfastened from the rack and the chassis slides forward on the slider rails, it may slip off the end of the rails and fall out of the rack. Caution Procedure Step 1 Ensure that the weight of the Cisco UCS 6296 is fully supported and that the chassis is being held by another person. Step 2 Remove the two screws holding the grounding cable to the chassis. Step 3 Disconnect the power cord and the console cables. Step 4 Disconnect all cables that are connected to SFP+ transceivers. Step 5 Remove the screws fastening the front rack-mount brackets to the mounting rails. Step 6 Gently slide the Cisco UCS 6296 toward you, off of the slider rails and out of the rack. Replacing a Fabric Interconnect Use the following procedure when you need to replace a single fabric interconnect with the same model of fabric interconnect. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 48 Installing the Cisco UCS 6200 Series Fabric Interconnect Removing a Cisco UCS 6248 UP

following procedure. For additional information, refer to the Cisco UCS Manager Getting Started Guide, the Cisco UCS Manager Infrastructure Management Guide, and the instructional videos available at this URL: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-manager/ products-installation-and-configuration-guides-list.html As a best practice, perform a full configuration backup before replacing the fabric interconnect. Procedure Step 1 Contact Cisco TAC or your Cisco Sales representative to have them transfer the license to the replacement fabric interconnect. Step 2 Label the ports and the cables that you using so that you can refer to this information later. The cabling and port numbering should be same for the replacement fabric interconnect. Step 3 Log into UCS Manager and verify the state of the fabric interconnect, either active or subordinate. From the CLI, enter the show cluster extended-state command. High availability (HA) should be running in UCS Manager. Example: FI-A# show cluster extended-state Cluster Id: 0x537d0580bf9911e0-0x8955000decd07984 A: UP, PRIMARY B: UP, SUBORDINATE A: memb state UP, lead state PRIMARY, mgmt services state: UP B: memb state UP, lead state SUBORDINATE, mgmt services state: UP heartbeat state PRIMARY_OK INTERNAL NETWORK INTERFACES: eth1, UP eth2, UP HA READY <<<<<<<<<<<<<←---------------- HA is READY Detailed state of the device selected for HA storage: Chassis 1, serial: FOX1344G1R1, state: active Chassis 2, serial: FOX1318GDKR, state: active FI-A# Step 4 Back up the software configuration. Step 5 Use the fabric evacuation procedure on the subordinate fabric interconnect to ensure there is no data traffic impact during the hardware replacement. Step 6 Power down the subordinate fabric interconnect by unplugging it from the power source. Example: FI-A# show cluster extended-state Cluster Id: 0x537d0580bf9911e0-0x8955000decd07984 A: UP, PRIMARY B: DOWN, INAPPLICABLE A: memb state UP, lead state PRIMARY, mgmt services state: UP B: memb state DOWN, lead state INAPPLICABLE, mgmt services state: DOWN heartbeat state SECONDARY_FAILED INTERNAL NETWORK INTERFACES: Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 49 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Replacing a Fabric Interconnect

Peer Fabric Interconnect is down Detailed state of the device selected for HA storage: Chassis 1, serial: FOX1344G1R1, state: active Chassis 2, serial: FOX1318GDKR, state: active FI-A# Step 7 Disconnect the cables from the front and back of the fabric interconnect and remove it. Follow the instructions in this section for removing a fabric interconnect. Step 8 Install the replacement fabric interconnect. Follow the instructions in this section for installing the fabric interconnect. Step 9 Connect the management and console cables to the replacement fabric interconnect. Step 10 Connect the L1/L2 cables that were disconnected to the replacement fabric interconnect. Step 11 Connect the data cable according to the labels that you created in Step 2. Step 12 Connect the power cable to the fabric interconnect and it will automatically boot and run POST tests. ---- Basic System Configuration Dialog ---- This setup utility will guide you through the basic configuration of the system. Only minimal configuration including IP connectivity to the Fabric interconnect and its clustering mode is performed through these steps. Type Ctrl-C at any time to abort configuration and reboot system. To back track or make modifications to already entered values, complete input till end of section and answer no when prompted to apply configuration. Enter the configuration method. (console/gui) ? console Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the cluster. Continue (y/n) ? y Enter the admin password of the peer Fabric interconnect: Connecting to peer Fabric interconnect... done Retrieving config from peer Fabric interconnect... done Peer Fabric interconnect Mgmt0 IP Address: 122.255.252.2 Peer Fabric interconnect Mgmt0 IP Netmask: 255.255.255.0 Cluster IP address : 122.255.252.1 Physical Switch Mgmt0 IPv4 address : 122.255.252.3 Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes Applying configuration. Please wait. Configuration file - Ok Cisco UCS 6300 Series Fabric Interconnect FI-B login: Step 13 If necessary, upgrade the UCS Manager software. If the replacement fabric interconnect is not running the same firmware version as the cluster, the setup utility can upgrade the firmware. Example: Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 50 Installing the Cisco UCS 6200 Series Fabric Interconnect Replacing a Fabric Interconnect

This setup utility will guide you through the basic configuration of the system. Only minimal configuration including IP connectivity to the Fabric interconnect and its clustering mode is performed through these steps. Type Ctrl-C at any time to abort configuration and reboot system. To back track or make modifications to already entered values, complete input till end of section and answer no when prompted to apply configuration. Enter the configuration method. (console/gui) ? console Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the cluster. Continue (y/n) ? y Enter the admin password of the peer Fabric interconnect: Connecting to peer Fabric interconnect... done Retrieving config from peer Fabric interconnect... done Installer has determined that the peer Fabric Interconnect is running a different firmware version than the local Fabric. Cannot join cluster. Local Fabric Interconnect UCSM version : 3.1(2c) Kernel version : 5.0(3)N2(3.12b) System version : 5.0(3)N2(3.12b) local_model_no : UCS-FI-6332 Peer Fabric Interconnect UCSM version : 3.1(2c) Kernel version : 5.0(3)N2(3.12c) System version : 5.0(3)N2(3.12c) peer_model_no : UCS-FI-6332 Do you wish to update firmware on this Fabric Interconnect to the Peer's version? (y/n): y Updating firmware of Fabric Interconnect....... [ Please don't press Ctrl+c while updating firmware ] Updating images Please wait for firmware update to complete.

Install has been successful. Firmware Updation Successfully Completed. Please wait to enter the IP address Peer Fabric interconnect Mgmt0 IPv4 Address: xx.xx.xx.xx Peer Fabric interconnect Mgmt0 IPv4 Netmask: xx.xx.xx.xx Cluster IPv4 address : xx.xx.xx.xx Peer FI is IPv4 Cluster enabled. Please Provide Local Fabric Interconnect Mgmt0 IPv4 Address Physical Switch Mgmt0 IP address : xx.xx.xx.xx Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes Applying configuration. Please wait. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 51 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Replacing a Fabric Interconnect

Configuration file - Ok Step 14 Enter the show cluster extended-state command to revalidate the cluster state and verify that HA is ready. Example: FI-A# show cluster extended-state Cluster Id: 0x537d0580bf9911e0-0x8955000decd07984 A: UP, PRIMARY B: UP, SUBORDINATE A: memb state UP, lead state PRIMARY, mgmt services state: UP B: memb state UP, lead state SUBORDINATE, mgmt services state: UP heartbeat state PRIMARY_OK INTERNAL NETWORK INTERFACES: eth1, UP eth2, UP HA READY <<<<<<<<<<<<<←---------------- HA is READY Detailed state of the device selected for HA storage: Chassis 1, serial: FOX1344G1R1, state: active Chassis 2, serial: FOX1318GDKR, state: active FI-A# Step 15 Disable fabric evacuation and verify that traffic is flowing in both fabric interconnects. Step 16 Apply the port license to the fabric interconnect. Upgrading from a Cisco UCS 6248 UP Fabric Interconnect to a Cisco UCS 6296 UP Fabric Interconnect Follow these steps to migrate from a Cisco UCS 6248 UP fabric interconnect to a Cisco UCS 6296 UP fabric interconnect. Before you begin • Ensure that the Cisco UCS 6248 UP fabric interconnect and the Cisco UCS 6296 UP fabric interconnect are running the same version of firmware and UCS Manager system software. • Make a detailed record of the cabling between FEXes and fabric interconnects. You must preserve the physical port mapping to maintain the server pinning already configured and to minimize downtime. • As a best practice, perform a Full State backup and All Configuration backup before performing the upgrade. For information about these types of backups and detailed instructions on how to perform them from Cisco UCS Manager, see the Backup and Restore section in the Cisco UCS Manager Administration Management Guide. • From Cisco UCS Manager, execute the show tech support command and save the output to a file. • Verify that failover is enabled on all the vNICs and vHBAs as necessary. • Look at all the critical Cisco UCS Manager faults and ensure that they are expected. Correct the faults if needed. Be prepared to compare the list of faults the any faults that are present following the migration. They should be the same as before the migration. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 52 Installing the Cisco UCS 6200 Series Fabric Interconnect Upgrading from a Cisco UCS 6248 UP Fabric Interconnect to a Cisco UCS 6296 UP Fabric Interconnect

Data Path is Ready section in the Cisco UCS Manager Firmware Management Guide provides the steps to verify the data path. Procedure Step 1 Mount the replacement fabric interconnect into either the same rack or an adjacent rack. Step 2 Using Cisco UCS Manager, verify the state (subordinate or active) of the fabric interconnects. Step 3 Back up all software configuration information, including state and system configuration. Step 4 Follow the Fabric Evacuation procedure on the subordinate fabric interconnect to ensure there is no data traffic impact during the upgrade. Step 5 Enter the unconfigure all server ports command to remove the configuration from the server ports in the subordinate fabric interconnect. This steps removes the configuration from the ports that connect to the IOM and FEXes. Step 6 Power down the subordinate fabric interconnect by unplugging it from the power source. Step 7 Disconnect the cables from the IOM or FEXes to the subordinate fabric interconnect ports in slot 1 on the Cisco UCS 6248 UP fabric interconnect. Step 8 Connect the cables from the IOM to the corresponding ports on the Cisco UCS 6296 UP fabric interconnect. Step 9 Disconnect the L1/L2, M1 management, and console cables on the Cisco UCS 6248 UP fabric interconnect. Step 10 Connect the M1 management and console cables on the Cisco UCS 6296 UP fabric interconnect. Step 11 Connect the L1/L2 cables that were disconnected to the Cisco UCS 6296 UP fabric interconnect. Step 12 Connect the power cable to the Cisco UCS 6296 UP fabric interconnect. It will automatically boot and run POST tests. Step 13 Connect the cables into the corresponding ports on slot 1 of the Cisco UCS 6296 UP fabric interconnect. Step 14 Enable and configure all uplink ports and verify connectivity to network uplinks. Step 15 Enable and configure the server ports that were unconfigured in Step 5. The IOM will start Auto Sync, which will take approximately 15 minutes. Clear any faults related to failover or data path. Step 16 Perform an HA lead failover. Step 17 (Optional) If Fibre Channel ports were configured previously, reconfigure them and reboot the Cisco UCS 6296 UP Fabric Interconnect. The SAN and Port Channels section of the Cisco UCS Manager Storage Management Guide provides detailed configuration steps. Repacking the Cisco UCS Fabric Interconnect for Return Shipment If you need to return the fabric interconnect, remove the chassis from the rack by following the steps in Removing a Cisco UCS 6248 UP, on page 48 or Removing a Cisco UCS 6296, on page 48, and repack it for shipment. If possible, use the original packing materials and container to repack the chassis. Contact your Cisco customer service representative to arrange for return shipment to Cisco. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 53 OL-24475-03 Installing the Cisco UCS 6200 Series Fabric Interconnect Repacking the Cisco UCS Fabric Interconnect for Return Shipment

OL-24475-03 54 Installing the Cisco UCS 6200 Series Fabric Interconnect Repacking the Cisco UCS Fabric Interconnect for Return Shipment

Connecting the Cisco UCS 6200 Series Fabric Interconnect This chapter includes the following sections: • Preparing for Network Connections, on page 55 • Connecting to the Console Port, on page 56 • Connecting the Management Port, on page 58 • Connecting to an SFP+ Ethernet or Fibre Channel Port, on page 59 • Connecting to a Fibre Channel Port, on page 62 • Maintaining SFP Transceivers and Fiber-Optic Cables, on page 65 Preparing for Network Connections The Cisco UCS 6200 Series Fabric Interconnect provides the following types of ports: • RS-232 port—create a local management connection. • Ethernet ports, encrypted and unencrypted—to connect to a LAN. • Fibre Channel ports—connect to a SAN. When preparing your site for network connections to the Cisco UCS 6200 Series Fabric Interconnect, consider the following for each type of interface, and gather all the required equipment before connecting the ports: • Cabling required for each interface type • Distance limitations for each signal type • Additional interface equipment required You can save time and confusion when making cabling changes if you make copies of the Chassis and Module Information, on page 84 and keep them accurate to your current configuration. Tip Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 55 OL-24475-03

You can connect the console port to a modem. If you do not connect it to a modem, connect it either before powering on the system or after the system has completed the boot process. Caution The console port on a Cisco UCS fabric interconnect provides an RS232 serial connection over an RJ-45 interface that can be used to perform initial setup on a newly installed system that does not as yet have other connectivity options, perform software recovery tasks when other connectivity is unavailable, monitor network statistics and errors, configure SNMP agent parameters, or download software updates. Any device connected to this port must be capable of asynchronous transmission. The front connector ports are in a 2x2 stacked RJ-45 jack. The figure below shows a close-up view of the console, management, and cross-connect ports on the Cisco UCS 6248. Figure 32: Management Connector Ports, LEDs Network management port 5 Beaconing LED/button 1 Console port 6 System Status LED 2 USB port 7 UCS cross connect port L1 3 UCS cross connect port L2 4 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 56 Connecting the Cisco UCS 6200 Series Fabric Interconnect Connecting to the Console Port

186705 You can use the console port to perform the following: Before you begin You may have to acquire some or all of the following: • The Cisco serial console management cable (CAB-CONSOLE-RJ45=). • A USB to DB9 serial adapter and any drivers the adapter requires. • Terminal emulation software such as PuTTY, HyperTerminal or Procomm Plus. • A computer that can support VT100 terminal emulation. Procedure Step 1 Plug the RJ-45 end of the serial management cable into the Console port on the fabric interconnect, and connect the DB-9 male end into the serial port on a laptop or other computer. If the computer you will use does not have a serial port, you will need to use the Serial to USB adapter. Be sure to install the drivers for your adapter. Step 2 Start your terminal software. Step 3 Configure the terminal software as follows: • The COM port for the connection you are about to establish is the connection to the fabric interconnect. You may need to look in the computer's device manager to confirm this. • The other connection parameters are 9600 baud, 8 data bits, no parity, 1 stop bit. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 57 OL-24475-03 Connecting the Cisco UCS 6200 Series Fabric Interconnect Connecting to the Console Port

A session window will start and you will see one of the following prompts: loader> or switch(boot)# or FI-A(local-mgmt)# You now have terminal access. Depending on the prompt, you may have all Cisco UCS Manager CLI commands or a very abbreviated set of configuration commands. Connecting the Management Port To prevent an IP address conflict, do not connect the management port to the network until the initial configuration is complete. For configuration instructions, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html. Caution The Ethernet management connector port has an RJ-45 interface that will connect to an external hub, switch, or router. Procedure Step 1 Connect the appropriate modular cable to the Ethernet management connector port: • Use modular, RJ-45, straight-through UTP cables to connect the port to an Ethernet switch or hub. • Use a cross-over cable to connect to a router interface. Step 2 Connect the other end of the cable to the device. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 58 Connecting the Cisco UCS 6200 Series Fabric Interconnect Connecting the Management Port

Installing or Removing SFP Transceivers Excessively removing and installing an SFP transceiver can shorten its life. Do not remove and install transceivers more often than necessary. We recommend that you disconnect cables before installing or removing SFP transceivers to prevent damage to the cable or transceiver. Caution Use an SFP transceiver to connect to a Fibre Channel port. Installing a Transceiver Use an SFP+ transceiver to connect to an Ethernet or Fibre Channel port. Excessively installing and removing an SFP or SFP+ transceiver can shorten its life. Do not remove and install transceivers unless it is absolutely necessary. We recommend disconnecting cables before installing or removing transceivers to prevent damage to the cable or transceiver. Caution Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Step 2 Remove the dust cover from the port cage. Step 3 Remove the dust cover from the port end of the transceiver. Step 4 Insert the transceiver into the port: • If the transceiver has a Mylar tab, position the transceiver with the tab on the bottom, and then gently insert the transceiver into the port until it clicks into place. • If the transceiver has a bale clasp, position the transceiver with the clasp on the bottom, close the clasp by pushing it up over the transceiver, and then gently insert the transceiver into the port until it clicks into place. If the transceiver does not install easily, ensure that it is correctly positioned and the tab or clasp are in the correct position before continuing. Caution If you cannot install the cable into the transceiver, insert or leave the dust plug in the cable end of the transceiver. Note Removing a Transceiver Use an SFP+ transceiver to connect to an Ethernet or Fibre Channel port. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 59 OL-24475-03 Connecting the Cisco UCS 6200 Series Fabric Interconnect Connecting to an SFP+ Ethernet or Fibre Channel Port

transceivers unless it is absolutely necessary. We recommend disconnecting cables before installing or removing transceivers to prevent damage to the cable or transceiver. Caution Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Step 2 If a cable is installed in the transceiver: a) Record the cable and port connections for later reference. b) Press the release latch on the cable, grasp the connector near the connection point, and gently pull the connector from the transceiver. c) Insert a dust plug into the cable end of the transceiver. If the transceiver does not remove easily in the next step, push the transceiver completely in and then ensure that the latch is in the correct position before continuing. Step 3 Remove the transceiver from the port: • If the transceiver has a Mylar tab latch, gently pull the tab straight out (do not twist), and then pull the transceiver out of the port. • If the transceiver has a bale clasp latch, open the clasp by pressing it downwards, and then pull the transceiver out of the port. If you cannot remove the SFP+ transceiver, reseat it by returning the bale clasp to the up position. Press the SFP+ transceiver inward and upward into the cage. Next, lower the bale clasp and pull the SFP+ transceiver straight out with a slight upward lifting force. Be careful not to damage the port cage during this process. Note Step 4 Insert a dust cover into the port end of the transceiver and place the transceiver on an antistatic mat or into a static shielding bag if you plan to return it to the factory. Step 5 If another transceiver is not being installed, protect the optical cage by inserting a clean cover. Installing or Removing Cables into SFP or SFP+ Transceivers Installing a Cable into a Transceiver To prevent damage to the copper cables, do not place more tension on them than the rated limit and do not bend to a radius of less than 1 inch if there is no tension in the cable, or 2 inches if there is tension in the cable. Caution Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 60 Connecting the Cisco UCS 6200 Series Fabric Interconnect Installing or Removing Cables into SFP or SFP+ Transceivers

cable in the transceiver. Caution Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Step 2 Remove the dust cover from the connector on the cable. Step 3 Remove the dust cover from the cable end of the transceiver. Step 4 Align the cable connector with the transceiver and insert the connector into the transceiver until it clicks into place. If the cable does not install easily, ensure that it is correctly positioned before continuing. For instructions on verifying connectivity, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html Removing a Cable from a Transceiver To prevent damage to the copper cables, do not place more tension on them than the rated limit and do not bend to a radius of less than 1 inch if there is no tension in the cable, or 2 inches if there is tension in the cable. Caution When pulling a cable from a transceiver, grip the body of the connector. Do not pull on the jacket sleeve, because this action can compromise the fiber-optic termination in the connector. Caution If the cable does not remove easily, ensure that any latch present on the cable has been released before continuing. Caution Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Step 2 Press the release latch on the cable, grasp the connector near the connection point, and gently pull the connector from the transceiver. Step 3 Insert a dust plug into the cable end of the transceiver. Step 4 Insert a dust plug onto the end of the cable. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 61 OL-24475-03 Connecting the Cisco UCS 6200 Series Fabric Interconnect Removing a Cable from a Transceiver

Installing or Removing SFP+ Transceivers Installing an SFP Transceiver Excessively removing and installing an SFP transceiver can shorten its life. Do not remove and install transceivers more often than necessary. We recommend that you disconnect cables before installing or removing SFP transceivers to prevent damage to the cable or transceiver. Caution Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Step 2 Remove the dust cover from the port cage. Step 3 Remove the dust cover from the port end of the transceiver. Step 4 Insert the transceiver into the port: • If the transceiver has a Mylar tab, position the transceiver with the tab on the bottom, and then gently insert the transceiver into the port until it clicks into place. • If the transceiver has a bale clasp, position the transceiver with the clasp on the bottom, close the clasp by pushing it up over the transceiver, and then gently insert the transceiver into the port until it clicks into place. If the transceiver does not install easily, ensure that it is correctly positioned and the tab or clasp are in the correct position before continuing. Caution If you cannot install the cable into the transceiver, insert or leave the dust plug in the cable end of the transceiver. Note Removing an SFP Transceiver Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Step 2 If a cable is installed in the transceiver, do the following: a) Record the cable and port connections for later reference. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 62 Connecting the Cisco UCS 6200 Series Fabric Interconnect Connecting to a Fibre Channel Port

connector from the transceiver. c) Insert a dust plug into the cable end of the transceiver. Step 3 Remove the transceiver from the port: • If the transceiver has a Mylar tab latch, gently pull the tab straight out (do not twist), and then pull the transceiver out of the port. • If the transceiver has a bale clasp latch, open the clasp by pressing it downwards, and then pull the transceiver out of the port. If you have difficulty removing a bale clasp SFP transceiver, reseat the SFP by returning the bale clasp to the up position. Press the SFP inward and upward into the cage. Next, lower the bale clasp and pull the SFP straight out with a slight upward lifting force. Be careful not to damage the port cage during this process. Note Figure 34: Alternate Removal Method for Bale Clasp SFP Transceivers 192832 Step 4 Insert a dust cover into the port end of the transceiver and place the transceiver on an antistatic mat or into a static-shielding bag if you plan to return it to the factory. Step 5 If another transceiver is not being installed, protect the optical cage by inserting a clean cover. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 63 OL-24475-03 Connecting the Cisco UCS 6200 Series Fabric Interconnect Removing an SFP Transceiver

To prevent damage to the fiber-optic cables, do not place more tension on them than the rated limit and do not bend to a radius of less than 1 inch if there is no tension in the cable, or 2 inches if there is tension in the cable. Caution Installing a Cable into an SFP Transceiver To prevent possible damage to the cable or transceiver, install the transceiver in the port before installing the cable in the transceiver. Caution Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Step 2 Remove the dust cover from the connector on the cable. Step 3 Remove the dust cover from the cable end of the transceiver. Step 4 Align the cable connector with the transceiver and insert the connector into the transceiver until it clicks into place. If the cable does not install easily, ensure that it is correctly positioned before continuing. For instructions on verifying connectivity, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html Removing a Cable from an SFP Transceiver When pulling a cable from a transceiver, grip the body of the connector. Do not pull on the jacket sleeve, because this action can compromise the fiber-optic termination in the connector. Caution If the cable does not remove easily, ensure that any latch present on the cable has been released before continuing. Caution Procedure Step 1 Attach an ESD wrist strap and follow its instructions for use. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 64 Connecting the Cisco UCS 6200 Series Fabric Interconnect Installing or Removing Cables into SFP Transceivers

from the transceiver. Step 3 Insert a dust plug into the cable end of the transceiver. Step 4 Insert a dust plug onto the end of the cable. Maintaining SFP Transceivers and Fiber-Optic Cables SFP and SFP+ transceivers and fiber-optic cables must be kept clean and dust-free to maintain high signal accuracy and prevent damage to the connectors. Attenuation (loss of light) is increased by contamination and should be below 0.35 dB. Consider the following maintenance guidelines: • Transceivers are static sensitive. To prevent ESD damage, wear an ESD wrist strap that is connected to the chassis. • Do not remove and insert a transceiver more often than is necessary. Repeated removals and installation can shorten its useful life. • Keep all optical connections covered when not in use. If they become dusty, clean before using to prevent dust from scratching the fiber-optic cable ends. • Do not touch ends of connectors to prevent fingerprints and other contamination. • Clean regularly; the required frequency of cleaning depends upon the environment. In addition, clean connectors if they are exposed to dust or accidentally touched. Both wet and dry cleaning techniques can be effective; refer to fiber-optic cleaning procedures for your site. • Inspect routinely for dust and damage. If damage is suspected, clean and then inspect fiber ends under a microscope to determine if damage has occurred. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 65 OL-24475-03 Connecting the Cisco UCS 6200 Series Fabric Interconnect Maintaining SFP Transceivers and Fiber-Optic Cables

OL-24475-03 66 Connecting the Cisco UCS 6200 Series Fabric Interconnect Maintaining SFP Transceivers and Fiber-Optic Cables

Technical Specifications This appendix includes the following sections: • System Specifications, on page 67 • Power Specifications, on page 68 • Transceiver Specifications, on page 69 System Specifications Table 14: Physical Specifications Specification Description Physical (Cisco UCS 6248 UP) 1.72 in. x 17.3 in. x 29.5 in. (4.4 x 43.9 x 74.9 cm) Dimensions (H x W x D) Weight (with two power supplies and one expansion 35 lb (15.875 kg) module installed) Physical (Cisco UCS 6296 UP) 3.47 in. (8.8 cm) x 17.3 in. (43.9 cm) x 30.0 in. (76.2 cm) Dimensions (H x W x D) Weight (with two power supplies and two expansion 50 lb (22.680 kg) modules installed) Environmental 32 to 104°F (0 to 40°C) Temperature, operating -40 to 158°F (-40 to 70°C) Temperature, nonoperating 5 to 95% Humidity (RH), noncondensing 0 to 10000 ft (0 to 3000 m) Altitude Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 67 OL-24475-03

baseboard through cables. There are three connectors on the baseboard, two for power delivery, and one for power supply control signals. Table 15: Specifications for the Cisco UCS 6248UP DC Power Supply (UCS-PSU-6248UP-DC=) Cisco UCS 6248UP fabric interconnect DC Power Supply Properties 750 W Maximum output power -40 to -72 VDC Input voltage 25 A maximum @ -40 VDC input DC-input current at max voltage 88 to 92% (50 to 100% load) Efficiency 820 Maximum input KVA rating If a replacement DC connector is needed, a Phoenix Contact part number PC 5/ 2-STF-7,62, order number 1975697 or direct equivalent. Connector information is available at: http://eshop.phoenixcontact.de/phoenix/treeViewClick.do?UID=1975697 DC input terminal block 4 ms Output holdup time Yes RoHS compliance Yes Hot swappable 2497 BTU/hr (750 W) Heat dissipation Table 16: Specifications for the Cisco UCS 6248UP AC Power Supply(UCS-PSU-6248UP-AC=) Cisco UCS 6248UP fabric interconnect AC Power Supply Properties 750 W Maximum output power 90 to 264 VAC Input voltage 50 to 60 Hz Frequency 87 to 92% (50 to 100% load) Efficiency Yes RoHS compliance Yes Hot swappable 2497 BTU/hr (600 W) Heat dissipation Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 68 Technical Specifications Power Specifications

Specification Description 90 to 264 VAC AC-input voltage 50 to 60 Hz nominal (Range: 47 to 63 Hz) AC-input frequency 9.2 Amps @ 90 VAC AC-input current 828 VA @ 90 VAC Maximum Input VA 750 W @ 12 VDC (up to two power supplies) Maximum output power per power supply 35 A

General Specifications for Cisco Fibre Channel SFP Transceivers The table below lists the general specifications for Cisco Fibre Channel SFP transceivers at 4 Gbps. Table 20: General Specifications for Cisco Fibre Channel SFP Transceivers at 4 Gbps Short Wavelength Description LC Connector type 850 nm Wavelength MMF Fibre type 62.5 microns—164.04 yd (150 m) 50 microns—328.08 yd (300 m) Core size—Cable distance -9 to -2.5 dBM Transmit power 1 Approximate; actual distance may vary depending on fiber quality and other factors. Environmental Conditions and Power Requirements Specification for SFP Transceivers Table 21: Environmental Conditions and Power Requirements Specifications for SFP Transceivers Maximum Minimum Symbol Parameter 85°C (185°F) -40°C (-40°F) TS Storage temperature 2 70°C (158°F) 0°C (32°F) TC Case temperature 2, 95% 5% RH Relative humidity 2 3.5 V 3.1 V VCCT,R Module supply voltage 2 2 Absolute maximum ratings are those values beyond which damage to the device may occur if these limits are exceeded for other than a short period of time. 3 Functional performance is not intended, device reliability is not implied, and damage to the device may occur over an extended period of time between absolute maximum ratings and the recommended operating conditions. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 70 Technical Specifications General Specifications for Cisco Fibre Channel SFP Transceivers

Cable and Port Specifications This appendix includes the following sections: • Accessory Kit for the Cisco UCS Fabric Interconnect, on page 71 • Console Cable, on page 72 • Console Port, on page 72 • Supported AC Power Cords and Plugs, on page 73 Accessory Kit for the Cisco UCS Fabric Interconnect The Cisco UCS 6248 Fabric Interconnect accessory kit includes the following items: • 2 slider rails • 2 rack-mount guides • 2 rack-mount brackets • 12 M4 x 0.7 x 8-mm Phillips countersunk screws • 10 10-32 rack nuts • 10 10-32 x 3/4-inch Phillips pan-head screws • 1 console cable with an RJ-45-RS-232 adapter and a DB9 adapter • 1 ground lug kit • 1 ESD wrist strap • 1 power cord clip (a wire clip that is used to retain the power cord) • 1 pointer document (specifies where to find the online product documentation) Additional parts can be ordered from your customer service representative. Note Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 71 OL-24475-03

connect into the RS-232 console connection on a laptop. Console Cable 187491 Table 22: Console Cable Connector Pinouts Signal Name P2, DB-9 Pins P1, P1-45 Pins Signal Name CTS 8 1 RTS DSR 6 2 DTR ZXD 2 3 TXD GND 5 4 GND GND 5 5 GND TXD 3 6 ZXD DTR 4 7 DSR RTS 7 8 CTS Console Port The console port is an asynchronous RS-232 serial port with an RJ-45 connector. The table below lists the pinouts for the console port on the Cisco UCS 6200 Series Fabric Interconnect. Table 23: Console Port Pinouts Signal Pin RTS 1 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 72 Cable and Port Specifications Console Cable

DB-9 RS-232 Adapter Cable as well as a terminal program like Hyperterminal or PuTTY (set to connect at 9600 baud, 8 data bits, 1 stop bit, no parity). Supported AC Power Cords and Plugs Each power supply has a separate power cord. Standard power cords or jumper power cords are available for connection to a power distribution unit having IEC 60320 C13 outlet receptacles. The jumper power cords, for use in cabinets, are available as an optional alternative to the standard power cords. The standard power cords have an IEC C13 connector on the end that plugs into the power supplies. The optional jumper power cords have an IEC C13 connector on the end that plugs into the power supplies, and an IEC C14 connector on the end that plugs into an IEC C13 outlet receptacle. Only the regular power cords or jumper power cords provided with the chassis are supported. Note Argentina Power Cord—SFS-250V-10A-AR Plug—250 VAC 10 A IRAM 2073 Length—8.2 feet / 2.5 meters Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 73 OL-24475-03 Cable and Port Specifications Supported AC Power Cords and Plugs

186571 2500 mm Cordset rating: 10 A, 250/500 V MAX Length: 8.2 ft Plug: EL 219 (IRAM 2073) Connector: EL 701 (IEC60320/C13) Australia and New Zealand Power Cord—CAB-9K10A-AU Plug—250 VAC 10 A 3112 Length—8.2 feet / 2.5 meters Figure 36: CAB-9K10A-AU Plug: EL 206 A.S. 3112-2000) Cordset rating: 10 A, 250 V/500V Length: 2500mm 186581 Connector: EL 701C (IEC 60320/C15) Peoples Republic of China Power Cord—SFS-250V-10A-CN Plug—250 VAC 10 A GB 2009 Length—8.2 feet / 2.5 meters Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 74 Cable and Port Specifications Australia and New Zealand

Cordset rating 10A, 250V (2500 mm) Plug: EL 218 (CCEE GB2009) 186573 Connector: EL 701 (IEC60320/C13) Taiwan Power Cord—CAB-ACTW Plug—250 VAC 10 A, IEC 60320-C13 Length—7.5 feet / 2.3 meters Europe Power Cord—CAB-9K10A-EU Plug—250 VAC 10 A M 2511 Length—8.2 feet / 2.5 meters Figure 38: CAB-9K10A-EU Connector: VSCC15 Cordset rating: 10A/16 A, 250 V Length: 8 ft 2 in. (2.5 m) Plug: M2511 186576 India, South Africa, and United Arab Emirates Power Cord—SFS-250V-10A-ID Plug—250 VAC 16A EL-208 Length—8.2 feet / 2.5 meters Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 75 OL-24475-03 Cable and Port Specifications Taiwan

OVE Cordset rating 16A, 250V (2500mm) Plug: EL 208 187490 Connector: EL 701 Israel Power Cord—SFS-250V-10A-IS Plug—250 VAC 10 A SI32 Length—8.2 feet / 2.5 meters Figure 40: SFS-250V-10A-IS Cordset rating 10A, 250V/500V MAX (2500 mm) Plug: EL 212 (SI-32) 186574 Connector: EL 701B (IEC60320/C13) EL-212 16A 250V Italy Power Cord—CAB-9K10A-IT Plug—250 VAC 10 A CEI 23-16 Length—8.2 feet / 2.5 meters Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 76 Cable and Port Specifications Israel

Plug: I/3G (CEI 23-16) Connector C15M (EN60320/C15 ) Cordset rating: 10 A, 250 V Length: 8 ft 2 in. (2.5 m) 186575 North America CAB-AC-250V/13A Power Cord—CAB-AC-250V/13A Plug—250 VAC 13 A IEC60320 Length—6.6 feet / 2.0 meters Figure 42: CAB-AC-250V/13A CAB-N5K6A-NA Power Cord—CAB-N5K6A-NA Plug—250 VAC 13 A NEMA 6-15 Length—8.2 feet / 2.5 meters Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 77 OL-24475-03 Cable and Port Specifications North America

Switzerland Power Cord—CAB-9K10A-SW Plug—250 VAC 10 A MP232 Length—8.2 feet / 2.5 meters Figure 44: CAB-9K10A-SW Cordset rating: 10 A, 250 V Length: 8.2 ft 186570 Plug: NEMA 6-15P Connector: IEC60320/C13 United Kingdom Power Cord—CAB-9K10A-UK Plug—250 VAC 10 A BS1363 (13 A fuse) Length—8.2 feet / 2.5 meters Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 78 Cable and Port Specifications Switzerland

Plug: Cordset rating: 10 A, 250 V/500 V MAX Length: 2500mm 186580 Connector: EL 701C (EN 60320/C15) EL 210 (BS 1363A) 13 AMP fuse Cabinet Jumper Power Cords Jumper Power Cord—CAB-C13-C14-JMPR Plug— 250 VAC 10 A, C13-C14 Connectors Length—2.2 feet / 0.7 meters Figure 46: CAB-C13-C14-JMPR Jumper Power Cord—CAB-C13-C14-AC Plug— 250 VAC 10 A, C13-C14 Connectors Length—9.8 feet / 3 meters Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 79 OL-24475-03 Cable and Port Specifications Cabinet Jumper Power Cords

Jumper Power Cord—CAB-C13-C14-2M Plug— 250 VAC 10 A, C13-C14 Connectors Length—6.6 feet / 2 meters Figure 48: CAB-C13-C14-2M Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 80 Cable and Port Specifications Cable and Port Specifications

Site Planning and Maintenance Records This appendix includes the following sections: • Site Preparation Checklist, on page 81 • Contact and Site Information, on page 83 • Chassis and Module Information, on page 84 Site Preparation Checklist Planning the location and layout of your equipment is essential for successful system operation, ventilation, and accessibility. Consider heat dissipation when sizing the air-conditioning requirements for an installation. Table 24: Site Planning Checklist Date Time Verified By Planning Activity Task No. Space evaluation: • Space and layout • Floor covering • Impact and vibration • Lighting • Maintenance access 1 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 81 OL-24475-03

2 Power evaluation: • Input power type • Power receptacles • Receptacle proximity to the equipment • Dedicated circuit for power supply • Dedicated (separate) circuits for redundant power supplies • UPS for power failures 3 Grounding evaluation: • Circuit breaker size • CO ground (AC- powered systems) 4 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 82 Site Planning and Maintenance Records Site Planning and Maintenance Records

equipment evaluation: • Cable type • Connector type • Cable distance limitations • Interface equipment (transceivers) 5 EMI evaluation: • Distance limitations for signaling • Site wiring • RFI levels 6 4 Verify that the power supply installed in the chassis has a dedicated AC source circuit. 5 UPS = uninterruptible power supply. 6 EMI = electromagnetic interference. 7 RFI = radio frequency interference. Contact and Site Information Use the following worksheet to record contact and site information. Table 25: Contact and Site Information Contact person Contact phone Contact e-mail Building/site name Data center location Floor location Address (line 1) Address (line 2) Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 83 OL-24475-03 Site Planning and Maintenance Records Contact and Site Information

Use the following worksheets to record information about the chassis and modules. Contract Number_______________________________________________ Chassis Serial Number___________________________________________ Product Number________________________________________________ Table 26: Network-Related Information System IP address System IP netmask Hostname Domain name IP broadcast address Gateway/router address DNS address Modem telephone number Table 27: Module Information Notes Module Serial Number Module Type Slot Fixed 1 Expansion 2 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 84 Site Planning and Maintenance Records Chassis and Module Information

Connected to FabricInterconnectAorB Connection Notes Port Channel Group LAN or SAN Pin Group Port FEX Chassis Port Slot 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 85 OL-24475-03 Site Planning and Maintenance Records Site Planning and Maintenance Records

Pin Group Port FEX Chassis Port Slot 26 27 28 29 30 31 32 1 2 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 86 Site Planning and Maintenance Records Site Planning and Maintenance Records

Troubleshooting Hardware Components This appendix includes the following sections: • Overview, on page 87 • SNMP Traps, on page 87 • Server Port Link State Transitions, on page 87 • System Hardware Best Practices, on page 88 Overview The key to success when troubleshooting the system hardware is to isolate the problem to a specific system component. The first step is to compare what the system is doing to what it should be doing. Because a startup problem can usually be attributed to a single component, it is more efficient to isolate the problem to a subsystem rather than troubleshoot each separate component in the system. Problems with the initial power up are often caused by a module that is not firmly connected to the backplane or a power supply that has been disconnected from the power cord connector. Overheating can also cause problems with the system, though typically only after the system has been operating for an extended period of time. The most common cause of overheating is the failure of a fan module. SNMP Traps You can set SNMP traps to monitor fans, power supplies, and temperature settings, or to test a call home application. For details, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at the following URL: http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html Server Port Link State Transitions When cables of server ports are removed from the Fabric Interconnect, Cisco UCS Manager displays a message indicating that the link is down because of link failure or not connected. Before the cable is plugged back in, a false "[F0276][cleared]" event might be logged after 20 to 30 seconds. When a port that is Up gets disconnected, it transitions to a Link Down status. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide 87 OL-24475-03

• If an SFP is inserted, the port status remains at Link Down, and the F0276 event is not cleared. This situation occurs when the SFP cable is plugged into the Fabric Interconnect port, and the SFP at the I/O module is disconnected. The "Link down or not connected" fault appears, and the fault is not cleared until the SFP is plugged back into the I/O module port. • If there is no SFP detected, the port status transitions from Link Down status to SFP Not Present. This situation occurs when the SFP is plugged into the I/O module and the other end is not plugged into the Fabric Interconnect port that is configured to be the server. • When the port status is no longer Link Down, the F0276 is cleared. System Hardware Best Practices Installation Best Practices When installing the chassis, follow these best practices: • Plan your site configuration and prepare the site before installing the chassis. • Verify that you have the appropriate power supplies for your chassis configuration. • Install the chassis following the rack and airflow guidelines presented in this guide. • Verify that the chassis is adequately grounded. Initialization Best Practices When the initial system boot is complete, verify the following: • Power supplies are supplying power to the system. • Fan modules are operating normally. • The system software boots successfully. System Operation Best Practices To ensure proper operation of your system, take the following actions: • Make a copy of the running configuration to CompactFlash for a safe backup. • Never use the init system CLI command unless you understand that you will lose the running and startup configuration as well as the files stored on bootflash:. Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide OL-24475-03 88 Troubleshooting Hardware Components System Hardware Best Practices