Intel S2600IP Technical Product Specification

Intel® Workstation Board W2600CR Technical Product Specification Intel order number G34153-003 Revision 1.1 June, 2012 Enterprise Platforms and Services Division - Marketing

Revision 1.1 Intel order number G34153-003 ii Revision History Date Revision Number Modifications February, 2012 1.0 Initial release. June, 2012 1.1  Updated thermal management and environmental summary.  Updated onboard power connectors information. Disclaimers Information in this document is provided in connection with Intel® products. No license, express or implied, by estoppel or otherwise, to any intellectual property rights is granted by this document. Except as provided in Intel's Terms and Conditions of Sale for such products, Intel assumes no liability whatsoever, and Intel disclaims any express or implied warranty, relating to sale and/or use of Intel products including liability or warranties relating to fitness for a particular purpose, merchantability, or infringement of any patent, copyright or other intellectual property right. Intel products are not intended for use in medical, lifesaving, or life sustaining applications. Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. This document contains information on products in the design phase of development. Do not finalize a design with this information. Revised information will be published when the product is available. Verify with your local sales office that you have the latest datasheet before finalizing a design. The Intel® Server Board S2600IP and Intel® Workstation Board W2600CR may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. This document and the software described in it is furnished under license and may only be used or copied in accordance with the terms of the license. The information in this manual is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Intel Corporation. Intel Corporation assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system, or transmitted in any form or by any means without the express written consent of Intel Corporation. Intel, Pentium, Itanium, and Xeon are trademarks or registered trademarks of Intel Corporation. *Other brands and names may be claimed as the property of others. Copyright © 2012 Intel Corporation

Table of Contents Revision 1.1 Intel order number G34153-003 iii Table of Contents 1. Introduction ........................................................................................................................ 1 1.1 Chapter Outline ...................................................................................................... 1 1.2 Server Board Use Disclaimer ................................................................................. 1 2. Product Overview ............................................................................................................... 2 2.1 Product Feature Set ............................................................................................... 3 2.1.1 Main Board Connector and Component Layout ...................................................... 5 2.1.2 Main Board Rear I/O Layout ................................................................................... 8 2.1.3 Server Board Mechanical Drawings ..................................................................... 10 3. Functional Architecture ................................................................................................... 19 3.1 Processor Support ............................................................................................... 20 3.1.1 Processor Socket Assembly ................................................................................. 20 3.1.2 Processor Population Rules ................................................................................. 22 3.1.3 Processor Initialization Error Summary................................................................. 22 3.2 Processor Functions Overview ............................................................................. 25 3.2.1 Intel® QuickPath Interconnect ............................................................................... 25 3.2.2 Integrated Memory Controller (IMC) and Memory Subsystem .............................. 26 3.2.3 Processor Integrated I/O Module (IIO) .................................................................. 32 3.3 Intel® C600-A Chipset Functional overview .......................................................... 33 3.3.1 Digital Media Interface (DMI) ................................................................................ 34 3.3.2 PCI Express* Interface ......................................................................................... 34 3.3.3 Serial ATA (SATA) Controller ............................................................................... 34 3.3.4 Serial Attached SCSI (SAS)/SATA (SCU) Controller ............................................ 35 3.3.5 AHCI .................................................................................................................... 35 3.3.6 Rapid Storage Technology ................................................................................... 35 3.3.7 PCI Interface ........................................................................................................ 36 3.3.8 Low Pin Count (LPC) Interface ............................................................................. 36 3.3.9 Serial Peripheral Interface (SPI) ........................................................................... 36 3.3.10 Compatibility Modules (DMA Controller, Timer/Counters, Interrupt Controller) ..... 36 3.3.11 Advanced Programmable Interrupt Controller (APIC) ........................................... 36 3.3.12 Universal Serial Bus (USB) Controllers ................................................................ 36 3.3.13 Gigabit Ethernet Controller ................................................................................... 37 3.3.14 RTC ..................................................................................................................... 37 3.3.15 GPIO .................................................................................................................... 37 3.3.16 Enhanced Power Management ............................................................................ 37 3.3.17 Manageability ....................................................................................................... 37 3.3.18 System Management Bus (SMBus* 2.0) .............................................................. 37 3.3.19 Integrated NVSRAM controller ............................................................................. 38 3.3.20 Virtualization Technology for Directed I/O (Intel® VT-d) ........................................ 38 3.3.21 JTAG Boundary-Scan .......................................................................................... 38

Intel® Server Board S2600IP and Intel® Workstation Board W2600CR TPS Revision 1.1 Intel order number G34153-003 iv 3.3.22 KVM/Serial Over Lan (SOL) Function ................................................................... 38 3.3.23 On-board SAS/SATA Support and Options .......................................................... 38 3.4 PCI Subsystem .................................................................................................... 39 3.5 Network Interface Controller ................................................................................. 40 3.5.1 Network Interface ................................................................................................. 40 3.6 I/O Module Support (not avaiable for Intel® Server Board S2600IP4L and Intel® Workstation Board W2600CR2L) ........................................................................................... 41 3.7 SAS ROC Module Support ................................................................................... 42 3.8 HD Audio Support (Intel® Workstation Board W2600CR Only) ............................. 43 3.9 USB3.0 Support (Intel® Workstation Board W2600CR Only) ................................ 44 3.10 IEEE1394b Support (Intel® Workstation Board W2600CR Only) .......................... 45 4. Platform Management Overview ..................................................................................... 46 4.1 Server Management Function Architecture .......................................................... 46 4.1.1 Feature Support ................................................................................................... 46 4.1.2 Basic and Advanced Features .............................................................................. 48 4.1.3 Integrated BMC Hardware: Emulex* Pilot III ......................................................... 49 4.2 Server Management Functional Specifications ..................................................... 51 4.2.1 BMC Internal Timestamp Clock ............................................................................ 51 4.2.2 System Event Log (SEL) ...................................................................................... 52 4.2.3 Sensor Data Record (SDR) Repository ................................................................ 52 4.2.4 Field Replaceable Unit (FRU) Inventory Device ................................................... 52 4.2.5 BMC Beep Codes ................................................................................................ 52 4.2.6 Diagnostic Interrupt (NMI) Button ......................................................................... 53 4.2.7 BMC Watchdog .................................................................................................... 53 4.3 Sensor Monitoring ................................................................................................ 54 4.3.1 Overview .............................................................................................................. 54 4.3.2 Core Sensor List for EPSD Platforms Based on Intel® Xeon® Processor E5 4600/2600/2400/1600/1400 Product Families ..................................................................... 54 4.3.3 BMC System Management Health Monitoring ...................................................... 56 4.3.4 Processor Sensors ............................................................................................... 56 4.4 Thermal and Acoustic Management ..................................................................... 60 4.4.1 Set Throttling Mode .............................................................................................. 60 4.4.2 Altitude ................................................................................................................. 60 4.4.3 Set Fan Profile ..................................................................................................... 60 4.4.4 Fan PWM Offset ................................................................................................... 61 4.4.5 Quiet Fan Idle Mode ............................................................................................. 61 4.4.6 Thermal Sensor Input for Fan Speed Control ....................................................... 61 4.4.7 Power Supply Status\Health Sensors ................................................................... 62 4.4.8 System Event Sensor ........................................................................................... 63 4.5 Platform Management Interface ........................................................................... 65 4.5.1 Channel Management .......................................................................................... 65

Table of Contents Revision 1.1 Intel order number G34153-003 v 4.5.2 User Model ........................................................................................................... 65 4.5.3 LAN Interface ....................................................................................................... 66 4.6 BMC Firmware Update ......................................................................................... 76 4.6.1 The BMC firmware release Number ..................................................................... 76 4.6.2 Boot Recovery Mode ............................................................................................ 76 4.6.3 Force Firmware Update Jumper ........................................................................... 77 4.6.4 Fast Firmware Update over USB .......................................................................... 77 4.7 Advanced Management Feature Support ............................................................. 78 4.7.1 Enabling Advanced Management Features .......................................................... 78 4.7.2 Keyboard, Video, Mouse (KVM) Redirection ........................................................ 78 4.7.3 Media Redirection ................................................................................................ 79 4.8 Intel® Intelligent Power Node Manager (NM) ........................................................ 81 4.8.1 Overview .............................................................................................................. 81 4.9 EU Lot 6 Mode ..................................................................................................... 82 5. System Security ................................................................................................................ 84 5.1 BIOS Password Protection ................................................................................... 84 5.2 Trusted Platform Module (TPM) Support .............................................................. 85 5.2.1 TPM security BIOS ............................................................................................... 85 5.2.2 Physical Presence ................................................................................................ 86 5.2.3 TPM Security Setup Options ................................................................................ 86 5.3 Intel® Trusted Execution Technology .................................................................... 88 6. Connector/Header Locations and Pin-outs .................................................................... 90 6.1 Power Connectors ................................................................................................ 90 6.1.1 Main Power Connector ......................................................................................... 90 6.1.2 CPU Power Connectors ....................................................................................... 90 6.1.3 Add-in Card Power Connectors ............................................................................ 90 6.2 Front Panel Header and Connectors .................................................................... 91 6.2.1 Front Panel Header .............................................................................................. 91 6.2.2 Front Panel USB Connector ................................................................................. 92 6.2.3 Local Control Panel Connector ............................................................................. 92 6.3 On Board Storage Connectors ............................................................................. 92 6.3.1 SATA Connectors: 6Gbps .................................................................................... 92 6.3.2 Multiport Mini-SAS/SATA Connectors .................................................................. 93 6.3.3 Intel® RAID C600 Upgrade Key Connector ........................................................... 94 6.3.4 HSBP_I2C Header ............................................................................................... 94 6.3.5 HDD LED Header ................................................................................................. 94 6.3.6 Internal Type- A USB Connector .......................................................................... 94 6.3.7 Internal eUSB SSD Header .................................................................................. 95 6.4 Management and Security Connectors ................................................................ 95 6.4.1 RMM4_Lite Connector ......................................................................................... 95 6.4.2 RMM4_NIC Connector ......................................................................................... 95

Intel® Server Board S2600IP and Intel® Workstation Board W2600CR TPS Revision 1.1 Intel order number G34153-003 vi 6.4.3 TPM Connector .................................................................................................... 96 6.4.4 PMBus* Connector ............................................................................................... 96 6.4.5 Chassis Intrusion Header ..................................................................................... 96 6.4.6 IPMB Connector ................................................................................................... 96 6.5 FAN Connectors ................................................................................................... 96 6.5.1 System FAN Connectors ...................................................................................... 97 6.5.2 CPU FAN Connector ............................................................................................ 97 6.6 Serial Port and Video Connectors ........................................................................ 97 6.6.1 Serial Port A Connector (DB9, for Intel® Server Board S2600IP only) .................. 97 6.6.2 Serial Port B Connector ........................................................................................ 98 6.6.3 Video Connector (For Intel® Server Board S2600IP only) ..................................... 98 6.6.4 Internal Video header (For Intel® Workstation Board W2600CR only) .................. 99 7. Jumper Blocks ................................................................................................................ 100 7.1 BIOS Default and Password Reset Usage Procedure ........................................ 101 7.1.1 Set BIOS to default (Clearing the CMOS) ........................................................... 101 7.1.2 Clearing the Password ....................................................................................... 101 7.2 Force BMC Update Procedure ........................................................................... 102 7.3 BIOS Recovery Jumper ...................................................................................... 102 7.4 ME Force Update Jumper .................................................................................. 103 8. Intel® Light Guided Diagnostics .................................................................................... 105 8.1 5-volt Stand-by LED ........................................................................................... 105 8.2 Fan Fault LED .................................................................................................... 105 8.3 CPU Fault LED ................................................................................................... 105 8.4 DIMM Fault LEDs ............................................................................................... 107 8.5 System ID LED, System Status LED and POST Code Diagnostic LEDs ............ 107 8.5.1 System ID LED ................................................................................................... 108 8.5.2 System Status LED ............................................................................................ 109 8.5.3 POST Code Diagnostic LEDs ............................................................................. 110 9. Design and Environmental Specifications .................................................................... 111 9.1 Products Design Specifications .......................................................................... 111 9.2 MTBF ................................................................................................................. 111 9.3 Server Board Power Requirements .................................................................... 114 9.3.1 Processor Power Support................................................................................... 115 9.4 Power Supply Output Requirements .................................................................. 115 9.4.1 Output Power/Currents ....................................................................................... 115 9.4.2 Cross Loading .................................................................................................... 115 9.4.3 Standby Output .................................................................................................. 116 9.4.4 Voltage Regulation ............................................................................................. 116 9.4.5 Dynamic Loading ............................................................................................... 116 9.4.6 Capacitive Loading ............................................................................................. 117 9.4.7 Grounding .......................................................................................................... 117

Table of Contents Revision 1.1 Intel order number G34153-003 vii 9.4.8 Closed loop stability ........................................................................................... 117 9.4.9 Residual Voltage Immunity in Standby mode ..................................................... 117 9.4.10 Common Mode Noise ......................................................................................... 117 9.4.11 Soft Starting ....................................................................................................... 118 9.4.12 Zero Load Stability Requirements ...................................................................... 118 9.4.13 Ripple/Noise ....................................................................................................... 118 9.4.14 Timing Requirements ......................................................................................... 118 Appendix A: Integration and Usage Tips ............................................................................ 121 Appendix B: BMC Sensor Tables ......................................................................................... 122 Appendix C: Platform Specific BMC Appendix ................................................................... 147 Introduction ....................................................................................................................... 147 Product ID ........................................................................................................................ 147 IPMI Channel ID Assignments .......................................................................................... 147 ACPI S3 Sleep State Support ........................................................................................... 147 Processor Support for Intel® Server Board S2600IP ......................................................... 147 Supported Chassis ........................................................................................................... 147 Chassis-specific sensors .................................................................................................. 148 Hot-plug fan support ......................................................................................................... 148 Fan redundancy support ................................................................................................... 148 Fan domain definition ....................................................................................................... 149 HSC Availability ................................................................................................................ 151 Power unit support ............................................................................................................ 152 Redundant Fans only for Intel® Server Chassis ................................................................ 152 Fan Fault LED support...................................................................................................... 152 Memory Throttling support ................................................................................................ 153 Introduction ....................................................................................................................... 153 Product ID ........................................................................................................................ 153 IPMI Channel ID Assignments .......................................................................................... 153 ACPI S3 Sleep State Support ........................................................................................... 153 Processor Support for Intel® Server Board W2600CR ...................................................... 154 Supported Chassis ........................................................................................................... 154 Chassis-specific sensors .................................................................................................. 154 Hot-plug fan support ......................................................................................................... 154 Fan redundancy support ................................................................................................... 154 Board specific feature ....................................................................................................... 154 Fan domain definition ....................................................................................................... 154 HSC Availability ................................................................................................................ 155

Intel® Server Board S2600IP and Intel® Workstation Board W2600CR TPS Revision 1.1 Intel order number G34153-003 viii Power unit support ............................................................................................................ 155 Redundant Fans only for Intel® Server Chassis ................................................................ 155 Fan Fault LED support...................................................................................................... 155 Memory Throttling support ................................................................................................ 155 Appendix D: POST Code Diagnostic LED Decoder ............................................................ 156 Appendix E: POST Error Messages and Handling.............................................................. 162 Glossary ................................................................................................................................ 169 Reference Documents .......................................................................................................... 172

List of Figures Revision 1.1 Intel order number G34153-003 ix List of Figures Figure 1. Intel® Server Board S2600IP ........................................................................................ 2 Figure 2. Intel® Workstation Board W2600CR ............................................................................. 2 Figure 3. Intel® Server Board S2600IP Major Components ......................................................... 6 Figure 4. Intel® Server Board S2600IP Rear I/O Layout .............................................................. 8 Figure 5. Intel® Workstation Board W2600CR Rear I/O Layout ................................................... 9 Figure 6. Mounting Hole Locations (1 of 2) ................................................................................ 10 Figure 7. Mounting Hole Locations (2 of 2) ................................................................................ 11 Figure 8. Major Connector Pin-1 Locations (1 of 3) ................................................................... 12 Figure 9. Major Connector Pin-1 Locations (2 of 3) ................................................................... 13 Figure 10. Major Connector Pin-1 Locations (3 of 3) ................................................................. 14 Figure 11. Primary Side Keep-out Zone .................................................................................... 16 Figure 12. Primary Side Card-Side Keep-out Zone ................................................................... 17 Figure 13. Second Side Keep-out Zone .................................................................................... 18 Figure 14. Intel® Server Board S2600IP Functional Block Diagram ........................................... 19 Figure 15. Intel® Workstation Board W2600CR Functional Block Diagram ................................ 20 Figure 16. Processor Socket Assembly ..................................................................................... 21 Figure 17. Processor Socket ILM .............................................................................................. 21 Figure 18. Integrated Memory Controller Functional Block Diagram .......................................... 26 Figure 19. DIMM Slot Layout ..................................................................................................... 30 Figure 20. External RJ45 NIC Port LED Definition .................................................................... 41 Figure 21. Supported I/O Module Options ................................................................................. 42 Figure 22. USB3.0 Discrete Host Controller Block Diagram ...................................................... 44 Figure 23. IEEE 1394b Discrete Host Controller Block Diagram ............................................... 45 Figure 24. Integrated BMC Hardware ........................................................................................ 51 Figure 25. High-level Fan Speed Control Process ..................................................................... 62 Figure 26. Setup Utility – TPM Configuration Screen ................................................................ 87 Figure 27. Video Connector Pin-out .......................................................................................... 98 Figure 28. Jumper Blocks (J1C3, J1F1, J1F2, J2F3, J2F2) ..................................................... 100 Figure 29. Stand by LED, Fan Fault LED and CPU Fault LED Location .................................. 106 Figure 30. DIMM Fault LED’s Location .................................................................................... 107 Figure 31. Location of System Status, System ID and POST Code Diagnostic LEDs.............. 108 Figure 32. Power Distribution Block Diagram .......................................................................... 114 Figure 33. Differential Noise test setup ................................................................................... 118 Figure 34. Output Voltage Timing ........................................................................................... 119 Figure 35. Turn On/Off Timing (Power Supply Signals) ........................................................... 120 Figure 36. POST Code Diagnostic LED Decoder .................................................................... 156

Intel® Server Board S2600IP and Intel® Workstation Board W2600CR TPS Revision 1.1 Intel order number G34153-003 x List of Tables Table 1. Product Feature Set ...................................................................................................... 3 Table 2. Mixed Processor Configurations Error Summary. ........................................................ 23 Table 3. UDIMM Support Guidelines ......................................................................................... 27 Table 4. RDIMM Support Guidelines ......................................................................................... 28 Table 5. LRDIMM Support Guidelines ....................................................................................... 28 Table 6. DIMM Nomenclature ................................................................................................... 29 Table 7. Intel® RAID C600 Upgrade Key Options ...................................................................... 35 Table 8. Intel® Server Board S2600IP/Workstation Board W2600CR PCI Bus Segment Characteristics .................................................................................................................... 40 Table 9. Supported Intel® Integrated RAID Modules .................................................................. 43 Table 10. Standard Speaker Channels ..................................................................................... 44 Table 11. Basic and Advanced Features ................................................................................... 49 Table 12. BMC Beep Codes...................................................................................................... 52 Table 13. NMI Signal Generation and Event Logging ................................................................ 53 Table 14. Supported BMC FW Health Sensor Offsets ............................................................... 56 Table 15. Processor Sensors .................................................................................................... 56 Table 16. Processor Status Sensor Implementation .................................................................. 57 Table 17. Supported ERR2 Timeout Sensor Offsets ................................................................. 58 Table 18. Supported CATERR Sensor Offsets .......................................................................... 59 Table 19. Support MSID Mismatch Sensor Offsets ................................................................... 60 Table 20. Supported Power Supply Status Sensor Offsets........................................................ 62 Table 21. Support System Event Sensor Offsets ...................................................................... 64 Table 22. Supported Firmware Update Status Sensor Offsets .................................................. 64 Table 23. Standard Channel Assignments ................................................................................ 65 Table 24. Supported RMCP+ Cipher Suites .............................................................................. 66 Table 25. Supported RMCP+ Payload Types ............................................................................ 67 Table 26. Factory Configured PEF Table Entries ...................................................................... 71 Table 27. Enabling Advanced Management Features ............................................................... 78 Table 28. TSetup Utility – Security Configuration Screen Fields ................................................ 88 Table 29. Main Power Connector Pin-out .................................................................................. 90 Table 30. CPU_1 Power Connector Pin-out .............................................................................. 90 Table 31. CPU_2 Power Connector Pin-out .............................................................................. 90 Table 32. Add-in Slot Power Pin-input ("OPT_12V_PWR") ....................................................... 91 Table 33. Front Panel Header Pin-out ....................................................................................... 91 Table 34. Front Panel USB Connector Pin-out .......................................................................... 92 Table 35. Local Front Panel Connector Pin-out ......................................................................... 92 Table 36. SATA 6Gbps Connector Pin-out ................................................................................ 92

List of Tables Revision 1.1 Intel order number G34153-003 xi Table 37. Multiport SAS/SATA Connector Pin-out ("SCU_0 (0-3)") ........................................... 93 Table 38. Multiport SAS/SATA Connector Pin-out ("SCU_1 (4-7)") ........................................... 93 Table 39. Intel® RAID C600 Upgrade Key Connector Pin-out .................................................... 94 Table 40. HSBP_I2C Header Pin-out ........................................................................................ 94 Table 41. HDD LED Header Pin-out .......................................................................................... 94 Table 42. Type-A USB Connector Pin-out ................................................................................. 94 Table 43. eUSB SSD Header Pin-out ........................................................................................ 95 Table 44. RMM4_Lite Connector Pin-out .................................................................................. 95 Table 45. RMM4_NIC Connector Pin-out .................................................................................. 95 Table 46. TPM Connector Pin-out ............................................................................................. 96 Table 47. PMBus* Connector Pin-out ........................................................................................ 96 Table 48. Chassis Intrusion Header Pin-out .............................................................................. 96 Table 49. IPMB Connector Pin-out ............................................................................................ 96 Table 50. 6-pin System FAN Connector Pin-out ........................................................................ 97 Table 51. 4-pin System FAN Connector Pin-out ........................................................................ 97 Table 52. CPU FAN Connector Pin-out ..................................................................................... 97 Table 53. Serial Port A Connector Pin-out ................................................................................. 97 Table 54. Serial Port B Connector Pin-out ................................................................................. 98 Table 55. Video Connector Pin-out details ................................................................................ 98 Table 56. Server Board Jumpers (J1C3, J1F1, J1F2, J2F3, J2F2) .......................................... 100 Table 57. System Status LED ................................................................................................. 109 Table 58. POST Code Diagnostic LEDs .................................................................................. 110 Table 59. Server Board Design Specifications ........................................................................ 111 Table 60. Intel® Server Board S2600IP MTBF Estimate .......................................................... 112 Table 61. Intel® Workstation Board W2600CR MTBF Estimate ............................................... 112 Table 62. Intel® Xeon® Processor Dual Processor TDP profile ................................................ 115 Table 63. Minimum Load Ratings ............................................................................................ 115 Table 64. Loading Conditions .................................................................................................. 116 Table 65.Voltage Regulation Limits ......................................................................................... 116 Table 66. Transient Load Requirements ................................................................................. 116 Table 67. Capacitive Loading Conditions ................................................................................ 117 Table 68. Ripples and Noise ................................................................................................... 118 Table 69. Output Voltage Timing ............................................................................................. 119 Table 70. Turn On/Off Timing .................................................................................................. 119 Table 71. Integrated BMC Core Sensors................................................................................. 124 Table 72. Intel® Server Board S2600IP - BMC Channel Assignments ..................................... 147 Table 73. Intel® Server Board S2600IP - Chassis-specific Sensors ......................................... 148 Table 74. Intel® Server Board S2600IP - Fan Domain Definition ............................................. 149 Table 75. Intel® Server Chassis P4208/P4216/P4224/P4308 (UPL) Power support ................ 152 Table 76. Intel® Server Chassis R2208/R2216/R2224/R2308/R2312 (BHP) Power support ... 152 Table 77. Intel® Workstation Board W2600CR - BMC Channel Assignments .......................... 153

Intel® Server Board S2600IP and Intel® Workstation Board W2600CR TPS Revision 1.1 Intel order number G34153-003 xii Table 78. Intel® Workstation Board W2600CR - Chassis-specific sensors .............................. 154 Table 79. Intel® Workstation Board W2600CR - Fan domain definition ................................... 154 Table 80. Intel® Server Chassis P4304 (UPL) Power support ................................................. 155 Table 81. POST Progress Code LED Example ....................................................................... 157 Table 82. POST Progress Codes ............................................................................................ 157 Table 83. MRC Progress Codes ............................................................................................. 159 Table 84. MRC Fatal Error Codes ........................................................................................... 160 Table 85. POST Error Codes and Messages .......................................................................... 162 Table 86. POST Error Beep Codes ......................................................................................... 167 Table 87. Integrated BMC Beep Codes ................................................................................... 168

List of Tables Revision 1.1 Intel order number G34153-003 xiii

Introduction Revision 1.1 1 Intel order number G34153-003 1. Introduction This Technical Product Specification (TPS) provides board-specific information detailing the features, functionality, and high-level architecture of the Intel® Server Board S2600IP and Intel® Workstation Board W2600CR. In addition, you can obtain design-level information for a given subsystem by ordering the External Product Specifications (EPS) for the specific subsystem. EPS documents are not publicly available and you must order them through your local Intel representative. 1.1 Chapter Outline This document is divided into the following chapters:  Chapter 1 – Introduction  Chapter 2 – Product Overview  Chapter 3 – Functional Architecture  Chapter 4 – Platform Management Overview  Chapter 5 – System Security  Chapter 6 – Connector/Header Locations and Pin-outs  Chapter 7 – Jumper Blocks  Chapter 8 – Intel® Light Guided Diagnostics  Chapter 9 – Design and Environmental Specifications  Appendix A: Integration and Usage Tips  Appendix B: BMC Sensor Tables  Appendix C: Platform Specific BMC Appendix  Appendix D: POST Code Diagnostic LED Decoder  Appendix E: POST Error Messages and Handling  Glossary  Reference Documents 1.2 Server Board Use Disclaimer Intel® Server Boards contain a number of high-density VLSI (Very-large-scale integration) and power delivery components that require adequate airflow for cooling. Intel ensures through its own chassis development and testing that when Intel® server building blocks are used together, the fully integrated system meets the intended thermal requirements of these components. It is the responsibility of the system integrator who chooses not to use Intel developed server building blocks to consult vendor datasheets and operating parameters to determine the amount of airflow required for their specific application and environmental conditions. Intel Corporation cannot be held responsible if components fail or the server board does not operate correctly when used outside any of the published operating or non-operating limits.

2 Revision 1.1 Intel order number G34153-003 2. Product Overview The Intel® Server Board S2600IP and Intel® Workstation Board W2600CR are Enterprise I/O Rich platforms for Pedestal and Rack Server and workstation market. Figure 1. Intel® Server Board S2600IP Figure 2. Intel® Workstation Board W2600CR

Product Overview Revision 1.1 3 Intel order number G34153-003 2.1 Product Feature Set Table 1. Product Feature Set Board Name Intel® Server Board S2600IP Intel® Workstation Board W2600CR Processors  Support for one or two Intel® Xeon® E5- 2600 Processor(s)  8 GT/s Intel® Quick Path Interconnect (Intel® QPI)  LGA 2011 Socket  Thermal Design Power up to 135 W  Support for one or two Intel® Xeon® E5- 2600 Processor(s)  8 GT/s Intel® Quick Path Interconnect (Intel® QPI)  LGA 2011 Socket  Thermal Design Power up to 150 W Chipset Intel® C600-A chipset with support for optional Intel® C600 RAID Upgrade Keys Memory  16 DDR3 DIMMs with 800MT/s, 1067MT/s, 1333MT/s,1600MT/s  Eight memory channels (four channels for each processor socket)  Channels A, B, C, D, E, F, G and H  Support for 800/1066/1333/1600 MHz/s Registered DDR3 Memory (RDIMM),  Unbuffered DDR3 memory ((UDIMM) and Load Reduced DDR3 memory (LRDIMM)  DDR3 standard I/O voltage of 1.5V and DDR3 Low Voltage of 1.35V Slots  Slot 1: PCIe Gen3 x16, connector from first processor.  Slot 2: PCIe Gen3 x8, connector from first processor.  Slot 3: PCIe Gen3 x16, connector from first processor.  Slot 4: PCIe Gen3 x8, connector from second processor.  Slot 5: PCIe Gen3 x16, connector from second processor.  Slot 6: PCIe Gen3 x8, connector, from second processor  Slot 7: PCIe Gen3 x16,connector, from second processor  Slot 8: PCIe Gen2 x4 electrical with x8 physical connector, from second processor IOM Connector: I/O Module (optional with Flex Cable*) Notes: The IOM connector is not available for all boards and only supports PCIe Gen1 Speed SM Conn: SAS Module (optional)  6Gbps or 12Gbps SAS ROC module Options Ethernet Intel® I350 Quad 1GbE fully integrated GbE MAC and PHY Network Controller Intel® I350 Dual 1GbE fully integrated GbE MAC and PHY Controller On-board storage controllers and options  One eUSB 2x5 pin connector to support 2mm low-profile eUSB solid state devices  Two 7-pin single port AHCI SATA connectors capable of supporting up to 6 GB/sec  Two SCU 4-port mini-SAS connectors capable of supporting up to 3 GB/sec SAS/SATA  Intel® Integrated RAID module support (Optional)  Intel® RAID C600 Upgrade Key support providing optional expanded SATA/SAS RAID capabilities Video  Integrated Matrox* G200 2D Video Graphics controller  1 Rear VGA port  Integrated Matrox* G200 2D Video Graphics controller  1 Internal VGA connector

4 Revision 1.1 Intel order number G34153-003 Board Name Intel® Server Board S2600IP Intel® Workstation Board W2600CR Audio None ALC889 HD Audio (7.1 + S/PDIF Out) I/O  4 Rear USB2.0 1 Rear Serial A  2 Type A USB2.0 Connectors  1 Internal Serial B  4 Rear USB2.0 2 Rear USB3.0  2 Type A USB2.0 Connector  1 1394b Connector  1 Internal Serial B Server Management  Onboard Server Engines* LLC Pilot III* Controller  Support for Intel® Remote Management Module 4 solutions  Intel® Light-Guided Diagnostics on field replaceable units  Support for Intel® System Management Software  Support for Intel® Intelligent Power Node Manager (Need PMBus*-compliant power supply) Security Intel® Trusted Platform Module (TPM) - AXXTPME5 (accessory option) Power-Supply 750W/1200W/1600W Common Redundant Power-Supply with PMBus* Support Suspend to RAM (S3) None Supported Form Factor Custom 14.2'' W x 15.0'' D Chassis Pedestal: P4000L chassis 2U Rack: R2000 chassis Pedestal: P4000L WS chassis

Product Overview Revision 1.1 5 Intel order number G34153-003 2.1.1 Main Board Connector and Component Layout The following figure shows the layout of the server board. Each connector and major component is identified by a number or letter, and a description is given below the figure. Callout Description Callout Description A Slot 1, PCI Express* Gen3 AD DIMM C1/C2/D1/D2 B RMM4 lite module connector AE System Fan connector 1/2/3/4/5/6 C Slot 2, PCI Express* Gen3 AF Type A USB connector D TPM AG Type A USB connector E Slot 3, PCI Express* Gen3 AH ROC Module connector F Slot 4, PCI Express* Gen3 AI LCP connector G RMM4 Connector AJ Front Panel connector H Slot 5, PCI Express* Gen3 AK Front Panel USB connector I Battery AL Main Power 4-pin connector J Slot 6, PCI Express* Gen3 AM MinISAS Port B(4-7) K Slot 7, PCI Express* Gen3 AN MiniSAS Port A(0-3)

6 Revision 1.1 Intel order number G34153-003 Callout Description Callout Description L Slot 8, PCI Express* Gen2 AO SATA Port 1 M DIMM E1/E2/F1/F2 AP SATA Port 0 N Status LED AQ CPLD Update O ID LED AR IPMB P Diagnostic LED AS HDD LED Q NIC 3/4 (S2600IP only) AT ME FRC UPDT R USB 2/3, NIC 2 AU CMOS CLR S USB 0/1, NIC 1 AV BIOS RCVRY T VGA Port (S2600IP only) AW PASSWD CLR U Serial Port A (S2600IP only) AX Storage Upgrade Key V Processor 2 Power connector AY eUSB SSD W Processor 2 Fan connector AZ HSBP connector X DIMM H2/H1/G2/G1 BA Chassis intrusion Y PMBus* connector BB Serial B connector Z Main Power connector BC BMC Force Update AA Processor 1 Power connector BD I/O Module connector (not available for S2600IP4L) AB DIMM B2/B1/A2/A1 BE System Fan connector 7 AC Processor 1 Fan connector Figure 3. Intel® Server Board S2600IP Major Components

Product Overview Revision 1.1 7 Intel order number G34153-003 Callout Description Callout Description A Slot 1, PCI Express* Gen3 AE DIMM C1/C2/D1/D2 B RMM4 Lite module AF System Fan connector 1/2/3/4/5/6 C Slot 2, PCI Express* Gen3 AG Type A USB connector D TPM AH Type A USB connector E Slot 3, PCI Express* Gen3 AI ROC Module connector F Slot 4, PCI Express* Gen3 AJ LCP connector G RMM4 Connector AK Front Panel connector H Slot 5, PCI Express* Gen3 AL Front Panel USB connector I Battery AM Main Power 4-pin connector J Slot 6, PCI Express* Gen3 AN 1394b connector(W2600CR only) K Slot 7, PCI Express* Gen3 AO MiniSAS Port B(4-7) L Slot 8, PCI Express* Gen2 AP MiniSAS Port A(0-3) M DIMM E1/E2/F1/F2 AQ SATA Port 1 N Status LED AR SATA Port 0 O ID LED AS CPLD Update P Diagnostic LED AT IPMB

8 Revision 1.1 Intel order number G34153-003 Callout Description Callout Description Q SPDIF connector AU HDD LED R USB 2/3, NIC 2 AV ME FRC UPDT S USB 0/1, NIC 1 AW CMOS CLR T USB3.0 0/1(W2600CR only) AX BIOS RCVRY U HD Audio (W2600CR only) AY PASSWD CLR V Processor 2 Power connector AZ Storage Upgrade Key W Front Audio connector BA eUSB SSD X Processor 2 Fan connector BB HSBP connector Y DIMM H2/H1/G2/G1 BC Chassis intrusion Z PMBus* connector BD Serial B connector AA Main Power connector BE BMC Force Update AB Processor 1 Power connector BF I/O Module connector (not available for W2600CR2L) AC DIMM B2/B1/A2/A1 BG Internal video connector AD Processor 1 Fan connector BH System Fan connector 7 Figure 3. Intel® Workstation Board W2600CR Major Components 2.1.2 Main Board Rear I/O Layout The following drawing shows the layout of the rear I/O components for the server boards. Callout Description Callout Description A Serial Port A E NIC Port 3(top) and 4(bottom) B Video Port F Diagnostics LED’s C NIC Port 1, USB Port 0 (bottom) and 1 (top) G ID LED D NIC Port 2, USB Port 2 (bottom) and 3 (top) H Status LED Figure 4. Intel® Server Board S2600IP Rear I/O Layout

Product Overview Revision 1.1 9 Intel order number G34153-003 Callout Description Callout Description A 7.1 Channel HD Audio Jack Connector E Diagnostics LED’s B USB3.0 Port 0(bottom) and 1(top) F ID LED C NIC Port 1, USB Port 0 (bottom) and 1 (top) G Status LED D NIC Port 2, USB Port 2 (bottom) and 3 (top) Figure 5. Intel® Workstation Board W2600CR Rear I/O Layout

10 Revision 1.1 Intel order number G34153-003 2.1.3 Server Board Mechanical Drawings Figure 6. Mounting Hole Locations (1 of 2)

Product Overview Revision 1.1 11 Intel order number G34153-003 Figure 7. Mounting Hole Locations (2 of 2)

12 Revision 1.1 Intel order number G34153-003 Figure 8. Major Connector Pin-1 Locations (1 of 3)

Product Overview Revision 1.1 13 Intel order number G34153-003 Figure 9. Major Connector Pin-1 Locations (2 of 3)

14 Revision 1.1 Intel order number G34153-003 Figure 10. Major Connector Pin-1 Locations (3 of 3)

Product Overview Revision 1.1 15 Intel order number G34153-003

16 Revision 1.1 Intel order number G34153-003 Figure 11. Primary Side Keep-out Zone

Product Overview Revision 1.1 17 Intel order number G34153-003 Figure 12. Primary Side Card-Side Keep-out Zone

18 Revision 1.1 Intel order number G34153-003 Figure 13. Second Side Keep-out Zone

Functional Architecture Revision 1.1 19 Intel order number G34153-003 3. Functional Architecture The architecture and design of the Intel® Server Board S2600IP and Intel® Workstation Board W2600CR is based on the Intel® C600-A chipset. The chipset is designed for systems based on the Intel® Xeon® Sandy Bridge-EP Processor series in an LGA 2011 Socket with Intel® Quick Path Interconnect (Intel® QPI) speed at 8GT/s. This chapter provides a high-level description of the functionality associated with each chipset component and the architectural blocks that make up the server boards. Figure 14. Intel® Server Board S2600IP Functional Block Diagram

20 Revision 1.1 Intel order number G34153-003 Figure 15. Intel® Workstation Board W2600CR Functional Block Diagram 3.1 Processor Support The server board includes two Socket-R (LGA2011) processor sockets and can support one or two of the following processors - Intel® Xeon® processor E5-2600 product family with a Thermal Design Power (TDP) of up to 150W. Note: The 150w TDP processors only supported on Intel® Workstation Board W2600CR and Intel® Server Board S2600IP only support update to 135w TDP processors. Previous generation Intel® Xeon® processors are not supported on the Intel server boards described in this document. For a complete updated list of supported processors, please visit the support website. 3.1.1 Processor Socket Assembly Each processor socket of the server board is pre-assembled with an Independent Latching Mechanism (ILM) and Back Plate which allow for secure placement of the processor and processor heat to the server board. The illustration below identifies each sub-assembly component.

Functional Architecture Revision 1.1 21 Intel order number G34153-003 Figure 16. Processor Socket Assembly Figure 17. Processor Socket ILM The square ILM has an 80x80mm heat sink mounting hole pattern and is used on the Intel® Server Board S2600IP and Intel® Workstation Board W2600CR. Note: Processor Thermal solutions for the Intel® Workstation Board W2600CR are NOT the same with Intel® Server Board S2600IP when install 150W TDP Processors. Heat Sink Server Board Independent Latching Mechanism (ILM) Back Plate 80mm 80mm

22 Revision 1.1 Intel order number G34153-003 3.1.2 Processor Population Rules Note: Although the server board does support dual-processor configurations consisting of different processors that meet the defined criteria below, Intel does not perform validation testing of this configuration. For optimal system performance in dual-processor configurations, Intel recommends that identical processors be installed. When using a single processor configuration, the processor must be installed into the processor socket labeled “CPU_1”. When two processors are installed, the following population rules apply:  Both processors must be of the same processor family.  Both processors must have the same number of cores.  Both processors must have the same cache sizes for all levels of processor cache memory.  Processors with different core frequencies can be mixed in a system, given the prior rules are met. If this condition is detected, all processor core frequencies are set to the lowest common denominator (highest common speed) and an error is reported.  Processors which have different QPI link frequencies may operate together if they are otherwise compatible and if a common link frequency can be selected. The common link frequency would be the highest link frequency that all installed processors can achieve.  Processor stepping within a common processor family can be mixed as long as it is listed in the processor specification updates published by Intel Corporation. 3.1.3 Processor Initialization Error Summary The following table describes mixed processor conditions and recommended actions for all Intel® server boards and Intel server systems designed around the Intel® Xeon® processor E5- 2600 product family and Intel® C600 chipset product family architecture. The errors fall into one of the following three categories:  Fatal: If the system can boot, it pauses at a blank screen with the text “Unrecoverable fatal error found. System will not boot until the error is resolved” and “Press to enter setup”, regardless of whether the “Post Error Pause” setup option is enabled or disabled. When the operator presses the key on the keyboard, the error message is displayed on the Error Manager screen, and an error is logged to the System Event Log (SEL) with the POST Error Code. The system cannot boot unless the error is resolved. The user needs to replace the faulty part and restart the system. For Fatal Errors during processor initialization, the System Status LED will be set to a steady Amber color, indicating an unrecoverable system failure condition.  Major: If the “Post Error Pause” setup option is enabled, the system goes directly to the Error Manager to display the error, and logs the POST Error Code to SEL. Operator

Functional Architecture Revision 1.1 23 Intel order number G34153-003 intervention is required to continue booting the system. Otherwise, if “POST Error Pause” is disabled, the system continues to boot and no prompt is given for the error, although the Post Error Code is logged to the Error Manager and in a SEL message.  Minor: The message is displayed on the screen or on the Error Manager screen, and the POST Error Code is logged to the SEL. The system continues booting in a degraded state. The user may want to replace the erroneous unit. The POST Error Pause option setting in the BIOS setup does not have any effect on this error. Table 2. Mixed Processor Configurations Error Summary. Error Severity System Action Processor family not Identical Fatal The BIOS detects the error condition and responds as follows:  Logs the POST Error Code into the System Event Log (SEL).  Alerts the BMC to set the System Status LED to steady Amber.  Displays “0194: Processor family mismatch detected” message in the Error Manager.  Takes Fatal Error action (see above) and will not boot until the fault condition is remedied. Processor model not Identical Fatal The BIOS detects the error condition and responds as follows:  Logs the POST Error Code into the System Event Log (SEL).  Alerts the BMC to set the System Status LED to steady Amber.  Displays “0196: Processor model mismatch detected” message in the Error Manager.  Takes Fatal Error action (see above) and will not boot until the fault condition is remedied. Processor cores/threads not identical Fatal The BIOS detects the error condition and responds as follows:  Logs the POST Error Code into the SEL.  Alerts the BMC to set the System Status LED to steady Amber.  Displays “0191: Processor core/thread count mismatch detected” message in the Error Manager. Takes Fatal Error action (see above) and will not boot until the fault condition is remedied. Processor cache not identical Fatal The BIOS detects the error condition and responds as follows:  Logs the POST Error Code into the SEL.  Alerts the BMC to set the System Status LED to steady Amber.  Displays “0192: Processor cache size mismatch detected message in the Error Manager.  Takes Fatal Error action (see above) and will not boot until the fault condition is remedied.

24 Revision 1.1 Intel order number G34153-003 Error Severity System Action Processor frequency (speed) not identical Fatal The BIOS detects the processor frequency difference, and responds as follows:  Adjusts all processor frequencies to the highest common frequency.  No error is generated – this is not an error condition.  Continues to boot the system successfully. If the frequencies for all processors cannot be adjusted to be the same, then this is an error, and the BIOS responds as follows:  Logs the POST Error Code into the SEL.  Alerts the BMC to set the System Status LED to steady Amber.  Does not disable the processor.  Displays “0197: Processor speeds unable to synchronize” message in the Error Manager.  Takes Fatal Error action (see above) and will not boot until the fault condition is remedied. Processor Intel® QuickPath Interconnect link frequencies not identical Fatal The BIOS detects the QPI link frequencies and responds as follows:  Adjusts all QPI interconnect link frequencies to highest common frequency.  No error is generated – this is not an error condition.  Continues to boot the system successfully. If the link frequencies for all QPI links cannot be adjusted to be the same, then this is an error, and the BIOS responds as follows:  Logs the POST Error Code into the SEL.  Alerts the BMC to set the System Status LED to steady Amber.  Displays “0195: Processor Intel(R) QPI link frequencies unable to synchronize” message in the Error Manager.  Does not disable the processor.  Takes Fatal Error action (see above) and will not boot until the fault condition is remedied. Processor microcode update missing Minor The BIOS detects the error condition and responds as follows:  Logs the POST Error Code into the SEL.  Displays “818x: Processor 0x microcode update not found” message in the Error Manager or on the screen.  The system continues to boot in a degraded state, regardless of the setting of POST Error Pause in the Setup. Processor microcode update failed Major The BIOS detects the error condition and responds as follows:  Logs the POST Error Code into the SEL.  Displays “816x: Processor 0x unable to apply microcode update” message in the Error Manager or on the screen. Takes Major Error action. The system may continue to boot in a degraded state, depending on the setting of POST Error Pause in Setup, or may halt with the POST Error Code in the Error Manager waiting for operator intervention.

Functional Architecture Revision 1.1 25 Intel order number G34153-003 3.2 Processor Functions Overview With the release of the Intel® Xeon® processor E5-2600 product family, several key system components, including the CPU, Integrated Memory Controller (IMC), and Integrated IO Module (IIO), have been combined into a single processor package and feature per socket; two Intel® QuickPath Interconnect point-to-point links capable of up to 8.0 GT/s, up to 40 lanes of Gen 3 PCI Express* links capable of 8.0 GT/s, and 4 lanes of DMI2/PCI Express* Gen 2 interface with a peak transfer rate of 5.0 GT/s. The processor supports up to 46 bits of physical address space and 48-bit of virtual address space. The following sections will provide an overview of the key processor features and functions that help to define the performance and architecture of the server board. For more comprehensive processor specific information, refer to the Intel® Xeon® processor E5-2600 product family documents listed in the Reference Document list. Processor Feature Details:  Up to eight execution cores  Each core supports two threads (Intel® Hyper-Threading Technology), up to 16 threads per socket  46-bit physical addressing and 48-bit virtual addressing  1 GB large page support for server applications  A 32-KB instruction and 32-KB data first-level cache (L1) for each core  A 256-KB shared instruction/data mid-level (L2) cache for each core  Up to 20 MB last level cache (LLC): up to 2.5 MB per core instruction/data last level cache (LLC), shared among all cores Supported Technologies:  Intel® Virtualization Technology (Intel® VT)  Intel® Virtualization Technology for Directed I/O (Intel® VT-d)  Intel® Virtualization Technology “Sandy Bridge” Processor Extensions  Intel® Trusted Execution Technology (Intel® TXT)  Intel® 64 Architecture  Intel® Streaming SIMD Extensions 4.1 (Intel® SSE4.1)  Intel® Streaming SIMD Extensions 4.2 (Intel® SSE4.2)  Intel® Advanced Vector Extensions (Intel® AVX)  Intel® Hyper-Threading Technology  Execute Disable Bit  Intel® Turbo Boost Technology  Intel® Intelligent Power Technology  Enhanced Intel® SpeedStep Technology 3.2.1 Intel® QuickPath Interconnect The Intel® QuickPath Interconnect is a high speed, packetized, point-to-point interconnect used in the processor. The narrow high-speed links stitch together processors in distributed shared memory and integrated I/O platform architecture. It offers much higher bandwidth with low latency. The Intel® QuickPath Interconnect has an efficient architecture allowing more

26 Revision 1.1 Intel order number G34153-003 interconnect performance to be achieved in real systems. It has a snoop protocol optimized for low latency and high scalability, as well as packet and lane structures enabling quick completions of transactions. Reliability, availability, and serviceability features (RAS) are built into the architecture. The physical connectivity of each interconnect link is made up of twenty differential signal pairs plus a differential forwarded clock. Each port supports a link pair consisting of two unidirectional links to complete the connection between two components. This supports traffic in both directions simultaneously. To facilitate flexibility and longevity, the interconnect is defined as having five layers: Physical, Link, Routing, Transport, and Protocol. The Intel® QuickPath Interconnect includes a cache coherency protocol to keep the distributed memory and caching structures coherent during system operation. It supports both low-latency source snooping and a scalable home snoop behavior. The coherency protocol provides for direct cache-to-cache transfers for optimal latency. 3.2.2 Integrated Memory Controller (IMC) and Memory Subsystem Integrated into the processor is a memory controller. Each processor provides four DDR3 channels that support the following:  Unbuffered DDR3 and registered DDR3 DIMMs  LR DIMM (Load Reduced DIMM) for buffered memory solutions demanding higher capacity memory subsystems  Independent channel mode or lockstep mode  Data burst length of eight cycles for all memory organization modes  Memory DDR3 data transfer rates of 800, 1066, 1333, and 1600 MT/s  64-bit wide channels plus 8-bits of ECC support for each channel  DDR3 standard I/O Voltage of 1.5 V and DDR3 Low Voltage of 1.35 V  1-Gb, 2-Gb, and 4-Gb DDR3 DRAM technologies supported for these devices: o UDIMM DDR3 – SR x8 and x16 data widths, DR – x8 data width o RDIMM DDR3 – SR,DR, and QR – x4 and x8 data widths o LRDIMM DDR3 – QR – x4 and x8 data widths with direct map or with rank multiplication  Up to 8 ranks supported per memory channel, 1, 2 or 4 ranks per DIMM  Open with adaptive idle page close timer or closed page policy  Per channel memory test and initialization engine can initialize DRAM to all logical zeros with valid ECC (with or without data scrambler) or a predefined test pattern  Isochronous access support for Quality of Service (QoS) CPU 2 CPU 1 2 DIMMs/Ch 2 DIMMs/Ch Figure 18. Integrated Memory Controller Functional Block Diagram

Functional Architecture Revision 1.1 27 Intel order number G34153-003  Minimum memory configuration: independent channel support with 1 DIMM populated  Integrated dual SMBus* master controllers  Command launch modes of 1n/2n  RAS Support: o Rank Level Sparing and Device Tagging o Demand and Patrol Scrubbing o DRAM Single Device Data Correction (SDDC) for any single x4 or x8 DRAM device. Independent channel mode supports x4 SDDC. x8 SDDC requires lockstep mode o Lockstep mode where channels 0 and 1 and channels 2 and 3 are operated in lockstep mode o Data scrambling with address to ease detection of write errors to an incorrect address. o Error reporting from Machine Check Architecture o Read Retry during CRC error handling checks by iMC o Channel mirroring within a socket  CPU1 Channel Mirror Pairs (A,B) and (C,D)  CPU2 Channel Mirror Pairs (E,F) and (G,H) o Error Containment Recovery  Improved Thermal Throttling with dynamic Closed Loop Thermal Throttling (CLTT)  Memory thermal monitoring support for DIMM temperature 3.2.2.1 Supported Memory Table 3. UDIMM Support Guidelines Ranks Per DIMM and Data Width Memory Capacity Per DIMM1 Speed (MT/s) and Voltage Validated by Slot per Channel (SPC) and DIMM Per Channel (DPC)2,3 2 Slots per Channel 1DPC 2DPC 1.35V 1.5V 1.35V 1.5V SRx8 Non- ECC 1GB 2GB 4GB n/a 1066, 1333 n/a 1066, 1333 DRx8 Non- ECC 2GB 4GB 8GB n/a 1066, 1333 n/a 1066, 1333 SRx16 Non- ECC 512MB 1GB 2GB n/a 1066, 1333 n/a 1066, 1333 SRx8 ECC 1GB 2GB 4GB 1066,1333 1066, 1333 1066 1066, 1333 DRx8 ECC 2GB 4GB 8GB 1066,1333 1066, 1333 1066 1066, 1333 Notes: 1. Supported DRAM Densities are 1Gb, 2Gb and 4Gb. Only 2Gb and 4Gb are validated by Intel. 2. Command Address Timing is 1N for 1DPC and 2N for 2DPC. Supported and Validated Supported but not Validate

28 Revision 1.1 Intel order number G34153-003 Table 4. RDIMM Support Guidelines Ranks Per DIMM and Data Width Memory Capacity Per DIMM1 Speed (MT/s) and Voltage Validated by Slot per Channel (SPC) and DIMM Per Channel (DPC)2 2 Slots per Channel 1DPC 2DPC 1.35V 1.5V 1.35V 1.5V SRx8 1GB 2GB 4GB 1066 1333 1066 1333 1600 1066 1333 1066 1333 1600 DRx8 2GB 4GB 8GB 1066 1333 1066 1333 1600 1066 1333 1066 1333 1600 SRx4 2GB 4GB 8GB 1066 1333 1066 1333 1600 1066 1333 1066 1333 1600 DRx4 4GB 8GB 16GB 1066 1333 1066 1333 1600 1066 1333 1066 1333 1600 QRx4 8GB 16GB 32GB 800 1066 800 800 QRx8 4GB 8GB 16GB 800 1066 800 800 Notes: 1. Supported DRAM Densities are 1Gb, 2Gb and 4Gb. Only 2Gb and 4Gb are validated by Intel®. 2. Command Address Timing is 1N. 3. QR RDIMM are supported but only validated by Intel/PMO in a homogenous environment. The coverage will have limited system level testing, no signal integrity testing, and no interoperability testing the passing QR RDIMMs will be web posted. Supported and Validated Supported but not Validate Supported w/Limited Validation4 Table 5. LRDIMM Support Guidelines Ranks Per DIMM and Data Width1 Memory Capacity Per DIMM2 Speed (MT/s) and Voltage Validated by Slot per Channel (SPC) and DIMM Per Channel (DPC)3,4 2 Slots per Channel 1DPC and 2DPC 1.35V 1.5V QRx4 (DDP)5 16GB 32GB 1066 1066, 1333 QRx8 (P)5 8GB 16GB 1066 1066, 1333 Notes: 1. Physical Rank is used to calculate DIMM Capacity. 2. Supported and validated DRAM Densities are 2Gb and 4Gb. 3. Command Address Timing is 1N. 4. The speeds are estimated targets and will be verified through simulation. 5. DDP - Dual Die Package DRAM stacking. P – Planer monolithic DRAM Die. Supported and Validated

Functional Architecture Revision 1.1 29 Intel order number G34153-003 3.2.2.2 Memory Population Rules Note: Although mixed DIMM configurations are supported, Intel only performs platform validation on systems that are configured with identical DIMMs installed. Each processor provides four banks of memory, each capable of supporting up to 2 DIMMs.  DIMMs are organized into physical slots on DDR3 memory channels that belong to processor sockets.  The memory channels from processor socket 1 are identified as Channel A, B, C and D. The memory channels from processor socket 2 are identified as Channel E, F, G, and H.  The silk screened DIMM slot identifiers on the board provide information about the channel, and therefore the processor to which they belong. For example, DIMM_A1 is the first slot on Channel A on processor 1; DIMM_E1 is the first DIMM socket on Channel E on processor 2.  The memory slots associated with a given processor are unavailable if the corresponding processor socket is not populated.  A processor may be installed without populating the associated memory slots provided a second processor is installed with associated memory. In this case, the memory is shared by the processors. However, the platform suffers performance degradation and latency due to the remote memory.  Processor sockets are self-contained and autonomous. However, all memory subsystem support (such as Memory RAS, Error Management,) in the BIOS setup are applied commonly across processor sockets. On the Intel® Server Board S2600IP and Intel® Workstation Board W2600CR a total of 16 DIMM slots is provided (2 CPUs – 4 Channels/CPU, 2 DIMMs/Channel). The nomenclature for DIMM sockets is detailed in the following table: Table 6. DIMM Nomenclature Processor Socket 1 Processor Socket 2 (0) Channel A (1) Channel B (2) Channel C (3) Channel D (0) Channel E (1) Channel F (2) Channel G (3) Channel H A1 A2 B1 B2 C1 C2 D1 D2 E1 E2 F1 F2 G1 G2 H1 H2

30 Revision 1.1 Intel order number G34153-003 Figure 19. DIMM Slot Layout The following are generic DIMM population requirements that generally apply to both the Intel® Server Board S2600IP and Intel® Workstation Board W2600CR.  DIMM slots on any memory channel must be filled following the “farthest fill first” rule.  A maximum of 8 ranks can be installed on any one channel, counting all ranks in each DIMM on the channel.  DIMM types (UDIMM, RDIMM, LRDIMM) must not be mixed within or across processor sockets.  Mixing ECC with non-ECC DIMMs (UDIMMs) is not supported within or across processor sockets.  Mixing Low Voltage (1.35V) DIMMs with Standard Voltage (1.5V) DIMMs is not supported within or across processor sockets.  Mixing DIMMs of different frequencies and latencies is not supported within or across processor sockets.  LRDIMM Rank Multiplication Mode and Direct Map Mode must not be mixed within or across processor sockets.  Only ECC UDIMMs support Low Voltage 1.35V operation.

Functional Architecture Revision 1.1 31 Intel order number G34153-003  QR RDIMMs may only be installed in DIMM Slot 1 or 2 on a channel.  2 DPC QR Low Voltage RDIMMs are not supported.  In order to install 3 QR LRDIMMs on the same channel, they must be operated with Rank Multiplication as RM = 2.  RAS Modes Lockstep, Rank Sparing, and Mirroring are mutually exclusive in this BIOS. Only one operating mode may be selected, and it will be applied to the entire system.  If a RAS Mode has been configured, and the memory population will not support it during boot, the system will fall back to Independent Channel Mode and log and display errors  Rank Sparing Mode is only possible when all channels that are populated with memory meet the requirement of having at least 2 SR or DR DIMM installed, or at least one QR DIMM installed, on each populated channel.  Lockstep or Mirroring Modes require that for any channel pair that is populated with memory, the memory population on both channels of the pair must be identically sized. DIMM population rules require that DIMMs within a channel be populated starting with the BLUE DIMM slot or DIMM farthest from the processor in a “fill-farthest” approach. In addition, when populating a Quad-rank DIMM with a Single- or Dual-rank DIMM in the same channel, the Quad-rank DIMM must be populated farthest from the processor. Note that Quad-rand DIMMs and UDIMMs are not allowed in three slots populated configurations. Intel MRC will check for correct DIMM placement. 3.2.2.3 Publishing System Memory  The BIOS displays the “Total Memory” of the system during POST if Display Logo is disabled in the BIOS setup. This is the total size of memory discovered by the BIOS during POST, and is the sum of the individual sizes of installed DDR3 DIMMs in the system.  The BIOS displays the “Effective Memory” of the system in the BIOS setup. The term Effective Memory refers to the total size of all DDR3 DIMMs that are active (not disabled) and not used as redundant units.  The BIOS provides the total memory of the system in the main page of the BIOS setup. This total is the same as the amount described by the first bullet above. If Quite Boot is disabled, the BIOS displays the total system memory on the diagnostic screen at the end of POST. This total is the same as the amount described by the first bullet above. 3.2.2.4 RAS Features The server board supports the following memory RAS modes:  Independent Channel Mode  Rank Sparing Mode  Mirrored Channel Mode  Lockstep Channel Mode Regardless of RAS mode, the requirements for populating within a channel given in the section 3.2.2.2 must be met at all times. Note that support of RAS modes that require matching DIMM

32 Revision 1.1 Intel order number G34153-003 population between channels (Mirrored and Lockstep) require that ECC DIMMs be populated. Independent Channel Mode is the only mode that supports non-ECC DIMMs in addition to ECC DIMMs. For RAS modes that require matching populations, the same slot positions across channels must hold the same DIMM type with regards to size and organization. DIMM timings do not have to match but timings will be set to support all DIMMs populated (that is, DIMMs with slower timings will force faster DIMMs to the slower common timing modes). 3.2.2.4.1 Independent Channel Mode Channels can be populated in any order in Independent Channel Mode. All four channels may be populated in any order and have no matching requirements. All channels must run at the same interface frequency but individual channels may run at different DIMM timings (RAS latency, CAS Latency, and so forth). 3.2.2.4.2 Rank Sparing Mode In Rank Sparing Mode, one rank is a spare of the other ranks on the same channel. The spare rank is held in reserve and is not available as system memory. The spare rank must have identical or larger memory capacity than all the other ranks (sparing source ranks) on the same channel. After sparing, the sparing source rank will be lost. 3.2.2.4.3 Mirrored Channel Mode In Mirrored Channel Mode, the memory contents are mirrored between Channel 0 and Channel 2 and also between Channel 1 and Channel 3. As a result of the mirroring, the total physical memory available to the system is half of what is populated. Mirrored Channel Mode requires that Channel 0 and Channel 2, and Channel 1 and Channel 3 must be populated identically with regards to size and organization. DIMM slot populations within a channel do not have to be identical but the same DIMM slot location across Channel 0 and Channel 2 and across Channel 1 and Channel 3 must be populated the same. 3.2.2.4.4 Lockstep Channel Mode In Lockstep Channel Mode, each memory access is a 128-bit data access that spans Channel 0 and Channel 1, and Channel 2 and Channel 3. Lockstep Channel mode is the only RAS mode that allows SDDC for x8 devices. Lockstep Channel Mode requires that Channel 0 and Channel 1, and Channel 2 and Channel 3 must be populated identically with regards to size and organization. DIMM slot populations within a channel do not have to be identical but the same DIMM slot location across Channel 0 and Channel 1 and across Channel 2 and Channel 3 must be populated the same. 3.2.3 Processor Integrated I/O Module (IIO) The processor’s integrated I/O module provides features traditionally supported through chipset components. The integrated I/O module provides the following features: 3.2.3.1 PCI Express* Interfaces The integrated I/O module incorporates the PCI Express* interface and supports up to 40 lanes of PCI Express*. Following are key attributes of the PCI Express* interface:

Functional Architecture Revision 1.1 33 Intel order number G34153-003 Intel® Server Board S2600IP and Intel® Workstation Board W2600CR supports PCI-e slots from two processors: From first processor:  Slot 1: PCIe Gen3 x16 connector  Slot 2: PCIe Gen3 x8 connector  Slot 3: PCIe Gen3 x16 connector From second processor:  Slot 4: PCIe Gen3 x8 connector  Slot 5: PCIe Gen3 x16 connector  Slot 6: PCIe Gen3 x8 connector  Slot 7: PCIe Gen3 x16 connector  Slot 8: PCIe Gen2 x4 electrical with x8 physical connector. Note: System will have video output on PCIe slots from first processor by default. If need add-in video card output on slots from second processor, please refer System Service guide to change Bios setup. 3.2.3.2 4.2.3.1.2 DMI2 Interface to the PCH The platform requires an interface to the legacy Southbridge (PCH) which provides basic legacy functions required for the server platform and operating systems. Since only one PCH is required and allowed for the system, any sockets which do not connect to PCH would use this port as a standard x4 PCI Express* 2.0 interface. 3.2.3.3 4.2.3.1.3 Integrated IOAPIC Provides support for PCI Express* devices implementing legacy interrupt messages without interrupt sharing. 3.2.3.4 4.2.3.1.4 Non Transparent Bridge PCI Express* non-transparent bridge (NTB) acts as a gateway that enables high performance, low overhead communication between two intelligent subsystems; the local and the remote subsystems. The NTB allows a local processor to independently configure and control the local subsystem, provides isolation of the local host memory domain from the remote host memory domain while enabling status and data exchange between the two domains. 3.2.3.5 4.2.3.1.5 Intel® QuickData Technology Used for efficient, high bandwidth data movement between two locations in memory or from memory to I/O. 3.3 Intel® C600-A Chipset Functional overview The Intel® C600-A Chipset in the Intel® Server Board S2600IP and Intel® Workstation Board W2600CR provide a connection point between various I/O components and Intel® Xeon Sandy Bridge-EP processors, which includes the following core platform functions:

34 Revision 1.1 Intel order number G34153-003  Digital Media Interface (DMI)  PCI Express* Interface  Serial ATA (SATA) Controller  Serial Attached SCSI (SAS)/SATA Controller  AHCI  Rapid Storage Technology  PCI Interface  Low Pin Count (LPC) Interface  Serial Peripheral Interface (SPI)  Compatibility Modules (DMA Controller, Timer/Counters, Interrupt Controller)  Advanced Programmable Interrupt Controller (APIC)  Universal Serial Bus (USB) Controllers  Gigabit Ethernet Controller  RTC  GPIO  Enhanced Power Management  Intel® Active Management Technology (Intel® AMT)  Manageability  System Management Bus (SMBus* 2.0)  Integrated NVSRAM controller  Virtualization Technology for Directed I/O (Intel® VT-d)  JTAG Boundary-Scan  KVM/Serial Over LAN (SOL) Function 3.3.1 Digital Media Interface (DMI) Digital Media Interface (DMI) is the chip-to-chip connection between the processor and The Intel® C600-A Chipset. This high-speed interface integrates advanced priority-based servicing allowing for concurrent traffic and true isochronous transfer capabilities. Base functionality is completely software-transparent, permitting current and legacy software to operate normally. 3.3.2 PCI Express* Interface The Intel® C600-A Chipset provides up to 8 PCI Express* Root Ports, supporting the PCI Express* Base Specification, Revision 2.0. Each Root Port x1 lane supports up to 5 Gb/s bandwidth in each direction (10 Gb/s concurrent). PCI Express* Root Ports 1-4 or Ports 5-8 can independently be configured to support four x1s, two x2s, one x2 and two x1s, or one x4 port widths. 3.3.3 Serial ATA (SATA) Controller The Intel® C600-A Chipset has two integrated SATA host controllers that support independent DMA operation on up to six ports and supports data transfer rates of up to 6.0 Gb/s (600 MB/s) on up to two ports (Port 0 and 1 Only) while all ports support rates up to 3.0 Gb/s (300 MB/s) and up to 1.5 Gb/s (150 MB/s). The SATA controller contains two modes of operation – a legacy mode using I/O space, and an AHCI mode using memory space. Software that uses legacy mode will not have AHCI capabilities.

Functional Architecture Revision 1.1 35 Intel order number G34153-003 The Intel® C600-A Chipset chipset supports the Serial ATA Specification, Revision 3.0. The Intel® C600-A Chipset also supports several optional sections of the Serial ATA II: Extensions to Serial ATA 1.0 Specification, Revision 1.0 (AHCI support is required for some elements). 3.3.4 Serial Attached SCSI (SAS)/SATA (SCU) Controller The Intel® C600-A Chipset supports up to 8 SAS ports that are compliant with SAS 2.0 Specification and all ports support rates up to 3.0 Gb/s. All 8 ports are also independently configurable and compliant with SATA Gen2 and support data transfer rates of up to 3.0 Gb/s. SAS/SATA controller is only available on specific Intel® C600-A Chipset SKUs with optional RAID C600 Upgrade key. Certain SKUs are also limited to support 4 of 8 SAS/SATA ports only. Please refer details below: Note: SAS/SATA (SCU) port 0-3 is by default enabled while port 4-7 is disabled. Table 7. Intel® RAID C600 Upgrade Key Options Intel® RAID C600 Upgrade Key Options (Product Codes) Key Color Description Default – No option key installed N/A 4 Port SATA with Intel® ESRT RAID 0,1,10 and Intel® RSTe RAID 0,1,5,10 RKSATA4R5 Black 4 Port SATA with Intel® ESRT2 RAID 0,1, 5, 10 and Intel® RSTe RAID 0,1,5,10 RKSATA8 Blue 8 Port SATA with Intel® ESRT2 RAID 0,1, 10 and Intel® RSTe RAID 0,1,5,10 RKSATA8R5 White 8 Port SATA with Intel® ESRT2 RAID 0,1, 5, 10 and Intel® RSTe RAID 0,1,5,10 RKSAS4 Green 4 Port SAS with Intel® ESRT2 RAID 0,1, 10 and Intel® RSTe RAID 0,1,10 RKSAS4R5 Yellow 4 Port SAS with Intel® ESRT2 RAID 0,1, 5, 10 and Intel® RSTe RAID 0,1,10 RKSAS8 Orange 8 Port SAS with Intel® ESRT2 RAID 0,1, 10 and Intel® RSTe RAID 0,1,10 RKSAS8R5 Purple 8 Port SAS with Intel® ESRT2 RAID 0,1, 5, 10 and Intel® RSTe RAID 0,1,10 Additional information for the on-board RAID features and functionality can be found in the Intel® RAID Software User’s Guide (Intel Document Number D29305-019). 3.3.5 AHCI The Intel® C600-A Chipset provides hardware support for Advanced Host Controller Interface (AHCI), a standardized programming interface for SATA host controllers. Platforms supporting AHCI may take advantage of performance features. AHCI also provides usability enhancements such as Hot-Plug. AHCI requires appropriate software support (for example, an AHCI driver) and for some features, hardware support in the SATA device or additional platform hardware. 3.3.6 Rapid Storage Technology The Intel® C600-A Chipset provides support for Intel® Rapid Storage Technology, providing both AHCI and integrated RAID functionality. The RAID capability provides high-performance RAID 0, 1, 5, and 10 functionality on the Intel® C600-A chipset. Matrix RAID support is provided to allow multiple RAID levels to be combined on a single set of hard drives, such as RAID 0 and RAID 1 on two disks. Other RAID features include hot-spare support, SMART alerting, and RAID 0 auto replace.

36 Revision 1.1 Intel order number G34153-003 3.3.7 PCI Interface The Intel® C600-A chipset PCI interface provides a 33 MHz, Revision 2.3 implementation. The Intel® C600-A chipset integrates a PCI arbiter that supports up to four external PCI bus masters in addition to the internal Intel® C600-A chipset requests. This allows for combinations of up to four PCI down devices and PCI slots. 3.3.8 Low Pin Count (LPC) Interface The Intel® C600-A chipset implements an LPC Interface as described in the LPC 1.1 Specification. The Low Pin Count (LPC) bridge function of the Intel® C600-A resides in PCI Device 31: Function 0. In addition to the LPC bridge interface function, D31:F0 contains other functional units including DMA, interrupt controllers, timers, power management, system management, GPIO, and RTC. 3.3.9 Serial Peripheral Interface (SPI) The Intel® C600-A chipset implements an SPI Interface as an alternative interface for the BIOS flash device. The SPI flash is required to support Gigabit Ethernet and Intel® Active Management Technology. The Intel® C600-A chipset supports up to two SPI flash devices with speeds up to 50 MHz. 3.3.10 Compatibility Modules (DMA Controller, Timer/Counters, Interrupt Controller) The DMA controller incorporates the logic of two 82C37 DMA controllers, with seven independently programmable channels. The Intel® C600-A chipset supports LPC DMA through the Intel® C600-A chipset’s DMA controller. The timer/counter block contains three counters that are equivalent in function to those found in one 82C54 programmable interval timer. These three counters are combined to provide the system timer function, and speaker tone. The Intel® C600-A chipset provides an ISA-Compatible Programmable Interrupt Controller (PIC) that incorporates the functionality of two 82C59 interrupt controllers. In addition, the Intel® C600- A chipset supports a serial interrupt scheme. All of the registers in these modules can be read and restored. This is required to save and restore system state after power has been removed and restored to the platform. 3.3.11 Advanced Programmable Interrupt Controller (APIC) In addition to the standard ISA compatible Programmable Interrupt controller (PIC) described in the previous section, the Intel® C600-A incorporates the Advanced Programmable Interrupt Controller (APIC). 3.3.12 Universal Serial Bus (USB) Controllers The Intel® C600-A chipset has up to two Enhanced Host Controller Interface (EHCI) host controllers that support USB high-speed signaling. High-speed USB 2.0 allows data transfers up to 480 Mb/s which is 40 times faster than full-speed USB. The Intel® C600-A chipset supports up to fourteen USB 2.0 ports. All fourteen ports are high-speed, full-speed, and low-speed capable.

Functional Architecture Revision 1.1 37 Intel order number G34153-003 3.3.13 Gigabit Ethernet Controller The Gigabit Ethernet Controller provides a system interface using a PCI function. The controller provides a full memory-mapped or IO mapped interface along with a 64 bit address master support for systems using more than 4 GB of physical memory and DMA (Direct Memory Addressing) mechanisms for high performance data transfers. Its bus master capabilities enable the component to process high-level commands and perform multiple operations; this lowers processor utilization by off-loading communication tasks from the processor. Two large configurable transmit and receive FIFOs (up to 20 KB each) help prevent data underruns and overruns while waiting for bus accesses. This enables the integrated LAN controller to transmit data with minimum interframe spacing (IFS). The LAN controller can operate at multiple speeds (10/100/1000 MB/s) and in either full duplex or half duplex mode. In full duplex mode the LAN controller adheres with the IEEE 802.3x Flow Control Specification. Half duplex performance is enhanced by a proprietary collision reduction mechanism. 3.3.14 RTC The Intel® C600-A chipset contains a real-time clock with 256 bytes of battery-backed RAM. The real-time clock performs two key functions: keeping track of the time of day and storing system data. The RTC operates on a 32.768 KHz crystal and a 3 V battery. 3.3.15 GPIO Various general purpose inputs and outputs are provided for custom system design. The number of inputs and outputs varies depending on the Intel® C600-A chipset configuration. 3.3.16 Enhanced Power Management The Intel® C600-A chipset’s power management functions include enhanced clock control and various low-power (suspend) states (for example, Suspend-to-RAM and Suspend-to-Disk). A hardware-based thermal management circuit permits software-independent entrance to low- power states. The Intel® C600-A chipset contains full support for the Advanced Configuration and Power Interface (ACPI) Specification, Revision 4.0a. 3.3.17 Manageability In addition to Intel AMT the Intel® C600-A chipset integrates several functions designed to manage the system and lower the total cost of ownership (TCO) of the system. These system management functions are designed to report errors, diagnose the system, and recover from system lockups without the aid of an external microcontroller. 3.3.18 System Management Bus (SMBus* 2.0) The Intel® C600-A chipset contains a SMBus* Host interface that allows the processor to communicate with SMBus* slaves. This interface is compatible with most I2C devices. Special I2C commands are implemented. The Intel® C600-A chipset’s SMBus* host controller provides a mechanism for the processor to initiate communications with SMBus* peripherals (slaves). Also, the Intel® C600-A chipset supports slave functionality, including the Host Notify protocol. Hence, the host controller supports eight command protocols of the SMBus* interface (see System Management Bus (SMBus*) Specification, Version 2.0): Quick Command, Send Byte, Receive Byte, Write Byte/Word, Read Byte/Word, Process Call, Block Read/Write, and Host Notify.

38 Revision 1.1 Intel order number G34153-003 The Intel® C600-A chipset’s SMBus* also implements hardware-based Packet Error Checking for data robustness and the Address Resolution Protocol (ARP) to dynamically provide address to all SMBus* devices. 3.3.19 Integrated NVSRAM controller The Intel® C600-A chipset has an integrated NVSRAM controller that supports up to 32KB external device. The host processor can read and write data to the NVSRAM component. 3.3.20 Virtualization Technology for Directed I/O (Intel® VT-d) The Intel® C600-A chipset provides hardware support for implementation of Intel® Virtualization Technology with Directed I/O (Intel® VT-d). Intel VT-d consists of technology components that support the virtualization of platforms based on Intel® Architecture Processors. Intel VT-d Technology enables multiple operating systems and applications to run in independent partitions. A partition behaves like a virtual machine (VM) and provides isolation and protection across partitions. Each partition is allocated its own subset of host physical memory. 3.3.21 JTAG Boundary-Scan The Intel® C600-A chipset adds the industry standard JTAG interface and enables Boundary- Scan in place of the XOR chains used in previous generations of chipsets. Boundary-Scan can be used to ensure device connectivity during the board manufacturing process. The JTAG interface allows system manufacturers to improve efficiency by using industry available tools to test the Intel® C600-A chipset on an assembled board. Since JTAG is a serial interface, it eliminates the need to create probe points for every pin in an XOR chain. This eases pin breakout and trace routing and simplifies the interface between the system and a bed-of-nails tester. 3.3.22 KVM/Serial Over Lan (SOL) Function These functions support redirection of keyboard, mouse, and text screen to a terminal window on a remote console. The keyboard, mouse, and text redirection enables the control of the client machine through the network. Text, mouse, and keyboard redirection allows the remote machine to control and configure the client by entering BIOS setup. The KVM/SOL function emulates a standard PCI serial port and redirects the data from the serial port to the management console using LAN. KVM has additional requirements of internal graphics and SOL may be used when KVM is not supported. 3.3.23 On-board SAS/SATA Support and Options The Intel® C600 chipset provides storage support by two integrated controllers: AHCI and SCU. By default the server board will support up to 10 SATA ports: Two white 6Gb/sec SATA ports from the AHCI controller labeled as “SATA_0” and “SATA_1” and two MINISAS 3Gb/sec SATA/SAS ports routed from the SCU controller labeled as “SCU0 (PORT0-3) and “SCU1(PORT4-7). The SCU Port can be configured with the two embedded software RAID options:  Intel® Embedded Server RAID Technology 2 (ESRT2) based on LSI* MegaRAID SW RAID technology supporting RAID levels 0,1, and 10.  Intel® Rapid Storage Technology (RSTe) supporting RAID levels 0, 1, 5, and 10. 3.3.23.1 Intel® Embedded Server RAID Technology 2 (ESRT2) Features of the embedded software RAID option Intel® Embedded Server RAID Technology 2

Functional Architecture Revision 1.1 39 Intel order number G34153-003 (ESRT2) include the following:  Based on LSI* MegaRAID Software Stack  Software RAID with system providing memory and CPU utilization  Supported RAID Levels – 0,1,5,10  4 and 8 Port SATA RAID 5 support provided with appropriate Intel® RAID C600 Upgrade Key  4 and 8 Port SAS RAID 5 support provided with appropriate Intel® RAID C600 Upgrade Key  Maximum drive support = 8  Open Source Compliance = Binary Driver (includes Partial Source files) or Open Source using MDRAID layer in Linux.  OS Support = Windows 7*, Windows 2008*, Windows 2003*, RHEL*, SLES, other Linux* variants using partial source builds.  Utilities = Windows* GUI and CLI, Linux GUI and CLI, DOS CLI, and EFI CLI 3.3.23.2 Intel® Rapid Storage Technology (RSTe) Features of the embedded software RAID option Intel® Rapid Storage Technology (RSTe) include the following:  Software RAID with system providing memory and CPU utilization  Supported RAID Levels – 0,1,5,10 o 4 Port SATA RAID 5 available standard (no option key required) o 8 Port SATA RAID 5 support provided with appropriate Intel® RAID C600 Upgrade Key o No SAS RAID 5 support  Maximum drive support = 32 (in arrays with 8 port SAS), 16 (in arrays with 4 port SAS), 128 (JBOD)  Open Source Compliance = Yes (uses MDRAID)  OS Support = Windows 7*, Windows 2008*, Windows 2003*, RHEL* 6.2 and later, SLES* 11 w/SP2 and later, VMWare* 5.x.  Utilities = Windows* GUI and CLI, Linux CLI, DOS CLI, and EFI CLI  Uses Matrix Storage Manager for Windows*  MDRAID supported in Linux* (does not require a driver) Note: No boot drive support to targets attached through SAS expander card 3.4 PCI Subsystem The primary I/O buses for the Intel® Server Board S2600IP and Intel® Workstation Board W2600CR are PCI Express* Gen3 with six independent PCI bus segments. The following tables list the characteristics of the PCI bus segments.

40 Revision 1.1 Intel order number G34153-003 Table 8. Intel® Server Board S2600IP/Workstation Board W2600CR PCI Bus Segment Characteristics Voltage Width Speed Type PCI I/O Card Slots 3.3 V x16 32 GB/S PCI Express* Gen3 X16 PCI Express* Gen3 throughput to Slot1 (x16 mechanically) 3.3 V X8 or none (with mux) 16 GB/S or none PCI Express* Gen3 x8 PCI Express* Gen3 throughput to Slot 2 (x8 mechanically) 3.3 V x16 or x8(with mux) 32 GB/S or 16 GB/S PCI Express* Gen3 x16 PCI Express* Gen3 throughput to Slot 3 (x16 mechanically) 3.3 V X8 or none (with mux) 16 GB/S or none PCI Express* Gen3 x8 PCI Express* Gen3 throughput to Slot 4 (x8 mechanically, Non- Persistent) 3.3 V x16 or x8(with mux) 32 GB/S or 16 GB/S PCI Express* Gen3 X16 PCI Express* Gen3 throughput to Slot 5 (x16 mechanically, Non- Persistent) 3.3 V X8 or none (with mux) 16 GB/S or none PCI Express* Gen3 x8 PCI Express* Gen3 throughput to Slot 6 (x8 mechanically, Non- Persistent) 3.3 V x16 or x8(with mux) 32 GB/S or 16 GB/S PCI Express* Gen3 X16 PCI Express* Gen3 throughput to Slot 7 (x16 mechanically, Non- Persistent) 3.3 V X4 4 GB/S PCI Express* Gen2 X4 PCI Express* Gen2 throughput to Slot 8 (x8 mechanically, Non- Persistent) 3.5 Network Interface Controller The Intel® Server Board S2600IP and Workstation Board S2600CR has a Intel® Ethernet Controller I350 GbE Controller. The controller supports PCI Express* PCIe v2.0 (5GT/s and 2.5GT/s). The controller enables four-port or two-port 1000BASE-T implementations using integrated PHY’s. The controller supports VMDq, SR-IOV, EEE, and DMA Coalescing. 3.5.1 Network Interface Network connectivity is provided by means of an onboard Intel® Ethernet Controller 1350-AM4 providing up to four 10/100/1000 Mb Ethernet ports. On the Intel® Server Board S2600IP, four external 10/100/1000 Mb RJ45 Ethernet ports are provided. On the Intel® Workstation Board W2600CR, two external 10/100/1000 Mb RJ45 Ethernet ports are provided. Each Ethernet port drives two LEDs located on each network interface connector. The LED at the right of the connector is the link/activity LED and indicates network connection when on, and transmit/receive activity when blinking. The LED at the left of the connector indicates link speed as defined in the following table.

Functional Architecture Revision 1.1 41 Intel order number G34153-003 LED Color LED State NIC State Green/Amber (Right) Off 10 Mbps Amber 100 Mbps Green 1000 Mbps Green (Left) On Active Connection Blinking Transmit/Receive activity Figure 20. External RJ45 NIC Port LED Definition 3.6 I/O Module Support (not avaiable for Intel® Server Board S2600IP4L and Intel® Workstation Board W2600CR2L) To broaden the standard on-board feature set, the server board supports the option of adding a single I/O module providing external ports for a variety of networking interfaces. The I/O module attaches to a high density 80-pin connector on the server board labeled “IO_Module”. Supported I/O modules include:

42 Revision 1.1 Intel order number G34153-003 Product Code and iPN Description AXX10GBNIAIOM MM# 917905 Dual SFP+ port 10GbE IO Module based on Intel® 82599 10GbE Ethernet Controller AXX10GBTWLIOM MM# 917907 Dual RJ45 Port, 10GBASE-T IO Module, based on Intel® I350 Ethernet chipset AXX1FDRIBIOM MM# 918607 Single Port, FDR speed Infiniband* module, with QSFP connector. AXX4P1GBPWLIOM MM# 917911 Quad Port 1GbE 1o Module based on Intel® Ethernet Controller I350 AXXIOMKIT MM# 920852 I/O Module Cable and Brackets Figure 21. Supported I/O Module Options Notes: 1. I/O Module cable and Brackets accessory is necessary to support I/O module on Intel® Server Board S2600IP. 2. IOM connector only supports PCIe Gen1 Speed 3.7 SAS ROC Module Support The system provides support for Intel® Integrated RAID mezzanine modules that utilize a high density 80-pin connector (labeled “SAS_MOD”) on the server board.

Functional Architecture Revision 1.1 43 Intel order number G34153-003 Features of this option include:  Does not utilize a PCIe slot  SKU options to support full or entry level hardware RAID  4 or 8 port and SAS/SATA or SATA –only ROC options  SKU options to support 512MB or 1GB embedded memory  Intel designed flash + optional support for super-cap backup (Maintenance Free Back Up) or improved Lithium Polymer battery Table 9. Supported Intel® Integrated RAID Modules External Name Description Product Code Intel® Integrated RAID Module RMS25CB080 8 Port SAS-2.1, Full HW RAID, 1GB, Mezz Slot RAID Levels 0,1,10, 5, 50, 6, 60 RMS25CB080 Intel® Integrated RAID Module RMS25CB040 4 Port SAS-2.1, Full HW RAID, 1GB, Mezz Slot RAID Levels 0,1,10, 5, 50, 6, 60 RMS25CB040 Intel® Integrated RAID Module RMT3CB080 8 Port SATA-3, Full HW RAID, 512MB, Mezz Slot RAID Levels 0,1,10, 5, 50, 6, 60 RMT3CB080 Intel® Integrated RAID Module RMS25KB080 8 Port SAS-2.1, Entry-level HW RAID, Mezz Slot RAID Levels 0,1,1E RMS25KB080 Intel® Integrated RAID Module RMS25KB040 4 Port SAS-2.1, Entry-level HW RAID, Mezz Slot RAID Levels 0,1,1E RMS25KB040 For additional product information, please refer the following Intel® document: Intel® Integrated RAID Module RMS25PB080, RMS25PB040, RMS25CB080, and RMS25CB040 Hardware User’s Guide – Intel Order Number G37519-003. 3.8 HD Audio Support (Intel® Workstation Board W2600CR Only) The Intel® Workstation Board W2600CR support 7.1 HD Audio through the High Definition Audio Codec Realtek* ALC889. The ALC889 provides ten DAC channels that simultaneously support 7.1 sound playback, plus two channels of independent stereo sound output (multiple streaming) through the front panel stereo outputs. Also supports two channels of S/PDIF In/Out

44 Revision 1.1 Intel order number G34153-003 (W2600CR only supports 1 S/PDIF Out port) and two channels of Microphone In (Front/Rear). The ALC889 is driven by the Intel® HD Audio Interface (Azalia) from Intel® C600-A chipset. Table 10. Standard Speaker Channels Channel name Identifier 1.0 Mono 2.0 Stereo 5.1 Surround 7.1 Surround Front Left SPEAKER_FRONT_LEFT No Yes Yes Yes Front Right SPEAKER_FRONT_RIGHT No Yes Yes Yes Front Center SPEAKER_FRONT_CENTER Yes No Yes Yes Low Frequency (Subwoofer) SPEAKER_LOW_FREQUENCY No No Yes Yes Back Left SPEAKER_BACK_LEFT No No Yes Yes Back Right SPEAKER_BACK_RIGHT No No Yes Yes Side Left SPEAKER_SIDE_LEFT No No No Yes Side Right SPEAKER_SIDE_RIGHT No No No Yes 3.9 USB3.0 Support (Intel® Workstation Board W2600CR Only) The Intel® Workstation Board W2600CR supports two USB3.0 SuperSpeed* ports, which is required as a standard Workstation feature. The TI 2 port USB3.0 discrete host controller TUSB7320 is used to meet this requirement. The TUSB7320 USB3.0 host controller complies with Universal Serial Bus 3.0 Specification and Intel’s eXtensible Host Controller Interface (xHCI). Figure 22. USB3.0 Discrete Host Controller Block Diagram

Functional Architecture Revision 1.1 45 Intel order number G34153-003 3.10 IEEE1394b Support (Intel® Workstation Board W2600CR Only) Intel® Workstation Board W2600CR offers one front panel IEEE 1394b port from a discrete controller (Texas Instruments XIO2221).The XIO2221 uses a PCIe x1 Gen 1 upstream interface from Intel® C600-A chipset. When connected to 1394b compliant devices, the XIO2221 can transfer data at speeds of up to 800 Mbps. The Intel® Workstation Board W2600CR includes an internal 1394b connector that allows for a front panel 1394b cable attach. Figure 23. IEEE 1394b Discrete Host Controller Block Diagram

46 Revision 1.1 Intel order number G34153-003 4. Platform Management Overview 4.1 Server Management Function Architecture 4.1.1 Feature Support 4.1.1.1 IPMI 2.0 Features The IPMI 2.0 features are as follows: 1. Baseboard management controller (BMC) 2. IPMI Watchdog timer 3. Messaging support, including command bridging and user/session support 4. Chassis device functionality, including power/reset control and BIOS boot flags support 5. Event receiver device: The BMC receives and processes events from other platform subsystems. 6. Field Replaceable Unit (FRU) inventory device functionality: The BMC supports access to system FRU devices using IPMI FRU commands. 7. System Event Log (SEL) device functionality: The BMC supports and provides access to a SEL. 8. Sensor Data Record (SDR) repository device functionality: The BMC supports storage and access of system SDRs. 9. Sensor device and sensor scanning/monitoring: The BMC provides IPMI management of sensors. It polls sensors to monitor and report system health. 10. IPMI interfaces 11. Host interfaces include system management software (SMS) with receive message queue support, and server management mode (SMM) 12. IPMB interface 13. LAN interface that supports the IPMI-over-LAN protocol Remote Management Control Protocol (RMCP, RMCP+) 14. Serial-over-LAN (SOL) 15. ACPI state synchronization: The BMC tracks ACPI state changes that are provided by the BIOS. 16. BMC self test: The BMC performs initialization and run-time self-tests and makes results available to external entities. See also the Intelligent Platform Management Interface Specification Second Generation, Version 2.0. 4.1.1.2 Non IPMI features The BMC supports the following non-IPMI features. This list does not preclude support for future enhancements or additions.  In-circuit BMC firmware update

Revision 1.1 47 Intel order number G34153-003  Fault resilient booting (FRB): FRB2 is supported by the watchdog timer functionality.  Chassis intrusion detection (dependent on platform support)  Basic fan control using Control version 2 SDRs  Fan redundancy monitoring and support  Power supply redundancy monitoring and support  Hot-swap fan support  Acoustic management: Support for multiple fan profiles  Signal testing support: The BMC provides test commands for setting and getting platform signal states.  The BMC generates diagnostic beep codes for fault conditions.  System GUID storage and retrieval  Front panel management: The BMC controls the system status LED and chassis ID LED. It supports secure lockout of certain front panel functionality and monitors button presses. The chassis ID LED is turned on using a front panel button or a command.  Power state retention  Power fault analysis  Intel® Light-Guided Diagnostics  Power unit management: Support for power unit sensor. The BMC handles power- good dropout conditions.  DIMM temperature monitoring: New sensors and improved acoustic management using closed-loop fan control algorithm taking into account DIMM temperature readings.  Address Resolution Protocol (ARP): The BMC sends and responds to ARPs (supported on embedded NICs).  Dynamic Host Configuration Protocol (DHCP): The BMC performs DHCP (supported on embedded NICs).  Platform environment control interface (PECI) thermal management support  E-mail alerting  Embedded web server  Integrated KVM.  Integrated Remote Media Redirection  Lightweight Directory Access Protocol (LDAP) support  Intel® Intelligent Power Node Manager support 4.1.1.3 New Manageability Features Intel® S1400/S1600/S2400/S2600 Server Platforms offer a number of changes and additions to the manageability features that are supported on the previous generation of servers. The following is a list of the more significant changes that are common to this generation Integrated BMC based Intel® Server boards:  Sensor and SEL logging additions/enhancements (for example, additional thermal monitoring capability)  SEL Severity Tracking and the Extended SEL

48 Revision 1.1 Intel order number G34153-003  Embedded platform debug feature which allows capture of detailed data for later analysis.  Provisioning and inventory enhancements: o Inventory data/system information export (partial SMBIOS table)  Enhancements to fan speed control.  DCMI 1.1 compliance (product-specific).  Support for embedded web server UI in Basic Manageability feature set.  Enhancements to embedded web server o Human-readable SEL o Additional system configurability o Additional system monitoring capability o Enhanced on-line help  Enhancements to KVM redirection o Support for higher resolution  Support for EU Lot6 compliance  Management support for PMBus* rev1.2 compliant power supplies  BMC Data Repository (Managed Data Region Feature)  Local Control Display Panel  System Airflow Monitoring  Exit Air Temperature Monitoring  Ethernet Controller Thermal Monitoring  Global Aggregate Temperature Margin Sensor  Memory Thermal Management  Power Supply Fan Sensors  Energy Star Server Support  Smart Ride Through (SmaRT)/ Closed Loop System Throttling (CLST)  Power Supply Cold Redundancy  Power Supply FW Update  Power Supply Compatibility Check  BMC FW reliability enhancements: o Redundant BMC boot blocks to avoid possibility of a corrupted boot block resulting in a scenario that prevents a user from updating the BMC. o BMC System Management Health Monitoring 4.1.2 Basic and Advanced Features The bellowing table lists basic and advanced feature support. Individual features may vary by platform. See the appropriate Platform Specific EPS addendum for more information.

Revision 1.1 49 Intel order number G34153-003 Table 11. Basic and Advanced Features Feature Basic Advanced IPMI 2.0 Feature Support X X In-circuit BMC Firmware Update X X FRB 2 X X Chassis Intrusion Detection X X Fan Redundancy Monitoring X X Hot-Swap Fan Support X X Acoustic Management X X Diagnostic Beep Code Support X X Power State Retention X X ARP/DHCP Support X X PECI Thermal Management Support X X E-mail Alerting X X Embedded Web Server X X SSH Support X X Integrated KVM X Integrated Remote Media Redirection X Lightweight Directory Access Protocol (LDAP) X X Intel® Intelligent Power Node Manager Support X X SMASH CLP X X 4.1.3 Integrated BMC Hardware: Emulex* Pilot III 4.1.3.1 Emulex* Pilot III Baseboard Management Controller Functionality The Integrated BMC is provided by an embedded ARM9 controller and associated peripheral functionality that is required for IPMI-based server management. Firmware usage of these hardware features is platform dependent. The following is a summary of the Integrated BMC management hardware features that comprise the BMC:  400MHz 32-bit ARM9 processor with memory management unit (MMU)  Two independent10/100/1000 Ethernet Controllers with Reduced Media Independent Interface (RMII)/ Reduced Gigabit Media Independent Interface (RGMII) support  DDR2/3 16-bit interface with up to 800 MHz operation  16 10-bit ADCs  Sixteen fan tachometers  Eight Pulse Width Modulators (PWM)  Chassis intrusion logic  JTAG Master  Eight I2C interfaces with master-slave and SMBus* timeout support. All interfaces are SMBus* 2.0 compliant.  Parallel general-purpose I/O Ports (16 direct, 32 shared)  Serial general-purpose I/O Ports (80 in and 80 out)  Three UARTs  Platform Environmental Control Interface (PECI)

50 Revision 1.1 Intel order number G34153-003  Six general-purpose timers  Interrupt controller  Multiple Serial Peripheral Interface (SPI) flash interfaces  NAND/Memory interface  Sixteen mailbox registers for communication between the BMC and host  LPC ROM interface  BMC watchdog timer capability  SD/MMC card controller with DMA support  LED support with programmable blink rate controls on GPIOs  Port 80h snooping capability  Secondary Service Processor (SSP), which provides the HW capability of offloading time critical processing tasks from the main ARM core. Emulex* Pilot III contains an integrated SIO, KVMS subsystem and graphics controller with the following features: 4.1.3.1.1 Super I/O (SIO) The BMC integrates a super I/O module with the following features:  Keyboard Style/BT interface for BMC support  Two Fully Functional Serial Ports, compatible with the 16C550  Serial IRQ Support  Up to 16 Shared GPIO available for host processor  Programmable Wake-up Event Support  Plug and Play Register Set  Power Supply Control 4.1.3.1.2 Graphics Controller The graphics controller provides the following features:  Integrated Graphics Core with 2D Hardware accelerator  High speed Integrated 24-bit RAMDAC DDR-2/3 memory interface with 16Mbytes of memory allocated and reported for graphics memory. 4.1.3.1.3 Remote Keyboard, Video, Mouse, and Storage (KVMS ) The Integrated BMC contains a remote KVMS subsystem with the following features:  USB 2.0 interface for Keyboard, Mouse and Remote storage such as CD/DVD ROM and floppy  USB 1.1/USB 2.0 interface for PS2 to USB bridging, remote Keyboard and Mouse  Hardware Based Video Compression and Redirection Logic  Supports both text and Graphics redirection  Hardware assisted Video redirection using the Frame Processing Engine  Direct interface to the Integrated Graphics Controller registers and Frame buffer  Hardware-based encryption engine

Revision 1.1 51 Intel order number G34153-003 Figure 24. Integrated BMC Hardware 4.2 Server Management Functional Specifications 4.2.1 BMC Internal Timestamp Clock The BMC maintains an internal timestamp clock that is used by various BMC subsystems, for example, time stamping SEL entries. As part of BMC initialization after AC power is applied or the BMC is reset, the BMC initializes this internal clock to the value retrieved from the SSB component’s RTC from a SMBus* slave read operation. This is the system RTC and is on the battery power well so it maintains the current time even when there is no AC supplied to the system. 4.2.1.1 System Clock Synchronization The BIOS must send the Set SEL Time command with the current system time to the BMC during system Power-on Self Test (POST). Synchronization during very early POST is preferred, so that any SEL entries recorded during system boot can be accurately time stamped. Additionally, during sleep state transitions other than S0 the BIOS will synchronize the time. If the time is modified through an OS interface, then the BMC’s time is not synchronized until the next system reboot.

52 Revision 1.1 Intel order number G34153-003 4.2.2 System Event Log (SEL) The BMC implements the system event log as specified in the Intelligent Platform Management Interface Specification, Version 2.0. The SEL is accessible regardless of the system power state through the BMC's in-band and out-of-band interfaces. The BMC allocates 95231 bytes (approximately 93 KB) of non-volatile storage space to store system events. The SEL timestamps may not be in order. Up to 3,639 SEL records can be stored at a time. Any command that results in an overflow of the SEL beyond the allocated space is rejected with an “Out of Space” IPMI completion code (C4h). 4.2.3 Sensor Data Record (SDR) Repository The BMC implements the sensor data record (SDR) repository as specified in the Intelligent Platform Management Interface Specification, Version 2.0. The SDR is accessible through the BMC’s in-band and out-of-band interfaces regardless of the system power state The BMC allocates 65,519 bytes of non-volatile storage space for the SDR. 4.2.4 Field Replaceable Unit (FRU) Inventory Device The BMC implements the interface for logical FRU inventory devices as specified in the Intelligent Platform Management Interface Specification, Version 2.0. This functionality provides commands used for accessing and managing the FRU inventory information. These commands can be delivered through all interfaces. The BMC provides FRU device command access to its own FRU device and to the FRU devices throughout the server. The FRU device ID mapping is defined in the Platform Specific Information. The BMC controls the mapping of the FRU device ID to the physical device. 4.2.5 BMC Beep Codes The BMC may generate beep codes upon detection of failure conditions. Beep codes are sounded each time the problem is discovered (for example, on each power-up attempt), but are not sounded continuously. Common supported codes are listed in below Table Additional platform-specific beep codes can be found in the appropriate Platform Specific Information. Each digit in the code is represented by a sequence of beeps whose count is equal to the digit. Table 12. BMC Beep Codes Code Reason for Beep Associated Sensors 1-5-2-1 No CPUs installed or first CPU socket is empty. CPU Missing Sensor 1-5-2-4 MSID Mismatch. MSID Mismatch Sensor. 1-5-4-2 Power fault: DC power is unexpectedly lost (power good dropout). Power unit – power unit failure offset. 1-5-4-4 Power control fault (power good assertion timeout). Power unit – soft power control failure offset. 1-5-1-2 VR Watchdog Timer sensor assertion VR Watchdog Timer

Revision 1.1 53 Intel order number G34153-003 Code Reason for Beep Associated Sensors 1-5-1-4 The system does not power on or unexpectedly powers off and a power supply unit (PSU) is present that is an incompatible model with one or more other PSUs in the system PS Status 4.2.6 Diagnostic Interrupt (NMI) Button The BMC generates an NMI pulse under certain conditions. The BMC-generated NMI pulse duration is at least 30 ms. Once an NMI has been generated by the BMC, the BMC does not generate another NMI until the system has been reset or powered down. The following actions cause the BMC to generate an NMI pulse: 1. Receiving a Chassis Control command to pulse the diagnostic interrupt. This command does not cause an event to be logged in the SEL. 2. Watchdog timer pre-timeout expiration with NMI/diagnostic interrupt pre-timeout action enabled. The following table shows the behavior regarding NMI signal generation and event logging by the BMC. Table 13. NMI Signal Generation and Event Logging Causal Event NMI Signal Generation Front Panel Diag Interrupt Sensor Event Logging Support Chassis Control command (pulse diagnostic interrupt) X – Front panel diagnostic interrupt button pressed X X Watchdog Timer pre-timeout expiration with NMI/diagnostic interrupt action X X 4.2.7 BMC Watchdog The BMC FW is increasingly called upon to perform system functions that are time-critical in that failure to provide these functions in a timely manner can result in system or component damage. Intel® S1400/S1600/S2400/S2600 Server Platforms introduce a BMC watchdog feature to provide a safe-guard against this scenario by providing an automatic recovery mechanism. It also can provide automatic recovery of functionality that has failed due to a fatal FW defect triggered by a rare sequence of events or a BMC hang due to some type of HW glitch (for example, power). This feature is comprised of a set of capabilities whose purpose is to detect misbehaving subsections of BMC firmware, the BMC CPU itself, or HW subsystems of the BMC component, and to take appropriate action to restore proper operation. The action taken is dependent on the nature of the detected failure and may result in a restart of the BMC CPU, one or more BMC HW subsystems, or a restart of malfunctioning FW subsystems. The BMC watchdog feature will only allow up to three resets of the BMC CPU (such as HW reset) or entire FW stack (such as a SW reset) before giving up and remaining in the uBOOT code. This count is cleared upon cycling of power to the BMC or upon continuous operation of

54 Revision 1.1 Intel order number G34153-003 the BMC without a watchdog-generated reset occurring for a period of > 30 minutes. The BMC FW logs a SEL event indicating that a watchdog-generated BMC reset (either soft or hard reset) has occurred. This event may be logged after the actual reset has occurred. Refer sensor section for details for the related sensor definition. The BMC will also indicate a degraded system status on the Front Panel Status LED after an BMC HW reset or FW stack reset. This state (which follows the state of the associated sensor) will be cleared upon system reset or (AC or DC) power cycle. Note: A reset of the BMC may result in the following system degradations that will require a system reset or power cycle to correct: 1. Timeout value for the rotation period can be set using this parameter; potentially incorrect ACPI Power State reported by the BMC. 2. Reversion of temporary test modes for the BMC (for example, as set through Set SM Signal command) back to normal operational modes. 3. FP status LED and DIMM fault LEDs may not reflect BIOS detected errors. 4.3 Sensor Monitoring 4.3.1 Overview The BMC monitors system hardware and reports system health. The information gathered from physical sensors is translated into IPMI sensors as part of the “IPMI Sensor Model”. The BMC also reports various system state changes by maintaining virtual sensors that are not specifically tied to physical hardware. This section describes the BMC sensors as well as describing how specific sensor types are modeled. Unless otherwise specified, the term “sensor” refers to the IPMI sensor-model definition of a sensor. 4.3.2 Core Sensor List for EPSD Platforms Based on Intel® Xeon® Processor E5 4600/2600/2400/1600/1400 Product Families Specific server boards may only implement a sub-set of sensors and/or may include additional sensors. The system-specific details of supported sensors and events are described in the Appendix of this document. The actual sensor name associated with a sensor number may vary between server boards or systems. Sensor Type Codes Sensor table given below lists the sensor identification numbers and information regarding the sensor type, name, supported thresholds, assertion and de-assertion information, and a brief description of the sensor purpose. Refer to the Intelligent Platform Management Interface Specification, Version 2.0 for sensor and event/reading-type table information.  Sensor Type The sensor type references the values in the Sensor Type Codes table in the Intelligent Platform Management Interface Specification Second Generation v2.0. It provides a context to interpret the sensor.  Event/Reading Type The event/reading type references values from the Event/Reading Type Code Ranges and the Generic Event/Reading Type Code tables in the Intelligent Platform Management Interface Specification Second Generation v2.0. Digital sensors are specific type of discrete sensors that only have two states.  Event Thresholds/Triggers

Revision 1.1 55 Intel order number G34153-003 The following event thresholds are supported for threshold type sensors: - [u,l][nr,c,nc] upper non-recoverable, upper critical, upper non-critical, lower non-recoverable, lower critical, lower non-critical uc, lc upper critical, lower critical Event triggers are supported event-generating offsets for discrete type sensors. The offsets can be found in the Generic Event/Reading Type Code or Sensor Type Code tables in the Intelligent Platform Management Interface Specification Second Generation, Version 2.0, depending on whether the sensor event/reading type is generic or a sensor-specific response.  Assertion/Deassertion Assertion and de-assertion indicators reveal the type of events this sensor generates: - As: Assertion - De: De-assertion  Readable Value/Offsets - Readable value indicates the type of value returned for threshold and other non- discrete type sensors. - Readable offsets indicate the offsets for discrete sensors that are readable by means of the Get Sensor Reading command. Unless otherwise indicated, event triggers are readable. Readable offsets consist of the reading type offsets that do not generate events.  Event Data Event data is the data that is included in an event message generated by the associated sensor. For threshold-based sensors, these abbreviations are used: - R: Reading value - T: Threshold value  Rearm Sensors The rearm is a request for the event status for a sensor to be rechecked and updated upon a transition between good and bad states. Rearming the sensors can be done manually or automatically. This column indicates the type supported by the sensor. The following abbreviations are used in the comment column to describe a sensor: - A: Auto-rearm - M: Manual rearm - I: Rearm by init agent  Default Hysteresis The hysteresis setting applies to all thresholds of the sensor. This column provides the count of hysteresis for the sensor, which can be 1 or 2 (positive or negative hysteresis).  Criticality Criticality is a classification of the severity and nature of the condition. It also controls the behavior of the front panel status LED.  Standby Some sensors operate on standby power. These sensors may be accessed and/or generate events when the main (system) power is off, but AC power is present.

56 Revision 1.1 Intel order number G34153-003 4.3.3 BMC System Management Health Monitoring The BMC tracks the health of each of its IPMI sensors and report failures by providing a “BMC FW Health” sensor of the IPMI 2.0 sensor type Management Subsystem Health with support for the Sensor Failure offset. Only assertions should be logged into the SEL for the Sensor Failure offset. The sensor number of the failed sensor is provided in event data byte 2, as per the IPMI 2.0 Specification. The BMC Firmware Health sensor asserts for any sensor when 10 consecutive sensor errors are read. These are not standard sensor events (that is, threshold crossings or discrete assertions). These are BMC Hardware Access Layer (HAL) errors like I2C NAKs or internal errors while attempting to read a register. If a successful sensor read is completed, the counter resets to zero. IPMI Sensor Characteristics 1. Event reading type code: 6Fh (Sensor specific) 2. Sensor type code: 28h (Management Subsystem Health) 3. Rearm type: Auto If this sensor is implemented, then the following sensor-specific offsets are supported. Table 14. Supported BMC FW Health Sensor Offsets Offset Description Event Logging 04h Sensor failure Assertion and deassertion 4.3.4 Processor Sensors The BMC provides IPMI sensors for processors and associated components, such as voltage regulators and fans. The sensors are implemented on a per-processor basis. Table 15. Processor Sensors Sensor Name Per-Processor Socket Description Processor Status Yes Processor presence and fault state Digital Thermal Sensor Yes Relative temperature reading by means of PECI Processor VRD Over-Temperature Indication Yes Discrete sensor that indicates a processor VRD has crossed an upper operating temperature threshold Processor Voltage Yes Threshold sensor that indicates a processor power-good state Processor Thermal Control (Prochot) Yes Percentage of time a processor is throttling due to thermal conditions 4.3.4.1 Processor Status Sensors The BMC provides an IPMI sensor of type processor for monitoring status information for each processor slot. If an event state (sensor offset) has been asserted, it remains asserted until one of the following happens: 1. A Rearm Sensor Events command is executed for the processor status sensor. 2. AC or DC power cycle, system reset, or system boot occurs.

Revision 1.1 57 Intel order number G34153-003 The BMC provides system status indication to the front panel LEDs for processor fault conditions shown in Table 16. CPU Presence status is not saved across AC power cycles and therefore will not generate a de- assertion after cycling AC power. Table 16. Processor Status Sensor Implementation Offset Processor Status Detected By 0 Internal error (IERR) Not Supported 1 Thermal trip BMC 2 FRB1/BIST failure Not Supported 3 FRB2/Hang in POST failure BIOS1 4 FRB3/Processor startup/initialization failure (CPU fails to start) Not Supported 5 Configuration error (for DMI) BIOS1 6 SM BIOS uncorrectable CPU-complex error Not Supported 7 Processor presence detected BMC 8 Processor disabled Not Supported 9 Terminator presence detected Not Supported Note: Fault is not reflected in the processor status sensor. 4.3.4.1.1 Processor Presence Upon BMC initialization, which occurs when AC power is applied or the BMC is reset, the processor presence offset is initialized to the deasserted state and then the BMC checks to see if the processor is present and sets the presence offset accordingly. This state is updated at each DC power-on and at system resets. The net effect of this is that there should be one event logged at BMC startup for processor presence for each installed processor, assuming the SDR is configured to generate the event. No additional events for processor presence are expected unless the sensor is manually re-armed using an IPMI command or a processor is removed or inserted. Note that removal and insertion should only occur when AC power is removed from the system so it is not expected that a SEL entry will be seen for this specific occurrence. 4.3.4.2 Processor Population Fault (CPU Missing) Sensor The BMC supports a Processor Population Fault sensor for monitoring for the condition in which processor slots are not populated as required by the platform HW to allow power-on of the system. At BMC startup, the BMC will check for the fault condition and set the sensor state accordingly. The BMC also checks for this fault condition at each attempt to DC power-on the system. At each DC power-on attempt, a beep code is generated if this fault is detected. Please refer Table 9 (BMC Beep Codes) for beep code details. The following steps are used to correct the fault condition and clear the sensor state: 1. AC power down the server. 2. Install the missing processor into the correct slot. 3. AC power on the server.

58 Revision 1.1 Intel order number G34153-003 Refer to the applicable platform BMC EPS as to the details of what population rules must be followed. 4.3.4.3 ERR2 Timeout Monitoring The BMC supports an ERR2 Timeout Sensor (1 per CPU) that asserts if a CPU’s ERR2 signal has been asserted for longer than a fixed time period (> 90 seconds). ERR[2] is a processor signal that indicates when the IIO (Integrated IO module in the processor) has a fatal error which could not be communicated to the core to trigger SMI. ERR[2] events are fatal error conditions, where the BIOS and OS will attempt to gracefully handle error, but may not be always do so reliably. A continuously asserted ERR2 signal is an indication that the BIOS cannot service the condition that caused the error. This is usually because that condition prevents the BIOS from running. When an ERR2 timeout occurs, the BMC asserts/deasserts the ERR2 Timeout Sensor, and logs a SEL event for that sensor. The default behavior for BMC core firmware is to initiate a system reset upon detection of an ERR2 timeout. The BIOS setup utility provides an option to disable or enable system reset by the BMC for detection of this condition. IPMI Sensor Characteristics 1. Event reading type code: 03h (Generic – digital discrete) 2. Sensor type code: 07h (Processor) 3. Rearm type: Auto Table 17. Supported ERR2 Timeout Sensor Offsets Offset Description Event Logging 01h State asserted Assertion and deassertion 4.3.4.4 CATERR Sensor and Market Segment ID (MSID) Mismatch The BMC supports a CATERR sensor for monitoring the system CATERR signal. The CATERR signal is defined as having 3 states; high (no event), pulsed low (possibly fatal may be able to recover), and low (fatal). All processors in a system have their CATERR pins tied together. The pin is used as a communication path to signal a catastrophic system event to all CPUs. The BMC has direct access to this aggregate CATERR signal. The BMC only monitors for the “CATERR held low” condition. A pulsed low condition is ignored by the BMC. If a CATERR-low condition is detected, the BMC logs an error message to the SEL against the CATERR sensor and then queries each CPU to determine if it was due to an MSID mismatch condition. An MSID mismatch occurs if a processor is installed into a system board that has incompatible power capabilities. The MSID mismatch condition is indicated in a processor machine check MSR. If PECI is non-functional (it isn’t guaranteed in this situation), then MSID mismatch can’t be detected in that case. If the CATERR is due to an MSID mismatch, then the BMC will log an additional SEL log against the MSID Mismatch sensor, light the CPU fault LED, emit a beep code, and let the system hang. Please refer Table 9 (BMC Beep Codes) for beep code details. If no MSID

Revision 1.1 59 Intel order number G34153-003 mismatch is detected, then the default action after logging the SEL entry is to reset the system. The BIOS setup utility provides an option to disable or enable system reset by the BMC for detection of this condition. 4.3.4.5 CATERR Sensor The BMC supports a CATERR sensor for monitoring the system CATERR signal. The sensor is rearmed on power-on (AC or DC power on transitions). It is not rearmed on system resets in order to avoid multiple SEL events that could occur due to a potential reset loop if the CATERR keeps recurring, which would be the case if the CATERR was due to an MSID mismatch condition. On EPSD boards, the CATERR signal from each CPU are tied together and routed to the BMC as one signal. When the BMC detects that this aggregate CATERR signal has asserted, it can then go through PECI to query each CPU to determine which one was the source of the error and write an OEM code identifying the CPU slot into an event data byte in the SEL entry. If PECI is non-functional (it isn’t guaranteed in this situation), then the OEM code should indicate that the source is unknown. Event data byte 2 and byte 3 for CATERR sensor SEL events ED2 - CATERR type. 0: Unknown 1: CATERR 2: CPU Core Error (not supported on EPSD Platforms Based on Intel® Xeon® Processor E5 4600/2600/2400/1600 Product Families) 3: MSID Mismatch ED3 - CPU bitmap that causes the system CATERR. [0]: CPU0 [1]: CPU1 [2]: CPU2 [3]: CPU3 IPMI Sensor Characteristics 1. Event reading type code: 03h (Digital discrete) 2. Sensor type code: 07h (Processor) 3. Rearm type: Manual The following sensor-specific offsets are supported: Table 18. Supported CATERR Sensor Offsets Offset Description Event Logging 01h State asserted Assertion and deassertion 4.3.4.6 MSID Mismatch Sensor The BMC supports a MSID Mismatch sensor for monitoring for the fault condition that will occur if there is a power rating incompatibility between a baseboard and an a processor

60 Revision 1.1 Intel order number G34153-003 The sensor is rearmed on power-on (AC or DC power on transitions). IPMI Sensor Characteristics 1. Event reading type code: 03h (Digital discrete) 2. Sensor type code: 07h (Processor) 3. Rearm type: Manual The following sensor-specific offsets are supported: Table 19. Support MSID Mismatch Sensor Offsets Offset Description Event Logging 01h State asserted Assertion and deassertion 4.4 Thermal and Acoustic Management The Intel® Server Board S2600IP and Intel® Workstation Board W2600CR offers multiple thermal and acoustic management features to maintain comprehensive thermal protection as well as intelligent fan speed control. The features can be adjusted in BIOS interface with path BIOS > Advanced > System Acoustic and Performance Configuration. 4.4.1 Set Throttling Mode Select the most appropriate memory thermal throttling mechanism for memory sub-system from [Auto], [DCLTT], [SCLTT] and [SOLTT]. [Auto] – BIOS automatically detect and identify the appropriate thermal throttling mechanism based on DIMM type, airflow input, DIMM sensor availability. [DCLTT] – Dynamic Closed Loop Thermal Throttling: for the SOD DIMM with system airflow input [SCLTT] – Static Close Loop Thermal Throttling: for the SOD DIMM without system airflow input [SOLTT] – Static Open Loop Thermal Throttling: for the DIMMs without sensor on dimm (SOD) The default setting is [Auto]. 4.4.2 Altitude Select the proper altitude that the system is distributed from [300m or less], [301m-900m], [901m-1500m], [Above 1500m] options. Lower altitude selection can lead to potential thermal risk. And higher altitude selection provides better cooling but with undesired acoustic and fan power consumption. If the altitude is known, higher altitude is recommended in order to provide sufficient cooling. The default setting is [301m – 900m]. 4.4.3 Set Fan Profile [Performance] and [Acoustic] fan profiles are available to select. The Acoustic mode offers best acoustic experience and appropriate cooling capability covering mainstream and majority of the add-in cards. Performance mode is designed to provide sufficient cooling capability covering all kinds of add-in cards on the market. The default setting is [Performance]

Revision 1.1 61 Intel order number G34153-003 4.4.4 Fan PWM Offset This feature is reserved for manual adjustment to the minimum fan speed curves. The valid range is from [0 to 100] which stands for 0% to 100% PWM adding to the minimum fan speed. This feature is valid when Quiet Fan Idle Mode is at Enabled state. The default setting is [0]. 4.4.5 Quiet Fan Idle Mode This feature can be [Enabled] or [Disabled]. If enabled, the fan will either stopped or shift to a lower speed when the aggregate sensor temperatures are satisfied indicating the system is at ideal thermal/light loading conditions. When the aggregate sensor temperatures not satisfied, the fan will shift back to normal control curves. If disabled, the fan will never stopped or shift into lower fan speed whatever the aggregate sensor temperatures are satisfied or not. The default setting is [Disabled] Note: 1. The above features may or may not be in effective depends on the actual thermal characters of a specific system. 2. Refer to System TPS for the board in Intel chassis thermal and acoustic management 3. Refer to Fan Control Whitepaper for the board in third party chassis fan speed control customization. 4.4.6 Thermal Sensor Input for Fan Speed Control The BMC uses various IPMI sensors as inputs to FSC. Some of the sensors are actual physical sensor and some are “virtual” sensors derived from calculation. The following IPMI thermal sensors are used as input to the fan speed control:  Front Panel Temperature Sensor1  Baseboard Temperature Sensor2  CPU Margin Sensors3,5,6  DIMM Thermal Margin Sensors3,5  Exit Air Temperature Sensor1, 4, 8  PCH Temperature Sensor4,6  On-board Ethernet Controller Temperature Sensors4, 6  Add-In Intel SAS/IO Module Temperature Sensors4, 6  PSU Thermal Sensor4, 9  CPU VR Temperature Sensors4, 7  DIMM VR Temperature Sensors4, 7  iBMC Temperature Sensor4, 7  Global Aggregate Thermal Margin Sensors3, 8 Note: 1. For fan speed control in Intel chassis 2. For fan speed control in 3rd party chassis 3. Temperature margin from throttling threshold 4. Absolute temperature 5. PECI value 6. On-die sensor

62 Revision 1.1 Intel order number G34153-003 7. On-board sensor 8. Virtual sensor 9. Available only when PSU has PMBus* A simplified model is shown as below which gives a high level concept of the fan speed control structure. Figure 25. High-level Fan Speed Control Process 4.4.7 Power Supply Status\Health Sensors The BMC supports one Power Supply Status sensor for each system power supply module. In order to track problems in which the PSU firmware is not operating to full capacity, an additional case (degraded condition if the PSU firmware is not operating to full capacity) is added to the existing Power Supply Status sensor offset definitions. This is handled by assertion of the “configuration error” offset of the PSU status sensor. These sensors are only supported for systems that use PMBus*-compliant power supplies. IPMI Sensor Characteristics 1. Event reading type code: 6Fh (Sensor Specific) 2. Event sensor type code: 08h (Power Supply) 3. Rearm type: Auto The following sensor-specific offsets are supported. Table 20. Supported Power Supply Status Sensor Offsets Offset Description Event Logging 00h Presence detected – Asserted if power supply module is present. Events are only logged for power supply presence upon changes in the presence state after AC power is applied (no events logged for initial state). Assertion and Deassertion

Revision 1.1 63 Intel order number G34153-003 Offset Description Event Logging 01h Power supply failure detected – Asserted if power supply module has failed. The following codes for failure modes are put into the SEL Event Data 2 byte:  01h - Output voltage fault  02h - Output power fault  03h - Output over-current fault  04h - Over-temperature fault  05h – Fan fault The SEL Event Data 3 byte will have the contents of the associated PMBus* Status register to allow for showing multiple conditions for the event. For example, Data 3 will have the contents of the VOLTAGE_STATUS register at the time an Output Voltage fault was detected. Refer to the PMBus Specification for details on specific resister contents Assertion and Deassertion 02h Predictive failure – Asserted if some condition, such as failing fan, has been detected that is likely to lead to a power supply module failure. The following codes for warning modes are put into the SEL Event Data 2 byte:  01h - Output voltage warning  02h - Output power warning  03h - Output over-current warning  04h - Over-temperature warning  05h - Fan warning  06h - Under-voltage warning  07h - Input over-current warning  08h - Input over-power warning The SEL Event Data 3 byte will have the contents of the associated PMBus* Status register to allow for showing multiple conditions for the event. For example, Data 3 will have the contents of the VOLTAGE_STATUS register at the time an Output Voltage Warning was detected. Refer to the PMBus Specification for details on specific resister contents. Assertion and Deassertion 03h Power supply AC lost – Asserted if there is no AC power input to power supply module. Assertion and Deassertion 06h Configuration error – The following codes for configuration errors are put into the SEL Event Data 2 byte:  01h - The BMC cannot access the PMBus* device on the PSU but its FRU device is responding.  02h - The PMBUS_REVISION command returns a version number that is not supported (only version 1.1 and 1.2 are supported for platforms covered under this FW EAS).  03h - The PMBus* device does not successfully respond to the PMBUS_REVISION command.  04h – The PSU is incompatible with one or more PSUs that are present in the system. 05h –The PSU FW is operating in a degraded mode (likely due to a failed firmware update). Assertion and Deassertion 4.4.8 System Event Sensor The BMC supports a System Event sensor and logs SEL event for following events.

64 Revision 1.1 Intel order number G34153-003 Table 21. Support System Event Sensor Offsets Offset Description Event Logging 02h OEM code (Undetermined system HW failure) Assertion and Deassertion 04h PEF action Assertion only For offset 2, OEM code will be logged in event data byte 2 to indicate the type of failure. Only one value will be supported at this time, but others may be added in the future. The code for this particular fault will be 0x00 (PECI access failure) and all other values reserved. Upon detection of the CPU PECI fault condition, the offset shall assert. It shall deassert upon system power cycle or system reset. Assertion of offset 02h shall contribute a “fatal” condition to the system status as reflected in the Front Panel system status LED. 4.4.8.1 Update Related SEL Logging BMC FW will support a single FW Update Status sensor. This sensor is used to generate SEL events related to update of embedded firmware on the platform. This includes updates to the BMC, BIOS, and ME FW. IPMI Sensor Characteristics 1. Event reading type code: 70h (OEM defined) 2. Sensor type code: 2Bh (Version Change) 3. Rearm type: Auto This sensor is an event-only sensor that is not readable. Event generation is only enabled for assertion events. Since this is an event-only sensor it should be defined by a type-3 SDR as defined in the IPMI 2.0 Specification. Table 22. Supported Firmware Update Status Sensor Offsets Offset Description Event Logging Contribution to system health 00h Update started Assertion OK 01h Update completed successfully Assertion OK 02h Update failure Assertion OK Event Data Bytes Byte 2: [Bits 7:4] Target of update 0000b = BMC 0001b = BIOS 0010b = ME 0011b = EWS (Embedded Web server) All other values reserved [Bits 3:1] Target instance (zero-based)

Revision 1.1 65 Intel order number G34153-003 [Bits 0:0] Reserved Byte 3: Reserved Note: All reserved bits must be set to zero. 4.5 Platform Management Interface This chapter describes the supported BMC communication interfaces: 1. Host SMS interface by means of low pin count (LPC)/keyboard controller style (KCS) interface 2. Host SMM interface by means of low pin count (LPC)/keyboard controller style (KCS) interface 3. Intelligent Platform Management Bus (IPMB) I2C interface 4. LAN interface using the IPMI-over-LAN protocols 4.5.1 Channel Management Every messaging interface is assigned an IPMI channel ID by IPMI 2.0. Commands are provided to configure each channel for privilege levels and access modes. Table 23 shows the standard channel assignments: Table 23. Standard Channel Assignments Channel ID Interface Supports Sessions 0 Primary IPMB No 1 LAN 1 Yes 2 LAN 2 Yes 3 LAN3 1 (Provided by the Intel® Dedicated Server Management NIC) Yes 4 Reserved Yes 5 USB No 6 Secondary IPMB No 7 SMM No 8 – 0Dh Reserved – 0Eh Self 2 – 0Fh SMS/Receive Message Queue No Notes: 1. Optional hardware supported by the server system. 2. Refers to the actual channel used to send the request. 4.5.2 User Model The BMC supports the IPMI 2.0 user model. 15 user IDs are supported. These 15 users can be assigned to any channel. The following restrictions are placed on user-related operations:

66 Revision 1.1 Intel order number G34153-003 1. User names for User IDs 1 and 2 cannot be changed. These are always “” (Null/blank) and “root” respectively. 2. User 2 (“root”) always has the administrator privilege level. 3. All user passwords (including passwords for 1 and 2) may be modified. 4. User IDs 3-15 may be used freely, with the condition that user names are unique. Therefore, no other users can be named “” (Null), “root,” or any other existing user name. 4.5.3 LAN Interface The BMC implements both the IPMI 1.5 and IPMI 2.0 messaging models. These provide out-of- band local area network (LAN) communication between the BMC and the network. Run-time determination of LAN channel capabilities can be determined by both standard IPMI defined mechanisms. 4.5.3.1 IPMI 1.5 Messaging The communication protocol packet format consists of IPMI requests and responses encapsulated in an IPMI session wrapper for authentication, and wrapped in an RMCP packet, which is wrapped in an IP/UDP packet. Although authentication is provided, no encryption is provided, so administrating some settings, such as user passwords, through this interface is not advised. Session establishment commands are IPMI commands that do not require authentication or an associated session. The BMC supports the following authentication types over the LAN interface. 1. None (no authentication) 2. Straight password/key 3. MD5 4.5.3.2 IPMI 2.0 Messaging IPMI 2.0 messaging is built over RMCP+ and has a different session establishment protocol. The session commands are defined by RMCP+ and implemented at the RMCP+ level, not IPMI commands. Authentication is implemented at the RMCP+ level. RMCP+ provides link payload encryption, so it is possible to communicate private/sensitive data (confidentiality). The BMC supports the cipher suites identified in Table 24. Table 24. Supported RMCP+ Cipher Suites ID Authentication Algorithm Integrity Algorithm(s) Confidentiality Algorithm(s) 01 RAKP-none None None 1 RAKP-HMAC-SHA1 None None 2 RAKP-HMAC-SHA1 HMAC-SHA1-96 None 3 RAKP-HMAC-SHA1 HMAC-SHA1-96 AES-CBC-128 6 RAKP-HMAC-MD5 None None

Revision 1.1 67 Intel order number G34153-003 ID Authentication Algorithm Integrity Algorithm(s) Confidentiality Algorithm(s) 7 RAKP-HMAC-MD5 HMAC-MD5-128 None 8 RAKP-HMAC-MD5 HMAC-MD5-128 AES-CBC-128 11 RAKP-HMAC-MD5 MD5-128 None 12 RAKP-HMAC-MD5 MD5-128 AES-CBC-128 Note: Cipher suite 0 defaults to callback privilege for security purposes. This may be changed by any administrator. For user authentication, the BMC can be configured with ‘null’ user names, whereby password/key lookup is done based on ‘privilege level only’, or with non-null user names, where the key lookup for the session is determined by user name. IPMI 2.0 messaging introduces payload types and payload IDs to allow data types other than IPMI commands to be transferred. IPMI 2.0 serial-over-LAN is implemented as a payload type. Table 25. Supported RMCP+ Payload Types Payload Type Feature IANA 00h IPMI message N/A 01h Serial-over-LAN N/A 02h OEM explicit Intel (343) 10h – 15h Session setup N/A 4.5.3.3 RMCP/ASF Messaging The BMC supports RMCP ping discovery in which the BMC responds with a pong message to an RMCP/ASF ping request. This is implemented per the Intelligent Platform Management Interface Specification Second Generation v2.0. 4.5.3.4 BMC LAN Channels The BMC supports three RMII/RGMII ports that can be used for communicating with Ethernet devices. Two ports are used for communication with the on-board NICs and one is used for communication with an Ethernet PHY located on an optional add-in card (or equivalent on-board circuitry). 4.5.3.4.1 Baseboard NICs The specific Ethernet controller (NIC) used on a server is platform-specific but all baseboard device options provide support for an NC-SI manageability interface. This provides a sideband high-speed connection for manageability traffic to the BMC while still allowing for a simultaneous host access to the OS if desired. The Network Controller Sideband Interface (NC-SI) is a DMTF industry standard protocol for the side band management LAN interface. This protocol provides a fast multi-drop interface for management traffic. The baseboard NIC(s) are connected to a single BMC RMII/RGMII port that is configured for RMII operation. The NC-SI protocol is used for this connection and provides a 100 Mb/s full- duplex multi-drop interface which allows multiple NICs to be connected to the BMC. The physical layer is based upon RMII, however RMII is a point-to-point bus whereas NC-SI allows 1

68 Revision 1.1 Intel order number G34153-003 master and up to 4 slaves. The logical layer (configuration commands) is incompatible with RMII. Multi-port baseboard NICs on some products will provide support for a dedicated management channel than can be configured to be hidden from the host and only used by the BMC. This mode of operation is configured by a BIOS setup option. 4.5.3.4.2 Dedicated Management Channel An additional LAN channel dedicated to BMC usage and not available to host SW is supported by an optional add-in card. There is only a PHY device present on the add-in card. The BMC has a built-in MAC module that uses the RGMII interface to link with the card’s PHY. Therefore, for this dedicated management interface, the PHY and MAC are located in different devices. The PHY on the card connects to the BMC’s other RMII/RGMII interface (that is, the one that is not connected to the baseboard NICs). This BMC port is configured for RGMII usage. In addition to the use of an add-in card for a dedicated management channel, on systems that support multiple Ethernet ports on the baseboard, the system BIOS provides a setup option to allow one of these baseboard ports to be dedicated to the BMC for manageability purposes. When this is enabled, that port is hidden from the OS. 4.5.3.4.3 Concurrent Server Management Use of Multiple Ethernet Controllers Provided the HW supports a management link between the BMC and a NIC port, the BMC FW supports concurrent OOB LAN management sessions for the following combination:  Two on-board NIC ports  One on-board NIC and the optional dedicated add-in management NIC.  Two on-board NICs and optional dedicated add-in management NIC. All NIC ports must be on different subnets for the above concurrent usage models. MAC addresses are assigned for management NICs from a pool of up to 3 MAC addresses allocated specifically for manageability. The total number of MAC addresses in the pool is dependent on the product HW constraints (for example,a board with 2 NIC ports available for manageability would have a MAC allocation pool of 2 addresses).For these channels, support can be enabled for IPMI-over-LAN and DHCP. For security reasons, embedded LAN channels have the following default settings: IP Address: Static All users disabled IPMI-enabled network interfaces may not be placed on the same subnet. This includes the Intel® Dedicated Server Management NIC and either of the BMC’s embedded network interfaces. Host-BMC communication over the same physical LAN connection – also known as “loopback” – is not supported. This includes “ping” operations. On baseboards with more than two onboard NIC ports, only the first two ports can be used as BMC LAN channels. The remaining ports have no BMC connectivity.

Revision 1.1 69 Intel order number G34153-003  Maximum bandwidth supported by BMC LAN channels are as follows:  BMC LAN1 (Baseboard NIC port) ----- 100M (10M in DC off state)  BMC LAN 2 (Baseboard NIC port) ----- 100M (10M in DC off state)  BMC LAN 3 (Dedicated NIC) ----- 1000M 4.5.3.5 Dedicated Management NIC MAC Address For EPSD Platforms Based on Intel® Xeon® Processor E5 4600/2600/2400/1600/1400 Product Families each server board has either five or seven MAC addresses assigned to it at the Intel factory. The printed MAC address is assigned to NIC1 on the server board. If the platform has two NIC built into the main board then there will be five MAC addresses assigned as follows:  NIC 1 MAC address (for OS usage)  NIC 2 MAC address = NIC 1 MAC address + 1 (for OS usage)  BMC LAN channel 1 MAC address = NIC1 MAC address + 2  BMC LAN channel 2 MAC address = NIC1 MAC address + 3  BMC LAN channel 3 (RMM) MAC address = NIC1 MAC address + 4 If the platform has four NIC built into the main board then there will be seven MAC addresses assigned as follows:  NIC 1 MAC address (for OS usage)  NIC 2 MAC address = NIC 1 MAC address + 1 (for OS usage)  NIC 3 MAC address = NIC 1 MAC address + 2 (for OS usage)  NIC 4 MAC address = NIC 1 MAC address + 3 (for OS usage)  BMC LAN channel 1 MAC address = NIC1 MAC address + 4  BMC LAN channel 2 MAC address = NIC1 MAC address + 5  BMC LAN channel 3 (RMM) MAC address = NIC1 MAC address + 6. 4.5.3.6 IPV6 Support In addition to IPv4, Intel® S1400/S1600/S2400/S2600 Server Platforms support IPv6 for manageability channels. Configuration of IPv6 is provided by extensions to the IPMI Set and Get LAN Configuration Parameters commands as well as through a Web Console IPv6 configuration web page. The BMC supports IPv4 and IPv6 simultaneously so they are both configured separately and completely independently. For example, IPv4 can be DHCP configured while IPv6 is statically configured or vice versa. 4.5.3.6.1 LAN Failover The BMC FW provides a LAN failover capability such that the failure of the system HW associated with one LAN link will result in traffic being rerouted to an alternate link. This functionality is configurable by IPMI methods as well as by the BMC’s Embedded UI, allowing for user to specify the physical LAN links constitute the redundant network paths or physical LAN links constitute different network paths. BMC will support only a all or nothing” approach – that is, all interfaces bonded together, or none are bonded together. The LAN Failover feature applies only to BMC LAN traffic. It bonds all available Ethernet devices but only one is active at a time. When enabled, If the active connection’s leash is lost, one of the secondary connections is automatically configured so that it has the same IP address

70 Revision 1.1 Intel order number G34153-003 (the next active LAN link will be chosen randomly from the pool of backup LAN links with link status as “UP”). Traffic immediately resumes on the new active connection. The LAN Failover enable/disable command may be sent at any time. After it has been enabled, standard IPMI commands for setting channel configuration that specify a LAN channel other than the first will return an error code. Standard IPMI commands for getting channel configuration will return the cached settings for the inactive channels. 4.5.3.7 BMC IP Address Configuration Enabling the BMC’s network interfaces requires using the Set LAN Configuration Parameter command to configure LAN configuration parameter 4, IP Address Source. 4.5.3.8 DHCP BMC Hostname The BMC allows setting a DHCP Hostname. DHCP Hostname can be set regardless of the IP Address source configured on the BMC. But this parameter is only used if the IP Address source is set to DHCP. 4.5.3.9 Address Resolution Protocol (ARP) The BMC can receive and respond to ARP requests on BMC NICs. Gratuitous ARPs are supported, and disabled by default. 4.5.3.10 Virtual Local Area Network (VLAN) The BMC supports VLAN as defined by IPMI 2.0 specifications. VLAN is supported internally by the BMC, not through switches. VLAN provides a way of grouping a set of systems together so that they form a logical network. This feature can be used to set up a management VLAN where only devices which are members of the VLAN will receive packets related to management and members of the VLAN will be isolated from any other network traffic. Please note that VLAN does not change the behavior of the host network setting, it only affects the BMC LAN communication. LAN configuration options are now supported (by means of the Set LAN Config Parameters command, parameters 20 and 21) that allow support for 802.1Q VLAN (Layer 2). This allows VLAN headers/packets to be used for IPMI LAN sessions. VLAN ID’s are entered and enabled by means of parameter 20 of the Set LAN Config Parameters IPMI command. When a VLAN ID is configured and enabled, the BMC only accepts packets with that VLAN tag/ID. Conversely, all BMC generated LAN packets on the channel include the given VLAN tag/ID. Valid VLAN ID’s are 1 through 4094, VLAN ID’s of 0 and 4095 are reserved, per the 802.1Q VLAN specification. Only one VLAN can be enabled at any point in time on a LAN channel. If an existing VLAN is enabled, it must first be disabled prior to configuring a new VLAN on the same LAN channel. Parameter 21 (VLAN Priority) of the Set LAN Config Parameters IPMI command is now implemented and a range from 0-7 will be allowed for VLAN Priorities. Please note that bits 3 and 4 of Parameter 21 are considered Reserved bits. Parameter 25 (VLAN Destination Address) of the Set LAN Config Parameters IPMI command is not supported and returns a completion code of 0x80 (parameter not supported) for any read/write of parameter 25.

Revision 1.1 71 Intel order number G34153-003 If the BMC IP address source is DHCP, then the following behavior is seen:  If the BMC is first configured for DHCP (prior to enabling VLAN), when VLAN is enabled, the BMC performs a discovery on the new VLAN in order to obtain a new BMC IP address.  If the BMC is configured for DHCP (before disabling VLAN), when VLAN is disabled, the BMC performs a discovery on the LAN in order to obtain a new BMC IP address. If the BMC IP address source is Static, then the following behavior is seen:  If the BMC is first configured for static (prior to enabling VLAN), when VLAN is enabled, the BMC has the same IP address that was configured before. It is left to the management application to configure a different IP address if that is not suitable for VLAN.  If the BMC is configure for static (prior to disabling VLAN), when VLAN is disabled, the BMC has the same IP address that was configured before. It is left to the management application to configure a different IP address if that is not suitable for LAN. 4.5.3.11 Secure Shell (SSH) Secure Shell (SSH) connections are supported for one SMASH-CLP session to the BMC. 4.5.3.12 Serial-over-LAN (SOL 2.0) The BMC supports IPMI 2.0 SOL. IPMI 2.0 introduced a standard serial-over-LAN feature. This is implemented as a standard payload type (01h) over RMCP+. Three commands are implemented for SOL 2.0 configuration. 1. “Get SOL 2.0 Configuration Parameters” and “Set SOL 2.0 Configuration Parameters”: These commands are used to get and set the values of the SOL configuration parameters. The parameters are implemented on a per-channel basis. 2. “Activating SOL”: This command is not accepted by the BMC. It is sent by the BMC when SOL is activated to notify a remote client of the switch to SOL. 3. Activating a SOL session requires an existing IPMI-over-LAN session. If encryption is used, it should be negotiated when the IPMI-over LAN session is established. Intel® Server Board S2600IP SOL sessions are supported on serial port A (COM1). Intel® Workstation Board S2600IP SOL sessions are supported on serial port B (COM2). 4.5.3.13 Platform Event Filter (PEF) The BMC includes the ability to generate a selectable action, such as a system power-off or reset, when a match occurs to one of a configurable set of events. This capability is called Platform Event Filtering, or PEF. One of the available PEF actions is to trigger the BMC to send a LAN alert to one or more destinations. The BMC supports 20 PEF filters. The first twelve entries in the PEF filter table are pre- configured (but may be changed by the user). The remaining entries are left blank, and may be configured by the user. Table 26. Factory Configured PEF Table Entries

72 Revision 1.1 Intel order number G34153-003 Event Filter Number Offset Mask Events 1 Non-critical, critical and non- recoverable Temperature sensor out of range 2 Non-critical, critical and non- recoverable Voltage sensor out of range 3 Non-critical, critical and non- recoverable Fan failure 4 General chassis intrusion Chassis intrusion (security violation) 5 Failure and predictive failure Power supply failure 6 Uncorrectable ECC BIOS 7 POST error BIOS: POST code error 8 FRB2 Watchdog Timer expiration for FRB2 9 Policy Correction Time Node Manager 10 Power down, power cycle, and reset Watchdog timer 11 OEM system boot event System restart (reboot) 12 Drive Failure, Predicted Failure Hot Swap Controller Additionally, the BMC supports the following PEF actions:  Power off  Power cycle  Reset  OEM action  Alerts The “Diagnostic interrupt” action is not supported. 4.5.3.14 LAN Alerting The BMC supports sending embedded LAN alerts, called SNMP PET (Platform Event traps), and SMTP email alerts. The BMC supports a minimum of four LAN alert destinations. 4.5.3.14.1 SNMP Platform Event Traps (PETs) This feature enables a target system to send SNMP traps to a designated IP address by means of LAN. These alerts are formatted per the Intelligent Platform Management Interface Specification Second Generation v2.0. A MIB file associated with the traps is provided with the BMC firmware to facilitate interpretation of the traps by external software. The format of the MIB file is covered under RFC 2578. 4.5.3.15 Alert Policy Table Associated with each PEF entry is an alert policy that determines which IPMI channel the alert is to be sent. There is a maximum of 20 alert policy entries. There are no pre-configured entries

Revision 1.1 73 Intel order number G34153-003 in the alert policy table because the destination types and alerts may vary by user. Each entry in the alert policy table contains four bytes for a maximum table size of 80 bytes. 4.5.3.15.1 E-mail Alerting The Embedded Email Alerting feature allows the user to receive e-mails alerts indicating issues with the server. This allows e-mail alerting in an OS-absent (for example, Pre-OS and OS-Hung) situation. This feature provides support for sending e-mail by means of SMTP, the Simple Mail Transport Protocol as defined in Internet RC 821. The e-mail alert provides a text string that describes a simple description of the event. SMTP alerting is configured using the embedded web server. 4.5.3.16 SM-CLP (SM-CLP Lite) SMASH refers to Systems Management Architecture for Server Hardware. SMASH is defined by a suite of specifications, managed by the DMTF, that standardize the manageability interfaces for server hardware. CLP refers to Command Line Protocol. SM-CLP is defined by the Server Management Command Line Protocol Specification (SM-CLP) ver1.0, which is part of the SMASH suite of specifications. The specifications and further information on SMASH can be found at the DMTF website (http://www.dmtf.org/). The BMC provides an embedded “lite” version of SM-CLP that is syntax-compatible but not considered fully compliant with the DMTF standards. 4.5.3.17 Embedded Web Server BMC Base manageability provides an embedded web server and an OEM-customizable web GUI which exposes the manageability features of the BMC base feature set. It is supported over all on-board NICs that have management connectivity to the BMC as well as an optional dedicated add-in management NIC. At least two concurrent web sessions from up to two different users is supported. The embedded web user interface shall support the following client web browsers:  Microsoft Internet Explorer 7.0*  Microsoft Internet Explorer 8.0*  Microsoft Internet Explorer 9.0*  Mozilla Firefox 3.0*  Mozilla Firefox 3.5*  Mozilla Firefox 3.6* The embedded web user interface supports strong security (authentication, encryption, and firewall support) since it enables remote server configuration and control. Embedded web server uses ports #80 and #443. The user interface presented by the embedded web user interface shall authenticate the user before allowing a web session to be initiated. Encryption using 128- bit SSL is supported. User authentication is based on user id and password. The GUI presented by the embedded web server authenticates the user before allowing a web session to be initiated. It presents all functions to all users but grays-out those functions that the user does not have privilege to execute. (For example, if a user does not have privilege to power control, then the item shall be displayed in grey-out font in that user’s UI display). The web GUI also provides a launch point for some of the advanced features, such as KVM and media redirection. These features are grayed out in the GUI unless the system has been updated to support these advanced features.

74 Revision 1.1 Intel order number G34153-003 Additional features supported by the web GUI includes:  Presents all the Basic features to the users.  Power on/off/reset the server and view current power state.  Displays BIOS, BMC, ME and SDR version information.  Display overall system health.  Configuration of various IPMI over LAN parameters for both IPV4 and IPV6  Configuration of alerting (SNMP and SMTP).  Display system asset information for the product, board, and chassis.  Display of BMC-owned sensors (name, status, current reading, enabled thresholds), including color-code status of sensors.  Provides ability to filter sensors based on sensor type (Voltage, Temperature, Fan and Power supply related)  Automatic refresh of sensor data with a configurable refresh rate.  On-line help.  Display/clear SEL (display is in easily understandable human readable format).  Supports major industry-standard browsers (Microsoft Internet Explorer* and Mozilla Firefox*).  The GUI session automatically times-out after a user-configurable inactivity period. By default, this inactivity period is 30 minutes.  Embedded Platform Debug feature - Allow the user to initiate a “diagnostic dump” to a file that can be sent to Intel for debug purposes.  Virtual Front Panel. The Virtual Front Panel provides the same functionality as the local front panel. The displayed LEDs match the current state of the local panel LEDs. The displayed buttons (for example,power button) can be used in the same manner as the local buttons.  Display of ME sensor data. Only sensors that have associated SDRs loaded will be displayed.  Ability to save the SEL to a file.  Ability to force HTTPS connectivity for greater security. This is provided through a configuration option in the UI.  Display of processor and memory information as is available over IPMI over LAN.  Ability to get and set Node Manager (NM) power policies.  Display of power consumed by the server.  Ability to view and configure VLAN settings.  Warn user the reconfiguration of IP address will cause disconnect.  Capability to block logins for a period of time after several consecutive failed login attempts. The lock-out period and the number of failed logins that initiates the lock-out period are configurable by the user.  Server Power Control - Ability to force into Setup on a reset. 4.5.3.18 Virtual Front Panel  Virtual Front Panel is the module present as “Virtual Front Panel” on the left side in the embedded web server when "remote Control" tab is clicked.  Main Purpose of the Virtual Front Panel is to provide the front panel functionality virtually.  Virutal Front Panel (VFP) will mimic the status LED and Power LED status and Chassis ID alone. It is automatically in sync with BMC every 40 seconds.  For any abnormal status LED state, Virtual Front Panel will get the reason behind the abnormal or status LED changes and displayed in VFP side.

Revision 1.1 75 Intel order number G34153-003  As Virtual Front Panel uses the chassis control command for power actions. It won’t log the Front button press event since Logging the front panel press event for Virtual Front Panel press will mislead the administrator.  For Reset from Virtual Front Panel, the reset will be done by a “Chassis control” command.  For Reset from Virtual Front Panel, the restart cause will be because of “Chassis control” command.  During Power action, Power button/Reset button should not accept the next action until current Power action is complete and the acknowledgment from BMC is received.  EWS will provide a valid message during Power action until it completes the current Power action.  The VFP does not have any effect on whether the front panel is locked by “Set Front Panel Enables” command.  The chassis ID LED provides a visual indication of a system being serviced. The state of the chassis ID LED is affected by the following actions:  Toggled by turning the chassis ID button on or off.  There is no precedence or lock-out mechanism for the control sources. When a new request arrives, previous requests are terminated. For example, if the chassis ID button is pressed, then the chassis ID LED changes to solid on. If the button is pressed again, then the chassis ID LED turns off.  Note that the chassis ID will turn on because of the original chassis ID button press and will reflect in the Virtual Front Panel after VFP sync with BMC. Virtual Front Panel won’t reflect the chassis LED software blinking by the software command as there is no mechanism to get the chassis ID Led status.  Only Infinite chassis ID ON/OFF by the software command will reflect in EWS during automatic /manual EWS sync up with BMC.  Virtual Front Panel help should available for virtual panel module.  At present, NMI button in VFP is disabled in Intel® S1400/S1600/S2400/S2600 Server Platforms. It can be used in future. 4.5.3.19 Embedded Platform Debug The Embedded Platform Debug feature supports capturing low-level diagnostic data (applicable MSRs, PCI config-space registers, and so on). This feature allows a user to export this data into a file that is retrievable from the embedded web GUI, as well as through host and remote IPMI methods, for the purpose of sending to an Intel engineer for an enhanced debugging capability. The files are compressed, encrypted, and password protected. The file is not meant to be viewable by the end user but rather to provide additional debugging capability to an Intel support engineer. 4.5.3.20 Data Center Management Interface (DCMI) The DCMI Specification is an emerging standard that is targeted to provide a simplified management interface for Internet Portal Data Center (IPDC) customers. It is expected to become a requirement for server platforms which are targeted for IPDCs. DCMI is an IPMI- based standard that builds upon a set of required IPMI standard commands by adding a set of DCMI-specific IPMI OEM commands. Intel® S1400/S1600/S2400/S2600 Server Platforms will be implementing the mandatory DCMI features in the BMC firmware (DCMI 1.1 Errata 1 compliance). Please refer to DCMI 1.1 errata 1 spec for details. Only mandatory commands will be supported. No support for optional DCMI commands. Optional power management and SEL roll over feature is not supported. DCMI Asset tag will be independent of baseboard FRU asset

76 Revision 1.1 Intel order number G34153-003 Tag. Please refer table DCMI Group Extension Commands for more details on DCMI commands. 4.5.3.21 Lightweight Directory Authentication Protocol (LDAP) The Lightweight Directory Access Protocol (LDAP) is an application protocol supported by the BMC for the purpose of authentication and authorization. The BMC user connects with an LDAP server for login authentication. This is only supported for non-IPMI logins including the embedded web UI and SM-CLP. IPMI users/passwords and sessions are not supported over LDAP. LDAP can be configured (IP address of LDAP server, port, and so on) from the BMC’s Embedded Web UI. LDAP authentication and authorization is supported over the any NIC configured for system management. The BMC uses a standard Open LDAP implementation for Linux. Only open LDAP is supported by BMC. Windows and Novel LDAP are not supported. 4.6 BMC Firmware Update 4.6.1 The BMC firmware release Number The BMC firmware releases are numbered as follows: AB.CD.XXX Where: CD is a two digit number starting with 00 for the first release. This is the number returned in minor ID in Get Device ID response. B is major release number (1 for the first major release typically at first production shipment). A is point release indicator (A=0 means this is a regular official release, A=1 means this is a point release for limited distribution). A hex representation of “AB” is returned in major ID in Get Device ID response. XXX is the build number for internal tracking. This number is returned in OEM bytes of Get Device ID response. 4.6.2 Boot Recovery Mode The BMC’s boot block supports firmware transfer updates. The Operational Firmware Transfer mode preserves several files in the PIA Linux file system. Boot Recovery mode cannot preserve the files because it does not understand Linux file systems, and treats it as a large binary data section. This means a Boot Recovery update completely replaces the PIA with the factory default version: an empty SEL, a default SDR, and default IPMI configuration and user settings. Boot Recovery mode can successfully complete an update in some situations where the Operational Firmware Transfer mode will fail. If there is an incompatibility or bug in the operational code causing it to crash or hang, only a Boot Recovery Mode Update works. Another example is if the flash layout of the sections changes across an update. Because the operational Firmware Transfer mode tries to preserve the contents of the PIA section, in this

Revision 1.1 77 Intel order number G34153-003 case, it will corrupt the flash where the old PIA section was. Because the Boot Recovery mode is blindly writing binary data to flash, in this case, it will succeed. There are two ways to enter Boot Recovery mode:  The Force Firmware Update jumper is asserted when A/C power is applied.  The operational code is corrupt and the boot loader cannot boot. In the Boot Recovery mode, the BMC only responds to the small set of commands listed above. Only the KCS SMS interface is supported; USB-based Fast Firmware Update is not supported. 4.6.3 Force Firmware Update Jumper The Force Firmware Update jumper can be used to put the BMC in Boot Recovery mode for a low-level update. It causes the BMC to abort its normal boot process and stay in the boot loader without executing any Linux code. The jumper is normally in the de-asserted position. The system must be completely powered off (A/C power removed) before the jumper is moved. After power is re-applied and the firmware update is complete, the system must be powered off again and the jumper must be returned to the de-asserted position before normal operation can begin. There is no boot–block-write protection jumper. 4.6.4 Fast Firmware Update over USB The BMC supports a Fast Firmware Update mode in addition to the standard KCS SMS interface. This is a special protocol that goes over the USB connection between the host and the BMC.

78 Revision 1.1 Intel order number G34153-003 4.7 Advanced Management Feature Support This section explains the advanced management features supported by the BMC firmware. 4.7.1 Enabling Advanced Management Features The Advanced management features are to be delivered as part of the BMC FW image. The BMC’s baseboard SPI flash contains code/data for both the Basic and Advanced features. An optional add-in card Intel® RMM4 lite is used as the activation mechanism. When the BMC FW initializes, it attempts to access the Intel® RMM4 lite. If the attempt to access Intel® RMM4 lite is successful, then the BMC activates the Advanced features. Advanced manageability features are supported over all NIC ports enabled for server manageability. This includes baseboard NICs as well as the LAN channel provided by the optional Dedicated NIC add-in card. RMM4 is comprised of two boards – RMM4 lite and the optional Dedicated Server Management NIC (DMN). If the optional Dedicated Server Management NIC is not used then the traffic can only go through the onboard Integrated BMC-shared NIC and share network bandwidth with the host system. Table 27. Enabling Advanced Management Features Manageability Hardware Benefits Intel® Integrated BMC Comprehensive IPMI based base manageability features Intel® Remote Management Module 4 – Lite Package contains one module – 1- Key for advance Manageability features. No dedicated NIC for management Enables KVM and media redirection from the onboard NIC Intel® Remote Management Module 4 Package includes 2 modules – 1 - key for advance features 2 - Dedicated NIC (1Gbe) for management Dedicated NIC for management traffic. Higher bandwidth connectivity for KVM and media Redirection with 1Gbe NIC. 4.7.2 Keyboard, Video, Mouse (KVM) Redirection The BMC firmware supports keyboard, video, and mouse redirection (KVM) over LAN. This feature is available remotely from the embedded web server as a Java applet. This feature is only enabled when the Intel® RMM4 lite is present. The client system must have a Java Runtime Environment (JRE) version 6.0 or later to run the KVM or media redirection applets. The BMC supports an embedded KVM application (Remote Console) that can be launched from the embedded web server from a remote console. USB1.1 or USB 2.0 based mouse and keyboard redirection are supported. It is also possible to use the KVM-redirection (KVM-r) session concurrently with media-redirection (media-r). This feature allows a user to interactively use the keyboard, video, and mouse (KVM) functions of the remote server as if the user were physically at the managed server.

Revision 1.1 79 Intel order number G34153-003 KVM redirection console support the following keyboard layouts: English, Dutch, French, German, Italian, Russian, and Spanish. KVM redirection includes a “soft keyboard” function. The “soft keyboard” is used to simulate an entire keyboard that is connected to the remote system. The “soft keyboard” functionality supports the following layouts: English, Dutch, French, German, Italian, Russian, and Spanish. The KVM-redirection feature automatically senses video resolution for best possible screen capture and provides high-performance mouse tracking and synchronization. It allows remote viewing and configuration in pre-boot POST and BIOS setup, once BIOS has initialized video. Other attributes of this feature include:  Encryption of the redirected screen, keyboard, and mouse  Compression of the redirected screen.  Ability to select a mouse configuration based on the OS type.  supports user definable keyboard macros. KVM redirection feature supports the following resolutions and refresh rates:  640x480 at 60Hz, 72Hz, 75Hz, 85Hz, 100Hz  800x600 at 60Hz, 72Hz, 75Hz, 85Hz  1024x768 at 60Hx, 72Hz, 75Hz, 85Hz  1280x960 at 60Hz  1280x1024 at 60Hz  1600x1200 at 60Hz  1920x1080 (1080p),  1920x1200 (WUXGA)  1650x1080 (WSXGA+) 4.7.2.1 Force-enter BIOS Setup KVM redirection can present an option to force-enter BIOS Setup. This enables the system to enter F2 setup while booting which is often missed by the time the remote console redirects the video. 4.7.3 Media Redirection The embedded web server provides a Java applet to enable remote media redirection. This may be used in conjunction with the remote KVM feature, or as a standalone applet. The media redirection feature is intended to allow system administrators or users to mount a remote IDE or USB CD-ROM, floppy drive, or a USB flash disk as a remote device to the server. Once mounted, the remote device appears just like a local device to the server, allowing system administrators or users to install software (including operating systems), copy files, update BIOS, and so on, or boot the server from this device. The following capabilities are supported:  The operation of remotely mounted devices is independent of the local devices on the server. Both remote and local devices are useable in parallel.  Either IDE (CD-ROM, floppy) or USB devices can be mounted as a remote device to the server.  It is possible to boot all supported operating systems from the remotely mounted device and to boot from disk IMAGE (*.IMG) and CD-ROM or DVD-ROM ISO files. See the Tested/supported Operating System List for more information.

80 Revision 1.1 Intel order number G34153-003  Media redirection supports redirection for both a virtual CD device and a virtual Floppy/USB device concurrently. The CD device may be either a local CD drive or else an ISO image file; the Floppy/USB device may be either a local Floppy drive, a local USB device, or else a disk image file.  The media redirection feature supports multiple encryption algorithms, including RC4 and AES. The actual algorithm that is used is negotiated with the client based on the client’s capabilities.  A remote media session is maintained even when the server is powered-off (in standby mode). No restart of the remote media session is required during a server reset or power on/off. An BMC reset (for example,due to an BMC reset after BMC FW update) will require the session to be re-established  The mounted device is visible to (and useable by) managed system’s OS and BIOS in both pre-boot and post-boot states.  The mounted device shows up in the BIOS boot order and it is possible to change the BIOS boot order to boot from this remote device.  It is possible to install an operating system on a bare metal server (no OS present) using the remotely mounted device. This may also require the use of KVM-r to configure the OS during install. USB storage devices will appear as floppy disks over media redirection. This allows for the installation of device drivers during OS installation. If either a virtual IDE or virtual floppy device is remotely attached during system boot, both the virtual IDE and virtual floppy are presented as bootable devices. It is not possible to present only a single-mounted device type to the system BIOS.

Revision 1.1 81 Intel order number G34153-003 4.8 Intel® Intelligent Power Node Manager (NM) 4.8.1 Overview Power management deals with requirements to manage processor power consumption and manage power at the platform level to meet critical business needs. Node Manager (NM) is a platform resident technology that enforces power capping and thermal-triggered power capping policies for the platform. These policies are applied by exploiting subsystem knobs (such as processor P and T states) that can be used to control power consumption. NM enables data center power management by exposing an external interface to management software through which platform policies can be specified. It also implements specific data center power management usage models such as power limiting, and thermal monitoring. The NM feature is implemented by a complementary architecture utilizing the ME, BMC, BIOS, and an ACPI-compliant OS. The ME provides the NM policy engine and power control/limiting functions (referred to as Node Manager or NM) while the BMC provides the external LAN link by which external management software can interact with the feature. The BIOS provides system power information utilized by the NM algorithms and also exports ACPI Source Language (ASL) code used by OS-Directed Power Management (OSPM) for negotiating processor P and T state changes for power limiting. PMBus*-compliant power supplies provide the capability to monitoring input power consumption, which is necessary to support NM. 4.8.1.1 Hardware Requirements NM is supported only on platforms that have the NM FW functionality loaded and enabled on the Management Engine (ME) in the SSB and that have a BMC present to support the external LAN interface to the ME. NM power limiting features requires a means for the ME to monitor input power consumption for the platform. This capability is generally provided by means of PMBus*-compliant power supplies although an alternative model using a simpler SMBus* power monitoring device is possible (there is potential loss in accuracy and responsiveness using non- PMBus* devices). The NM SmaRT/CLST feature does specifically require PMBus*-compliant power supplies as well as additional hardware on the baseboard. 4.8.1.2 Features NM provides feature support for policy management, monitoring and querying, alerts and notifications, and an external interface protocol. The policy management features implement specific IT goals that can be specified as policy directives for NM. Monitoring and querying features enable tracking of power consumption. Alerts and notifications provide the foundation for automation of power management in the data center management stack. The external interface specifies the protocols that must be supported in this version of NM. 4.8.1.3 ME Firmware Update On server platforms, the ME FW uses a single operational image with a limited-functionality recovery image. In order to upgrade an operational image, a boot to recovery image must be performed. Unlike last generation platforms, the ME FW does not support an IPMI update mechanism except for the case that the system is configured with a dual-ME (redundant) image. In order to conserve flash space, which the ME FW shares with BIOS, EPSD systems only support a single ME image. For this case, ME update is only supported by means of BIOS performing a direct update of the flash component. The recovery image only provides the basic functionality that is required to perform the update; therefore other ME FW features are not functional therefore when the update is in progress.

82 Revision 1.1 Intel order number G34153-003 4.8.1.4 SmaRT/CLST The power supply optimization provided by SmaRT/CLST relies on a platform HW capability as well as ME FW support. When a PMBus*-compliant power supply detects insufficient input voltage, an overcurrent condition, or an over-temperature condition, it will assert the SMBAlert# signal on the power supply SMBus* (that is, the PMBus*). Through the use of external gates, this results in a momentary assertion of the PROCHOT# and MEMHOT# signals to the processors, thereby throttling the processors and memory. The ME FW also sees the SMBAlert# assertion, queries the power supplies to determine the condition causing the assertion, and applies an algorithm to either release or prolong the throttling, based on the situation. System power control modes include: 1. SmaRT: Low AC input voltage event; results in a onetime momentary throttle for each event to the maximum throttle state 2. Electrical Protection CLST: High output energy event; results in a throttling hiccup mode with fixed maximum throttle time and a fix throttle release ramp time. 3. Thermal Protection CLST: High power supply thermal event; results in a throttling hiccup mode with fixed maximum throttle time and a fix throttle release ramp time. When the SMBAlert# signal is asserted, the fans will be gated by HW for a short period (~100ms) to reduce overall power consumption. It is expected that the interruption to the fans will be of short enough duration to avoid false lower threshold crossings for the fan tach sensors; however, this may need to be comprehended by the fan monitoring FW if it does have this side-effect. ME FW will log an event into the SEL to indicate when the system has been throttled by the SmaRT/CLST power management feature. This is dependent on ME FW support for this sensor. Please refer ME FW EPS for SEL log details. 4.8.1.4.1 Dependencies on PMBus*-compliant Power Supply Support The SmaRT/CLST system feature depends on functionality present in the ME NM SKU. This feature requires power supplies that are compliant with the PMBus Specification, Revision 1.2. 4.9 EU Lot 6 Mode The European Union has set forth a stringent standby power consumption target for systems that are used as primary computing devices in office environments. Owing to the fact in office environments, pedestal servers are being used more and more as workstations and that Value servers could make their way into Home servers, this solution is being requested for some pedestal servers. HW support for EU Lot6 will only be available for specific EPSD pedestal products. In order to meet the standby power requirements for EU Lot6 it is necessary to remove power to the BMC, along with other components, when in the S5 state. As this operational mode impacts system feature support, the user has the option of enabling and disabling this mode from the BIOS setup screen utility. BIOS is responsible for enabling/disabling the system hardware for EU Lot6 operation. It notifies the BMC of the current state with the OEM command Set EULot6 Mode. The BMC saves the state in persistent store and uses it to control special EU Lot6 internal processing during boot, sensor monitoring, and so on as needed.

Revision 1.1 83 Intel order number G34153-003 Wake-on-LAN (WOL) is not supported in EU Lot6 mode. 4.9.1.1 Impact to System Features The following system features are lost or impacted when EU Lot6 mode is enabled:  Increased boot time (~15-20s) when system is DC power cycled. o This is due to the fact that both the BMC and BIOS are booting at the same time when the system is powered on (to S0 state). BIOS will need to allow extra time for the BMC to initialize to the point where it can communicate with BIOS. Note: Even when EU Lot6 is not enabled and the system is AC cycled, this increased boot time is applicable if a user immediately attempts to power the system up (for example, pressing the power button), as in this case both the BMC and BIOS are booting at the same time.  No LAN manageability when on standby, and therefore no remote OOB power on by BMC.  No support for SOL or KVM for monitoring the entire boot process. o Since BMC is initializing at the same time as BIOS, it will not be possible to have an SOL or KVM session established from the beginning of the system boot.  FP status LED will behave differently (it will be off when on standby) rather than showing fault conditions present at the time the system was powered down.  No beep code due to uninstalled CPU.  No monitoring of any sensors when on standby.  No detection/logging of any ThermTrip faults. o These result as the system is shut down by HW so BMC will not be available to detect that they occurred.  Sensor monitoring after DC power-on will be delayed by the time it takes BMC to initialize its sensor subsystem (~15 to 20s), possibly losing SEL events or failing to provide correct FP LED status LED indication. Note: This delay occurs on each AC cycle even when EU Lot6 mode is disabled.  Chassis intrusion not detected when in standby

84 Revision 1.1 Intel order number G34153-003 5. System Security 5.1 BIOS Password Protection The BIOS uses passwords to prevent unauthorized tampering with the server setup. Passwords can restrict entry to the BIOS Setup, restrict use of the Boot Popup menu, and suppress automatic USB device reordering. There is also an option to require a Power On password entry in order to boot the system. If the Power On Password function is enabled in Setup, the BIOS will halt early in POST to request a password before continuing POST. Both Administrator and User passwords are supported by the BIOS. An Administrator password must be installed in order to set the User password. The maximum length of a password is 14 characters. A password can have alphanumeric (a-z, A-Z, 0-9) characters and it is case sensitive. Certain special characters are also allowed, from the following set: ! @ # $ % ^ & * ( ) - _ + = ? The Administrator and User passwords must be different from each other. An error message will be displayed if there is an attempt to enter the same password for one as for the other. The use of “Strong Passwords” is encouraged, but not required. In order to meet the criteria for a “Strong Password”, the password entered must be at least 8 characters in length, and must include at least one each of alphabetic, numeric, and special characters. If a “weak” password is entered, a popup warning message will be displayed, although the weak password will be accepted. Once set, a password can be cleared by changing it to a null string. This requires the Administrator password, and must be done through BIOS Setup or other explicit means of changing the passwords. Clearing the Administrator password will also clear the User password. Alternatively, the passwords can be cleared by using the Password Clear jumper if necessary. Resetting the BIOS configuration settings to default values (by any method) has no effect on the Administrator and User passwords. Entering the User password allows the user to modify only the System Time and System Date in the Setup Main screen. Other setup fields can be modified only if the Administrator password has been entered. If any password is set, a password is required to enter the BIOS setup. The Administrator has control over all fields in the BIOS setup, including the ability to clear the User password and the Administrator password. It is strongly recommended that at least an Administrator Password be set, since not having set a password gives everyone who boots the system the equivalent of Administrative access. Unless an Administrator password is installed, any User can go into Setup and change BIOS settings at will. In addition to restricting access to most Setup fields to viewing only when a User password is entered, defining a User password imposes restrictions on booting the system. In order to simply boot in the defined boot order, no password is required. However, the F6 Boot popup prompts for a password, and can only be used with the Administrator password. Also, when a

System Security Revision 1.1 85 Intel order number G34153-003 User password is defined, it suppresses the USB Reordering that occurs, if enabled, when a new USB boot device is attached to the system. A User is restricted from booting in anything other than the Boot Order defined in the Setup by an Administrator. As a security measure, if a User or Administrator enters an incorrect password three times in a row during the boot sequence, the system is placed into a halt state. A system reset is required to exit out of the halt state. This feature makes it more difficult to guess or break a password. In addition, on the next successful reboot, the Error Manager displays a Major Error code 0048, which also logs a SEL event to alert the authorized user or administrator that a password access failure has occurred. 5.2 Trusted Platform Module (TPM) Support The Trusted Platform Module (TPM) option is a hardware-based security device that addresses the growing concern on boot process integrity and offers better data protection. TPM protects the system start-up process by ensuring it is tamper-free before releasing system control to the operating system. A TPM device provides secured storage to store data, such as security keys and passwords. In addition, a TPM device has encryption and hash functions. The server board implements TPM as per TPM PC Client Specifications revision 1.2 by the Trusted Computing Group (TCG). A TPM device is optionally installed onto a high density 14-pin connector labeled “TPM” on the server board, and is secured from external software attacks and physical theft. A pre-boot environment, such as the BIOS and operating system loader, uses the TPM to collect and store unique measurements from multiple factors within the boot process to create a system fingerprint. This unique fingerprint remains the same unless the pre-boot environment is tampered with. Therefore, it is used to compare to future measurements to verify the integrity of the boot process. After the system BIOS completes the measurement of its boot process, it hands off control to the operating system loader and in turn to the operating system. If the operating system is TPMenabled, it compares the BIOS TPM measurements to those of previous boots to make sure the system was not tampered with before continuing the operating system boot process. Once the operating system is in operation, it optionally uses TPM to provide additional system and data security (for example, Microsoft Vista* supports Bitlocker drive encryption). 5.2.1 TPM security BIOS The BIOS TPM support conforms to the TPM PC Client Implementation Specification for Conventional BIOS and to the TPM Interface Specification, and the Microsoft Windows BitLocker* Requirements. The role of the BIOS for TPM security includes the following:  Measures and stores the boot process in the TPM microcontroller to allow a TPM enabled operating system to verify system boot integrity.  Produces EFI and legacy interfaces to a TPM-enabled operating system for using TPM.  Produces ACPI TPM device and methods to allow a TPM-enabled operating system to send TPM administrative command requests to the BIOS.

86 Revision 1.1 Intel order number G34153-003  Verifies operator physical presence. Confirms and executes operating system TPM administrative command requests.  Provides BIOS Setup options to change TPM security states and to clear TPM ownership. For additional details, refer to the TCG PC Client Specific Implementation Specification, the TCG PC Client Specific Physical Presence Interface Specification, and the Microsoft BitLocker* Requirement documents. 5.2.2 Physical Presence Administrative operations to the TPM require TPM ownership or physical presence indication by the operator to confirm the execution of administrative operations. The BIOS implements the operator presence indication by verifying the setup Administrator password. A TPM administrative sequence invoked from the operating system proceeds as follows: 1. User makes a TPM administrative request through the operating system’s security software. 2. The operating system requests the BIOS to execute the TPM administrative command through TPM ACPI methods and then resets the system. 3. The BIOS verifies the physical presence and confirms the command with the operator. 4. The BIOS executes TPM administrative command(s), inhibits BIOS Setup entry and boots directly to the operating system which requested the TPM command(s). 5.2.3 TPM Security Setup Options The BIOS TPM Setup allows the operator to view the current TPM state and to carry out rudimentary TPM administrative operations. Performing TPM administrative options through the BIOS setup requires TPM physical presence verification. Using BIOS TPM Setup, the operator can turn ON or OFF TPM functionality and clear the TPM ownership contents. After the requested TPM BIOS Setup operation is carried out, the option reverts to No Operation. The BIOS TPM Setup also displays the current state of the TPM, whether TPM is enabled or disabled and activated or deactivated. Note that while using TPM, a TPM-enabled operating system or application may change the TPM state independent of the BIOS setup. When an operating system modifies the TPM state, the BIOS Setup displays the updated TPM state. The BIOS Setup TPM Clear option allows the operator to clear the TPM ownership key and allows the operator to take control of the system with TPM. You use this option to clear security settings for a newly initialized system or to clear a system for which the TPM ownership security key was lost. 5.2.3.1 Security Screen To enter the BIOS Setup, press the F2 function key during boot time when the OEM or Intel logo displays. The following message displays on the diagnostics screen and under the Quiet Boot