ACS-Door HX Open - Compute - Requirements For Open Rack - Rev1.0

Download as pdf or txt
Download as pdf or txt
You are on page 1of 26

ACS Door Heat Exchanger Requirements for Open Rack

Revision 1.0

Authors: John Fernandes & Jacob Na (Facebook), Stefan Djuranec & Michael
Thompson (nVent)
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Table of Contents
1. SCOPE 3
2. LICENSE 3
3. ACKNOWLEDGEMENTS 3
4. REVISION TABLE 3
5. TERMINOLOGY 3
6. POTENTIAL APPLICATIONS 4
6.1 Traditional real door heat exchanger (air-to-liquid) 4
6.2 Liquid-cooled hybrid solution (air-to-liquid and liquid-to-liquid) 5
6.3 Air-assisted solution (liquid-to-air) 6
7. PHYSICAL REQUIREMENTS 6
7.1 Physical dimension of Door Heat Exchanger (Door HX) 6
7.2 Layout Impact to Data Center 7
7.3 Physical Interfaces connections 8
8. OPERATING (DC ENVIRONMENTAL) CONDITIONS 9
8.1 Air-side conditions 9
8.2 Water-side conditions 9
9. PERFORMANCE / METROLOGY 10
10. MONITORING & CONTROL 12
11. SERVICEABILITY 15
12. RELIABILITY & QUALITY 16
12.1 Pressure/leak testing 16
12.2 Sealing of in-rack airflow 18
13. ENVIRONMENTAL & REGULATIONS 19
13.1 Storage, Transportation & Contamination 19
13.2 Vibration & Shock 19
13.3 Regulations / Compliance 20
13.4 Labels and Markings 21
14. APPENDIX A 21
14.1 Mounting of adapter frame to Open Rack V3 21
15. REFERENCES 25
16. ABOUT OPEN COMPUTE PROJECT 26

2
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

1. SCOPE
This document defines the technical requirements and potential applications for a rack-mount door heat
exchanger to be used with Open Rack. This document is specifically scoped for a rear-side, rack mounted
heat exchanger solution; however, physical dimensions of the heat exchanger should be compatible for
both rear and front mounting.

2. LICENSE
© Copyright Facebook Inc., Schroff Inc., 2021. All rights reserved.

3. ACKNOWLEDGEMENTS
The ACS Door Heat Exchanger Requirements for Open Rack was created through collaboration with many
participating companies facilitated by the ACS Door Heat Exchanger sub-project under the Rack and Power
group.
The ACS Door Heat Exchanger sub-project would like to kindly acknowledge the following for their
contributions to this document:
● Robert Sewell, 2BM Ltd.
● Fabio Polo, Bluebox
● Jason Zeiler, Sebastian Varela, Cam Turner, CoolIT Systems Inc.
● Mark Luxford, Cooltera Ltd.
● Harsha Bojja, Tiffany Jin, Noman Mithani, Facebook Inc.
● Matthew Archibald, nVent
● Juan Carlos Cacho Alonso, Rittal

4. REVISION TABLE

Revision Date Comments


0.9 August 10, 2020 Draft for IC review
0.95 January 21, 2020 Adjustment to requirements document; modification of license section
Addition of appendix A; adjustments to accommodate two-phase
1.0 June 15, 2021
coolants

5. TERMINOLOGY
The following is a list of terms or acronyms used frequently in this document.

3
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

• Door HX, rack-mount door heat exchanger assembly


• CDU, coolant distribution unit. A component used to drive a closed loop of coolant that absorbs
heat from IT cabinets or equipment and rejects it to the facility cooling (water) system. This
component can be sized to mount within an IT cabinet or installed on the data center floor to
support multiple IT cabinets.
• RPU, reservoir and pumping unit. A component like the CDU, but without a heat exchanger to
reject heat to the facility cooling system. It is designed to drive a closed-loop of coolant within or
across IT cabinets and, when combined with a liquid-to-air heat rejection device like a Door HX,
exhaust heat to facility air. Therefore, a cooling system employing an RPU can function in a facility
designed for 100% air-cooling.
• TCS, Technology Cooling System, is the cooling system from the CDU to IT cabinets or equipment.
This term is defined by ASHRAE to describe a type of liquid cooling implementation [1].
• FWS, Facilities Water Systems, is the facility cooling system. This term is defined by ASHRAE to
describe a type of liquid cooling implementation [1].

Figure 1: Simplified depiction of a traditional rear door heat exchanger

6. POTENTIAL APPLICATIONS
6.1 Traditional real door heat exchanger (air-to-coolant)
The most widespread application of a door heat exchanger is where the solution replaces the rear
door of a rack and relies on the facility-provided coolant to make the rack thermally neutral to the
room. Figure 1 provides a simplified representation of a rack outfitted with a rear door heat
exchanger. In general, IT hardware within the rack is air-cooled and the door heat exchanger uses
4
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

facility coolant to absorb heat from exhaust air to return air to the facility at or near inlet air
temperature to the rack. This rear door heat exchanger can either be a passive or active solution.
The latter has fans integrated that help drive airflow to achieve the required amount of cooling and
minimize/counteract back pressure on IT chassis fans. Depending on requirements, a passive
solution may or may not have integrated sensors for monitoring.

Figure 2: Simplified depiction of a rear door heat exchanger and facility coolant used to enable liquid
cooling of IT gear

6.2 Liquid-cooled hybrid solution (air-to-coolant and coolant-to-coolant)


This solution is an extension of the traditional approach with the added feature of liquid-cooling IT
hardware components in the rack. Availability of facility coolant at the rack is leveraged to liquid-
cool high-power components in IT chassis. Rack-level liquid cooling infrastructure such as cold
plates and manifolds are required. A CDU may also be employed for better control of coolant to the
cold plates in each IT chassis. The rear door heat exchanger continues to serve a traditional
function of capturing heat from exhaust air in order to return air to the room at or near the inlet air
temperature to the rack. This approach may help enable higher IT component and rack powers.
The rear door heat exchanger can be either a passive or active design. Careful selection of the
Door HX and its internal circuitry must be considered so that the Door HX can achieve the cooling
performance required at a flow rate that matches liquid cooling requirements of the IT chassis.
If a two-phase coolant is employed, appropriate design modifications would have to be considered.

5
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Figure 3: Simplified depiction of hybrid solution which use facility air to cool an internal coolant loop

6.3 Air-assisted solution (coolant-to-air)


This solution can be envisioned as the opposite of the one described in section 2.2. Heat
exchanged in an internal (closed) coolant loop is exhausted to facility air using a door heat
exchanger. Figure 3 provides a simplified depiction of how this system functions. Rack-level liquid
cooling infrastructure such as cold plates, manifolds and CDU (coolant distribution unit; term used
in a general sense here) are used to cool high-power IT components such as CPUs, GPUs, etc.
The CDU drives the internal fluid loop and the door heat exchanger exhausts heat by using exiting
air from the IT gear. Therefore, such a rack is 100% air-cooled and cold plates installed on
components will receive above-ambient temperature coolant for heat transfer. This solution can
provide the benefits of liquid cooling in an air-cooled facility. The Door HX can be either a passive
or active type and is selected based on the capability of IT (server) fans and required cooling
capacity (kW).
Designs must consider tube routing to the CDU. The door hinges should be placed on the side of
the frame that allows the Door HX to open and close without stressing the tubes.

7. PHYSICAL REQUIREMENTS
7.1 Physical dimension of Door Heat Exchanger (Door HX)
Face area dimensions of the Door HX should be compatible with Open Rack architectures.
Installation of the door to a rack should not impact deployable rack-to-rack pitch or require the
addition of auxiliary components. Guidance for Door HX designs, whether active or passive, are as
follows:

6
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

● The overall width of Door HX should be within the Open Rack’s primary structure/frame,
including structural elements or the external frame housing the heat exchanger core and
an adapter frame for mounting against the rack.
● The overall height of Door HX should be within Open Rack’s primary structure/frame, not
including components touching the facility floor such as castors and levelling feet. The only
components permitted to extend beyond said definition are coolant connections and
physical interfaces specific to active doors such as power delivery and communication. The
adapter frame may have adjustable feet for height alignment and load bearing. In general,
leveling feet (or features) should be compliant with rack-specific requirements.
● Maximum preferred depth is 1ft (305mm), including the adapter frame for mounting against
the rack. The detail dimension of rack and depth of Door HX is shown in Fig. 4 [2]. This
requirement may be violated as long as the aisle spacing and/or ability to open the door by
at least 90° (as outlined in Fig. 5) are met. Note that this should be agreed upon between
the end-user and solution provider.
See Appendix A for Door HX mechanical requirements specific to Open Rack V3 with in-rack blind-mate
manifolds and RPU (reservoir and pumping unit).

Figure 4: Rear and top view of Open Rack showing maximum dimensions of mounted Door HX assembly

7.2 Layout Impact to Data Center


The Door HX shall be removable and mounted to Open Rack with a hinged solution for quick access
to the rear side of the rack for easy servicing and maintenance, such as fans, power-bus bar,
manifolds, etc. The operation of the Door HX should not impact any pre-existing data center layout
or design aspects. This includes:

7
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

● When mounted to a rack and primed with coolant, the combined weight should be ≤1400kg
(3086lbs) for Open Rack v2 and ≤1600kg (3520lbs) for Open Rack v3. Correspondingly,
dry weight of the Door HX assembly and fluid volume should be provided for engineers to
determine if this could meet data center floor loading limitations based on coolant
employed. The maximum weight of Door HX with fluid volume shall be less than 150kg.
The preferred weight of Door HX with fluid volume should be ≤ 100kg.
● Depending on layout and design of the data center, width of the aisle between rack faces
can vary. The door heat exchanger defined here will require a minimum 3.94ft (1200mm)
spacing in the aisle to permit opening of the hinged door heat exchanger at least 90°
without interfering with adjacent racks as well as those across the aisle. The diagram
describing this detail is in Fig. 5.

Figure 5: Required minimum aisle width to accommodate opening of Door HX by 90°

7.3 Physical Interfaces connections


As previously outlined, a Door HX may be either a passive or active variant. Depending on the
type, following connections might be required
● Connections to facility coolant: Connections should be permitted from both top and/or
bottom sides and should accommodate different interface types for flexibility on-site.
Rigid pipe terminations at the door heat exchanger assembly are recommended to be 1”
ID BSP type, with potential to accommodate/grow up to 1.5” ID. There should not be
more than two connections to facility coolant; one connection for cold-coolant supply and
another for hot-coolant return.
● Service connections: To facilitate draining and bleeding of a Door HX, service ports at
both the top and bottom sides should be included.
● Electrical connections: The active Door HX should include active fans mounted on the
accessible side of heat exchanger assembly. To provide power to the assembly, 110 or
230VAC (50/60Hz) supply options via standard C14 or C20 plug-in connector are
recommended. Additional recommendations include door heat exchanger assembly being
powered by the IT rack it is mounted against as well as support for A/B power supply
redundancy (AC only). DC power may be supplied directly from the Open Rack busbar to

8
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

the Door HX solution. In this case, it’s important to ensure that PSU and BBU shelves, and
bus-bar design account for additional load.
● Communication: RS485 or RJ45 (Ethernet) ports should be supported along with the
following protocols - Modbus RTU/IP, SNMP v2/3, HTTP (webserver) or IPv4/6. Similar to
electrical connections, from a communication standpoint, it is recommended the Door HX
assembly be a part of the rack it supports.

8. OPERATING (DC ENVIRONMENTAL) CONDITIONS


This section outlines supply conditions and operational targets of cooling resources provided by the facility
for both IT gear within the rack as well as the Door HX. The final thermal solution of the system should be
optimized and energy-efficient under both data center environmental and server operational conditions with
the lowest capital and operating costs.

8.1 Air-side conditions


OCP Ready guidelines stipulate that compliant gear should meet Class A1 allowable environmental
specifications as outlined by ASHRAE [3].

8.2 Water-side conditions


Following details outline the data center water-side supply or operating conditions.
● Coolant supply (inlet) temperature is a function of the data center or facility’s capability. As
general guidance, it’s recommended to not drop below 16°C to avoid excessive
condensation in the door heat exchanger solution. This is within range of W1 class by
ASHRAE’s definition [4]. Note, this does not stipulate that coolant temperature be
maintained between 16~17°C, but rather defines a minimum requirement.
● Coolant-side temperature difference across Door HX: This value should be maximized to
boost efficiency of facility-level cooling infrastructure, unless explicitly specified/defined by
the data center (for example, ≥ 8°C). In practice, this parameter would depend on desired
air-side performance and cooling capacity, it is usually specified by the customer, and
involves a tradeoff between Door HX and chiller performance.
● Supported coolant types: May include water, glycol/water mixture and single or two-phase
dielectric (highlighted for reference only). Coolant is usually specified by the customer.
● Operating pressure is a function of the data center or facility’s capability. As general
guidance, a floor-level coolant distribution unit may have a range of 140 – 450kPa [5].
Direct use of facility water may expose the equipment to higher operating pressures,
especially in multi-story facilities. The door heat exchanger assembly must be designed to
operate under said conditions.
● If water is used as the coolant for the door heat exchanger assembly, it is recommended
to follow ASHRAE’s guidelines for water quality [4]. Note there are specific
recommendations based on whether the door heat exchanger assembly is connected to
the FWS (Facilities Water System) or TCS (Technology Cooling System) loops.
○ Facility level coolant: Chapter 5, Table 5.3 Water Quality Specifications for Facility
Water System (FWS) Loop
○ Equipment-level coolant: Chapter 6, Table 6.2 Water Quality Specifications - TCS
Cooling Loop

9
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

In addition, the following guidance may also be leveraged - VDI 2035 Part 2, Table 1
• Filtration requirements depend on the type of implementation. In the traditional application
of a Door HX, IT gear in the rack is air-cooled, and filtration at the facility-level should be
adequate. For a hybrid solution, coolant is used by Door HX and IT gear (cold plates,
manifolds and in-rack CDU). Due to smaller flow dimensions in this case, specific filtration
requirements may need to be addressed, as outlined by ASHRAE [1].

9. PERFORMANCE / METROLOGY

Before performance and operational requirements can be highlighted, parameters of importance should be
defined (as outlined in the following table). These apply to both active and passive types of the Door HX.

Table 1 – Parameters of importance and associated units

Parameter Units

Rated cooling capacity kW

Power consumption (at rated capacity) W

Coolant inlet temperature °C

Coolant outlet temperature °C

Coolant flow rate m3/h or l/min

Coolant system pressure difference kPa; curve based on flow rate


should be provided

Air inlet temperature (approaching door) °C

Air outlet temperature (leaving door) °C

Air flow rate (through door; at 0Pa pressure difference m3/h or ft3/min
between rack interior and room ambient)

Air-side pressure drop across door assembly (for passive Pa; curve based on flow rate should
type) be provided

Requirements are highlighted as follows:

● For the active variant, N+1 rotor/fan redundancy is a must. In the event of a failure, increasing
speed of functioning fans is permitted. To prevent recirculation or leakage through a failed unit,
fans may be equipped with baffles (or equivalent component).

10
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

● Face area of the heat exchanger within the supporting frame should be maximized for performance
and minimal backpressure. Finned footprint of Door HX core should be outlined for
simulation/analysis.
● Curves (or table) of cooling capacity based on given coolant and air flow rates, inlet temperatures
to the door, coolant of choice and operation at sea-level. Curves (or tables) corresponding to fan
failure operation should also be included; specifically, impact to airflow requirements to achieve the
rated cooling capacity.
○ Test conditions should be based on IEC 62610-4 [6] for verification and reporting of
performance parameters such as cooling capacity, etc.
○ For the purpose of defining a rated cooling capacity, the following boundary conditions may
be assumed

Table 2 – Reference conditions for verification of cooling capacity

Parameter Value

Coolant inlet temperature 16°C

Air inlet temperature (approaching door) 45°C

Air outlet temperature (leaving door) 30°C

Altitude 0ft (sea-level)

Coolant type Water (purified)

● For the passive type, low air-side pressure drop is highly recommended to minimize or eliminate
the need for server fans to operate at higher speeds, thereby increasing cooling power. Impedance
curve (pressure drop versus volumetric flow rate) based on sea-level operation should be provided.
○ For the passive type, it’s recommended to maintain a pressure drop ≤ 15Pa at rated airflow
○ For the active variant, total power consumption should be ≤ 2% of rated cooling capacity
(not including operation under fan failure)
● Low coolant-side pressure drop is highly recommended for efficient operation of the facility.
Impedance curves for different coolant operations should be provided.
○ Pressure drop across the entire door heat exchanger assembly (including piping, valves,
sensorics, etc.) should be ≤ 100kPa at rated coolant flow rate
○ Nominal operating coolant pressure should be ≤ 600kPa. This value is adequate to cover
a row-level CDU (coolant distribution unit); however, a facility-level CDU might provide a
higher pressure. In such cases, the vendor and end-user should work together to address
this.
○ Maximum allowable coolant pressure should be ≤ 1000kPa (short-term operation; safety
measure may be triggered beyond this value).

11
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

10. MONITORING & CONTROL

To ensure effective and efficient operation of the door heat exchanger, sensors and control represent
valuable implementations or additions to the solution. Clear definition of sensor parameters (such as
thresholds and alarms) enable the operator to ensure the device continues to function as intended in the
long-term. Since passive and active variants function and may be operated differently,
definitions/requirements are listed separately.

Table 3 – Sensors, controls and alarms for a passive Door HX

SENSORS – REQUIRED

Type Detail/Comment

Air temperature, In More than one sensor preferred

Air temperature, Out More than one sensor preferred

SENSORS - OPTIONAL

Type Detail/Comment

Air differential pressure

Water temperature, In

Water temperature, Out

Water flow rate

Water pressure

Air humidity

CONTROLS – OPTIONAL LOOP TYPE

Parameter Detail/Comment

Air temperature, Out

Air temperature, Differential

Water temperature, Out

Water temperature, Differential

12
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

CONTROLS – MANUAL

Parameter Detail/Comment

Water valve position Manually adjusted at deployment;


Pressure Independent balancing and
Control Valves (PICV) are recommended,
especially for multi-door installations.

ALARMS

Name/Type Detail/Comment

General A single, binary definition that indicates


there is an alarm condition

Power status Binary (ON/OFF)

Air temperature, In

Air temperature, Out

Door open Optional

Leak detection Optional

Air differential pressure Optional

Water temperature, In Optional

Water temperature, Out Optional

Water flow rate Optional

Water pressure Optional

Water valve position Optional

Smoke Optional

As an active rear door heat exchanger has more components and functionality compared to its passive
counterpart, the following definitions/requirements apply in addition to those defined for the passive
solution.

13
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Table 4 – Additional sensors, controls and alarms for an active Door HX

SENSORS – REQUIRED

Type Detail/Comment

Fan speed

Number of fans installed To check deployed configuration

Input power AC or DC input power

CONTROLS – OPTIONAL LOOP TYPE

Parameter Detail/Comment

Air temperature, In

Air differential pressure

CONTROLS – MANUAL

Parameter Detail/Comment

Fan speed Manually adjusted at deployment or in-operation

ALARMS

Name/Type Detail/Comment

Fan speed

Monitoring and reporting should employ the Redfish® standard (Redfish Scalable Platforms Management
API) [7]. For more information, refer to:
https://www.opencompute.org/wiki/Hardware_Management/SpecsAndDesigns

The table below outlines parameters that may be defined in the header section of the Redfish Door HX
JSON file.

Table 5 – Parameters in header section of JSON file

14
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Parameter Detail/Comment

Name

Manufacturer Static

Model number Static

Serial number Static

Manufacture date Static

Door HX type Static; active or passive

Hardware revision

Firmware revision

Physical location Optional

Asset tag Optional

Redfish profile and reference documentation are currently WIP.

11. SERVICEABILITY
In general, the Door HX solution should be designed with ease of maintenance or serviceability in-mind.
This includes (but may not be limited to) the following:
● Due to weight of the charged solution, features such as a spring-loaded, height adjustable swivel
castor or equivalent may be employed for ease and safety of serviceability operations that require
opening of the Door HX. Any implemented features must meet physical requirements outlined in
Section 7.
● Ease of replacing Door HX - draining, un-mounting, installation and priming/charging
● Ease of access to physical interfaces (coolant connections, power supply and communication
cables)
● Collection of expelled coolant (in case of leaks) and condensation in a drip tray with a feature to
enable draining in-operation (or intermediate/external collection in facilities that do not have a
raised floor or run-off area)
○ As a general guidance, coolant supply to the door heat exchanger may be raised to ≥ 16°C
to avoid excessive condensation in the door heat exchanger (or specific value depending
on the air-side operating conditions of the facility)
● Ease of replacing fan modules (on active variant of Door HX)
● Ease of replacing sensors or other accessible parts
● A serviceable air vent located at the highest point of the Door HX to allow for proper on-site charging
of the system. An automatic air vent is recommended to allow air bubbles to escape the system

15
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

over time and to reduce pressure increases due to thermal expansion of the air and coolant during
operation without liquid spilling. Additional considerations need to be taken when the Door HX inlet
and outlets are located at the top of the unit.
● When connected to facility liquid, the fluidic connections must stay clear of regular human traffic
paths but needs to be easily accessible for servicing. For a hybrid solution, fluidic connections
between the Door HX, rack manifolds and CDU must follow the guidelines below.
○ The routing of the flexible rubber hoses must not violate the minimum bend radius
specification of the hose (For both open and closed Door HX positions)
○ During servicing, when the Door HX is in an open position, the hoses should be routed in
such a way to avoid any trip hazards and to allow accessibility to rear features of the CDU
○ Routing should avoid hoses in positions that cause unwanted stress to the fluidic
connections or peripheral equipment such as the CDU or IT gear

12. RELIABILITY & QUALITY

This section will focus on aspects such as pressure or leak testing, sealing of airflow within the rack and
other areas that can influence operational quality of a Door HX solution.

12.1 Pressure/leak testing

Ensuring the door heat exchanger solution has undergone adequate checks to prevent leaks in-
operation is critical to deploying with confidence. Checks at both manufacturer and end-user
locations are generally recommended. However, it’s up to both parties to agree upon a course of
action. Following sections outline recommended procedures for pressure testing and leak
prevention.

● Checks at manufacturer (required)

A common leak detection test is the pressure decay method. The complete door heat
exchanger assembly should be pressurized with gas to a factor times the maximum
allowable pressure (defined as ≤ 1000kPa in the section on Performance & Metrology). In
general, the factor required for pressure testing is based on which standard is employed.
IEC 62638-1 [8] stipulates setting twice (2) the abnormal or single-fault condition and
ASME B31.n series, specifically B31.3 for process piping [9], states setting 1.5x design
pressure (both can be based on maximum allowable pressure as outlined earlier in this
section). As to which standard should be followed, that’s a decision to be made between
manufacturer and end-user.

The leak value can be calculated as follows:𝑄 = (𝑃1 − 𝑃2 )𝑉/𝑡

Where,

Q is the leak rate in mbar.l/s

P1 is pressure at the start of the test in mbar

P2 is pressure at the end of the test in mbar

V is internal volume of the test sample in l (liters)

16
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

t is duration of the test in s (seconds)

Since both P1, V and are known, test duration (t) is the variable that is adjusted to achieve
a desired leak rate (Q) and depends on the sensitivity of the test equipment (primarily, the
pressure transducer). Industry standards define ‘water tight’ as a leak rate of 10-2 mbar.l/s
when helium is employed as the test medium.

Table 6 – Helium leak rate for different seal qualities [10]

Leak definition (seal quality) Helium leak rate (mbar.l/s)

Water-tight 10-2

Vapor-tight 10-3

Bacterial-tight 10-4

Fuel-tight 10-5

Virus-tight 10-6

Gas-tight 10-7

Insulator seal-tight 10-10

Gases other Helium may be employed for testing, such as air or Nitrogen. However, the
leak rate definitions in Table 1 should be adjusted using multiplication factors based on
whether the leak rate qualifies as either laminar viscous or molecular flow.

Table 7 – Converting leakage rate of helium to other gases

Multiply factor (multiply helium leakage rate by)

Medium to convert to

Laminar flow Molecular flow

Argon 0.88 0.316

Air 1.08 0.374

17
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Nitrogen 1.12 0.374

Water vapor 2.09 0.469

Hydrogen 2.23 1.410

Prior to shipment, the product should be charged with Nitrogen at the nominal operating
pressure limit. At the integrator or end-user’s site, if pressure is deemed present at-release
(auditory signal), the product could be considered to have passed transportation.

● Checks at integrator or end-user (recommended)

Prior to deployment, the product (post-integration) should undergo a final pressure check
(like the criteria in Table 7, but with compressed air) prior to charging. To accommodate
this, the Door HX assembly may be outfitted with a separate valve.

● Leak detection in operation (recommended)


○ Regular inspection of the system to check for droplets/leaks
○ Optional electronic leak detection at cabinet or floor level

12.2 Sealing of in-rack airflow

Sealing on the adapter frame and/or Door HX should be adequate to minimize leakage of server
exhaust or entrainment of facility air. It’s important to note that sealing within the rack is equally
critical and should be considered as well.

For an IT rack employing a Door HX, the quality of airflow sealing at both sub-rack (or chassis) and
cabinet (or rack) levels can be determined in accordance with IEC 62610-6 [11]. This standard
helps define metrics for airflow recirculation and bypass, as well as measurement methods and
should serve as guidance for rack integrators. In general, both recirculation and bypass ratios
should be kept to a minimum to ensure cooling efficiency. Following are a few points to take note
of:

● Installation of a rear-mounted door heat exchanger should not impact sub-rack or chassis
inlet temperatures. When employing a passive door heat exchanger, added impedance
should be understood to not severely impact recirculation. This could be addressed with
adequate in-rack sealing.
● A rear-mounted door heat exchanger solution should interface with adequate sealing to
the rack frame to avoid unnecessary, rack-level airflow bypass. When employing an active
door heat exchanger, fan control logic should be appropriately tuned to ensure airflow is

18
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

efficiently used or prioritized through the IT chassis. Once again, adequate in-rack sealing
is critical here.

13. ENVIRONMENTAL & REGULATIONS


13.1 Storage, Transportation & Contamination
The following environmental tests are recommended for execution per MIL-STD-810G [12], IEC
60068-2 [13]. The systems pre- and post-test should be verified for any functional impact.

Table 8 – Non-operational environmental conditions

Parameters System

Gaseous Contamination Severity Level G1 per ANSI/ISA 71.04-1985 [14]

Storage Relative Humidity 10% – 90%

Storage Temperature (Long Term) -40°C – +70°C

Transportation Temperature (Short Term) -55°C – +85°C

System should meet the following set of tests as applicable to reliability, and compliance needs
● Hydrostatic Pressure Test
○ Normal, Abnormal operation, and Creep resistance based hydrostatic pressure
tests per IEC/UL62638-1 are to be executed.
○ The system is expected to meet the criteria of being able to sustain three times,
and two times the maximum working pressure per design specifications while
testing for normal, abnormal operational conditions respectively.
○ Creep resistance tests involve pre-conditioning of the units followed by hydrostatic
pressure tests as listed above with the general requirement of no visible signs of
rupture, cracking, embrittlement, and loose parts, or assemblies.
● Tensile Strength Test
○ Tensile strength of the system is recommended to be tested per ISO 527 [15],
IEC/UL62638-1. Tensile strength of the pre-conditioned unit should not be < 60%
of tensile strength before the test.
○ System should not see any visible signs of cracks, rupture, as well as functional
impact.
● Mechanical Strength Test
○ Force test on critical assemblies, part of the system is recommended to be
executed per IEC/UL62638-1.
○ System should not see any visible signs of cracks, rupture, as well as functional
impact.

13.2 Vibration & Shock


Table 9 - Vibration and shock requirements
19
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Non-Operational

Rack 0.25G, 1-200-1 Hz, 1


Product Sine Sweep Octave/min, 1 Cycle

Product Vibration Test Mechanical Handling-I 1. Rotational Flat Edge


Drop
2. Rotational Corner Drop
3. Side/inclined Impact
4. 22 deg Tilt
5. Flat/Vertical Drop
6. ASTM D880-92 [16],
D6179-07 Level-II [17]

Truck Transportation Test ASTM D4169-16, D4728-17 (1-


200 Hz, 180 min)

Mechanical Handling-II 1. Rotational Flat Edge


Drop
2. Rotational Corner Drop
3. Side/inclined Impact
4. ASTM D880-92, D6179-
07 Level-II

13.3 Regulations / Compliance


● Comply with IEC/UL 60950-1 [18], 62638-1 3rd edition [8]
● General safety construction requirements
○ All products shall not contain any hazardous, flammable, or toxic substances classified by
law or local regulation, and MSDS (Material Safety Data Sheet) must be provided. No liquid
spillage is allowed during normal and abnormal operating conditions.
○ Galvanic corrosion should be mitigated, and essential coating/protection requirements
should be enforced to curtain high temperature, relative humidity conditions.
● Plastic parts flammability
○ Plastic parts shall be made of min. V-1 material.
○ Tubing can be made of min. HB75 class material if the thickness of the material is < 3 mm,
or min. HB40 class material if the thickness of the material is ≥ 3 mm.
● Leakage fail-safe
○ The system shall provide safeguards (such as a barrier or drip pan or supplementary
containment vessel) against any spillage of internal liquid to effectively limit the spread of
the spillage.
○ The leak from tubing connectors and similar joints in the liquid system shall not bridge an
insulation in the final product.
○ If the liquid is corrosive, the leak shall not touch any connection of a protective ground
conductor.

20
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

13.4 Labels and Markings


Labels and marking indicating the compliance of the system and/or sub-assemblies should be
affixed for clear visibility.

14. APPENDIX A
This section of the document covers Door HX mechanical or physical requirements specific to Open Rack
V3 with in-rack blind-mate manifolds and RPU (reservoir and pumping unit).
14.1 Mounting of adapter frame to Open Rack V3
A Door HX can be mounted to the rear of the Open Rack v3 via an adapter frame that interfaces
with the rack on one side and the hinges off the Door HX on the other side. This adapter has certain
mechanical requirements to allow proper mounting, serviceability, and access to data center
features.
• The adapter frame is to be mounted to the Open Rack via M8 screws into nutserts placed
on the rear of the rack at positions shown below.

Figure 6: Rear and isometric views showing M8 threaded insert locations on rear of Open Rack for
mounting Door HX assembly

• Opening of the door by at least 90° will be needed to remove hot and cold manifolds from
the rear of Open Rack in hybrid and air-assisted solutions outlined above. The door will
need to be designed in such a way as to not interfere with an adjacent rack when fully
opened.

21
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Figure 7: Top view of adapter frame and door heat exchanger in open position to allow removal of
manifold (orange)

• Removal of manifolds from Open Rack will need to be possible without the removal of the
adapter frame or Door HX to be serviced in the data center. The manifolds are installed
into the rack via sheet metal brackets on top and bottom and mounted using M8 screws
that thread into nutserts on the rack. The bottom manifold bracket must be in toed into slot
on the bottom of the rack and then the whole manifold rotated along that pivot. Then the
top bracket is flush with the rear surface of the rack.
o Note: the design and mounting of blind mate manifolds is still in progress and is
subject to change. This appendix will be updated as new information becomes
available.

Figure 8: Isometric view of manifold inserted down into slot in Open Rack then fastened to bracket

22
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Figure 9: Isometric view of manifold rotated toward rack after bottom bracket inserted, then mounted to
rear surface

14.2 Layout impact to data center


The operation of the Door HX should not impact any pre-existing data center layout or design
aspects. This includes:
• Leveling feet should be mounted to bottom of adapter frame to provide load bearing and
reduce shearing forces on mounting hardware attaching the adapter frame to the rear of
Open Rack. Additionally, casters may be needed on the Door HX to prevent a large
moment on the hinges between the door and adapter frame.

Figure 10: Isometric view of Door HX opened with provisions for airflow sealing and weight distribution
23
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

• Other considerations for layout:


o Large hose connections (quick disconnect valves) from manifold and Door HX to
RPU should swivel to allow rotation of Door HX without twisting of hoses (hose is
fixed to manifold barb).
o Location of inlet and outlet ports on Door HX will dictate length of hose used, ability
to retract or collect within the adapter frame when Door is closed, and angle at
which Door can be opened.
o Orientation of Door HX hinge on the adapter frame will have an impact of the length
of hose used, ease of quick connections to swivel as Door is opened and closed,
and routing to the hot/cold manifolds and RPU supply/return.
o Closed Door HX assembly should keep from air flow leakage around the outer
perimeter and lower section of door. A gasket or similar material can be used to
seal around the door to the adapter frame and between the adapter frame and rear
surface of rack. In addition, a blanking panel should be added to cover any open
areas that may allow air bypass.
o The Open Rack and Door HX combined assembly should be able to traverse the
data center environment using its caster wheels and be able to climb and descend
ramps in the data center by having a minimum 15-degree approach and exit angle.
This is measured from the tangent of the caster wheel to the outermost surface of
the assembly.
o Keep-outs: the adapter frame shall have keep-outs allowing the installation and
removal of the manifolds, access to mounting hardware for the bus bar on Open
Rack, and access to mounting hardware for the bottom bracket of manifolds.

Figure 11: Isometric view of hose lengths connecting manifold and RPU to Door HX

Figure 12: Side view of minimum approach and exit angles needed to traverse data center

24
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

Figure 13: Isometric view of keep outs required for manifold and bus bar mounting hardware

15. REFERENCES
[1] “Water-Cooled Servers - Common Designs, Components, and Processes”, ASHRAE White paper by
Technical Committee 9.9, 2019,
http://tc0909.ashraetcs.org/documents/ASHRAE_TC0909_Water_Cooled_Servers_11_April_2019.pdf
[2] Open Rack Specification V2, https://www.opencompute.org/documents/facebook-open-rack-v2-
specification-rev-12
[3] “Thermal Guidelines for Data Processing Environments”, ASHRAE Datacom Series 1, 4th edition, 2015.
[4] “Liquid Cooling Guidelines for Datacom Equipment Centers”, ASHRAE Datacom Series 4, 2nd edition,
2014.
[5] ACS Liquid Cooling Cold Plate Requirements Document,
https://docs.google.com/document/d/1HR5O_bPcgpcjhEMGIejJ6hUor7DDx6_ek6bzvKcc7X8/edit
[6] “Mechanical structures for electronic equipment – Thermal management for cabinets in accordance with
IEC 60297 and IEC 60917 series – Part 4: Cooling performance tests for water supplied heat exchangers
in electronic cabinets”, International Standard, IEC 62610-4, 1st edition, 2013.
[7] https://www.dmtf.org/standards/redfish
[8] “Audio/Video, Information and Communication Technology Equipment – Part 1: Safety Requirements”,
International Standard, IEC 62368-1, 3 edition, 2018.
rd

[9] “Process Piping”, The American Society of Mechanical Engineers (ASME) B31.3, 2018.
[10] Fundamentals of leak detection,
https://www.leyboldproducts.com/media/pdf/90/c7/87/Fundamentals_of_Leak_Detection_EN.pdf
[11] “Mechanical structures for electronic equipment – Thermal management for cabinets in accordance
with IEC 60297 and IEC 60917 series – Part 6: Air recirculation and bypass of indoor cabinets”, International
Standard, IEC 62610-6, 1st edition, 2020.

25
Open Compute Project • ACS Door Heat Exchanger Requirements for Open Rack

[12] “MIL-STD-810G, Department of Defense Test Method Standard for Environmental Engineering
Considerations and Laboratory Tests”, United States Department of Defense, 2014.
[13] “Environmental testing – Part 2: Tests – ALL PARTS”, International Standard, IEC 60068-2, 1st edition,
2020.
[14] “Environmental Conditions for Process Measurement and Control Systems: Airborne Contaminants”,
ANSI/ISA 71.04-1985.
[15] “Plastics – Determination of tensile properties – Part 1: General principles”, International Standard, ISO
527, 2019.
[16] “Standard Test Method for Impact Testing for Shipping Containers and Systems”, ASTM D880-92,
2015.
[17] “Standard Test Methods for Rough Handling of Utilized Loads and Large Shipping Cases and Crates”,
ASTM D6179-07, 2014.
[18] “Information technology equipment – Safety – Part 1: General requirements”, International Standard,
IEC 60950-1, 2.2nd edition, 2013.

16. ABOUT OPEN COMPUTE PROJECT


The Open Compute Project Foundation is a 501(c)(6) organization which was founded in 2011 by
Facebook, Intel, and Rackspace. Our mission is to apply the benefits of open source to hardware and
rapidly increase the pace of innovation in, near and around the data center and beyond. The Open Compute
Project (OCP) is a collaborative community focused on redesigning hardware technology to efficiently
support the growing demands on compute infrastructure. For more information about OCP, please visit us
at http://www.opencompute.org

26

You might also like