The Netherlands has begun enforcing the energy savings obligation’s (ESO’s) data center requirements, notifying operators that they must take immediate action to meet the mandate to maintain the IT space supply air temperatures at 27^oC (80.6^oF). This requirement is particularly challenging for enterprise operators or colocation operators with multiple tenants that have heterogeneous equipment inventories with varied operating temperature maximums.

The Netherlands’ ESO requires that data center operators evaluate and implement 12 designated energy efficiency measures where they are technically feasible, and have a payback period of five years or less (Table 1). The ESO requires that energy users report their conformance with implementing the required energy efficiency measures once every four years. The first report on energy saving should have been submitted in December 2023 and needs to include the following information:

• Implemented energy saving efforts.
• Energy saving efforts that are deemed technically and economically feasible and will be implemented in the next four years (including an implementation schedule).
• Energy saving efforts that will not be implemented due to technical and/or economic constraints. The report must provide information and data that details the technical or economic constraints precluding the implementation of a specified measure (projects with a payback period of five years or less must be implemented).

While data center operators are being notified regarding their conformance to obligation PH1 (set point of the room coolers), there are several other obligations that may be underutilized and are likely to result in follow-up investigations by regulatory authorities:

• Installation of free cooling systems with integrated controls (PH3).
• Full server virtualization (FI1).
• Deployment of power management settings (FI2).
• Deployment of an energy/power tracking, monitoring and reporting system (GA1).

Operators are encouraged to review this list and identify any obligations they are not assessing closely that may impact their operations.

Table 1 The Netherlands’ energy saving obligation requirements

Implications for operators

Netherlands regulatory officials have started to audit submitted reports and are notifying those operators whose reports have been deemed unsatisfactory. It appears that their first enforcement step is being taken with regards to measure number PH1. PH1 states that the temperature at the discharge of the computer room air handling units (CRAH) or computer room air conditioning (CRAC) system should be set at 27^oC (80.6^oF). Two IT operators have reported that their colocation operators (two different providers) were informed that the Netherlands regulatory authority is requiring the operators to increase the IT space operating temperature to 27^oC (80.6^oF) to comply with measure PH1.

There are three important factors to consider regarding the implementation of PH1:

• Several technical and economic impediments may preclude implementing measure PH1. The IT operators have the right to manage their operations to the IT space temperature level that ensures the reliability of their IT infrastructure and is within their facility specific technical and economic constraints.
• The effects of the higher IT space temperatures on the IT equipment should be assessed under controlled conditions to identify any applicable constraints and an acceptable IT space temperature level to ensure the reliability of the operator’s IT infrastructure.
• IT operators need to assess and modify their colocation contract language to ensure that the process(es) required to comply with regulations, such as the EU’s Energy Efficiency Directive (EED) and the Netherlands’ ESO, are defined.

The ESO regulation allows operators to opt out of a designated efficiency measure where there are technical or economic (where a return on investment will take five years or more) obstacles to its implementation.

Increasing IT space temperature: the challenges

Increasing the IT equipment inlet temperature requires caution, as different types and ages of equipment have varying sensitivities to operating temperatures:

• IT equipment that is older than five years, particularly storage devices, may be more sensitive and more likely to fail at higher operating temperatures.
• High workload capacity servers, such as those supporting high-performance computing and AI workloads, and servers with high core count CPUs may require higher cooling levels to run reliably. ASHRAE's most recent data center temperature and humidity guidelines (2021) set a recommended operating range of 18^oC to 22^oC (64.4^oF to 71.6^oF) for high-density systems (Category H1).
• Different manufacturers and equipment models will have different fan speed curves. Some fans are pushed to full output as the IT equipment approaches 27^oC (80.6^oF), with fan power increasing by 10% to 20% of total IT equipment power. The additional IT equipment fan power can negate energy consumption savings in the cooling system.
• The temperature set point for the room coolers must be set based on the air temperature at the inlet to IT equipment, and not at the cooling unit’s discharge point. Ideally, the two temperatures should be the same but may differ in practice. Consistent inlet temperatures above the recommended maximum of 27^oC (80.6^oF) can result in increased IT equipment failure rates as measured by the IT equipment x-factor (the change in the equipment failure rate as inlet temperature increases). The value of the x-factor is dependent on equipment type and manufacturer.

Technical constraints may limit the operating temperature to a value below 27^oC (80.6^oF) to ensure the reliability of the IT infrastructure.

Implementing the higher IT space temperature can be complicated by a lack of specificity in the contract between the colocation operator and the IT tenant, and service level agreements governing data center operations. The operators need to implement an incremental process, discussed below, and recognize that the final air supply or IT equipment inlet temperature will need to be less than 27^oC (80.6^oF) to address legitimate constraints.

Uptime Institute analysis suggests that the temperature should be increased in 0.5^oC to 1.0^oC (1^oF to 2^oF) increments. Energy measurements should be implemented, ideally for a week, on the IT systems (preferably at the rack level), the CRAC and CRAH units, and the chiller system. IT equipment should also be monitored for over-temperature alarms. The procedure for raising IT space temperature in an active data center is discussed in more detail in the Uptime report Considerations of raised supply air temperatures.

Technical and economic feasibility needs to be checked and validated at each temperature increment. Temperature increases should be stopped below 27^oC (80.6^oF) if some portion of the IT equipment begins to show consistent over-temperature alarms, or the overall data center energy consumption increases.

The phased implementation of temperature increases and collection of operational data at each increment will enable the data center and IT operators to document the IT equipment inlet temperature value that minimizes cooling system and IT energy use, while ensuring the reliability of the IT equipment infrastructure. This data should be provided in an updated compliance report justifying a technically and economically acceptable IT space temperature (which may be less than the specified 27^oC/80.6^oF).

Deploying power management functions

A similar challenge is associated with deploying server power management functions on servers with an average utilization of less than 80% (F12). Server power management functions are incompatible with the performance requirements of some workloads due to increased server system latency and associated response time delays,

To comply with this ESO requirement, IT operators need to have a defined process to identify and assign power management compatible applications to power managed servers. Operators need to report the defined process for assessing the use of power management and where it will be deployed on the server infrastructure to the authorities.

In a colocation environment, the direction to implement power management will be directed to the colocation data center operator; the IT tenants will need to implement power management where it is technically feasible. Again, this needs to be clearly defined in the governing contract and IT operators need to have a process in place for informing their colocation operators of their performance.

An IT operator should consider preparing a single ESO report for their overall Netherlands’ IT operations and submitting it separately from the colocation operator. The colocation operator can direct the authorities to the individual reports prepared by their tenants to demonstrate compliance.

Using clear contractual language

IT and colocation operators share responsibility for meeting regulatory requirements. This highlights the importance for clear contractual terms addressing legislative and regulatory mandates – which is often missing from most current contracts.

There needs to be more consensus within the industry regarding the best approach to delegating compliance responsibilities across all parties involved in a colocation agreement. Uptime has identified the need for mutually agreeable language governing data exchange (see Colo and cloud contracts need sustainability data exchange). This report discusses obligations under the EU’s Corporate Sustainability Reporting Directive (CSRD) and related climate disclosure requirements in other jurisdictions (see EU data center sustainability laws begin to take shape), the EU’s EED data reporting requirements and the energy management system (EMS) audit and action plan reporting requirements.

Next steps

The Netherlands' assessment of and follow-up on operators' ESO reporting indicates that some EU member states will likely exercise their regulatory authority in relation to country-level energy efficiency regulations and the EED. Under the EED, member states are likely to assess that information and key performance indicators were accurately reported and submitted on time; and that EMS energy audits and action plan reports were also thorough and completed as per the published schedule.

To meet the energy efficiency mandate requirements, operators need to prepare now. Actions to initiate include:

• Create a detailed inventory of IT equipment characteristics, including equipment model type, CPU SKU (stock keeping unit) and core count, memory quantity, number of storage devices, Sever Efficiency Rating Tool efficiency, and PerfCPU (the geometric mean of the seven normalized 100% performance CPU worklet measurements).
• Monitor and collect IT equipment utilization data at the equipment level, as well as power consumption at the rack or power distribution unit level.
• Establish a process to determine applications that are compatible with power management functions and deploy those applications on power managed servers.
• Install temperature sensors and management software to build an IT space temperature profile and automate the control of IT space CRAH and CRAC units. The use of an automate software system will facilitate the control the IT space temperature as close to 27^oC as feasible while maximizing energy savings.
• Automate the control of the central chiller system and maximize the cooling efficiency ratio (the ratio of the cooling energy delivered divided by the energy consumed to provide the cooling) and utilization of the available free cooling capacity.

Coordinating the organizational commitment, budget and project implementation schedules required to implement the full menu of actions in existing data centers will likely take several years. Fortunately, the items listed above will more than pay for themselves: they will deliver improved business and environmental performance and a foundation for compliance with future regulatory requirements.

Note: The regulatory analysis provided in this update is the opinion of Uptime Intelligence. Data center operators should validate the interpretations with their legal staff and any relevant regulatory authorities.