In the laboratories of the Institute

JINR MIСC: a new level of development

On 17 April, the next stage of modernization of the JINR Multifunctional Information Computing Complex (MICС) was completed at the Meshcheryakov Laboratory of Information Technologies of JINR. At this stage, the engineering equipment providing power supply and cooling was upgraded: climate control systems, power distribution modules.

Module No. 2 was modernized and initial modernization of module No. 1 was carried out. These modules house server equipment and the Tier-2 computing cluster that is the most productive in the Russian Data Intensive Grid (RDIG). The work was carried out by the MLIT staff headed by Chief Engineer of the Laboratory Andrey Dolbilov and Chief Engineer of the MICC Alexey Vorontsov.

A module is a system of airflow isolation that combines two rows of freestanding racks with equipment by means of a ceiling panel and a system of sliding doors that provides cooling. Thus, an insulated cold corridor is developed inside the module; heated air is discharged into the common space of the MICC hall.

In total, in addition to the standalone supercomputer "Govorun" and robotic tape libraries IBM TS3500 and IBM TS4500, there are four such isolated modules in the machine hall of the 2nd floor of MLIT. This is the module providing the operation of the Tier-1 computing cluster, the modules No. 1 and No. 2 - "responsible" for Tier-2, as well as the MICC module No. 4 - providing the operation of Tier-2, heterogeneous HybriLIT platform, cloud infrastructure, complex of information and computing resources and EOS data storage. Module No. 4 was developed in 2020, it is equipped with advanced engineering equipment and before modernization differed quite a lot in engineering solutions from modules No. 1 and No. 2. Also, a module that houses critical servers of standard business computing type (administrative systems and databases, others) and a network equipment module that houses the basic network services for MICC, local and global networks are in the machine room. In the foreseeable future, module No. 5 will be developed that will combine the network infrastructure with extra computing resources and data storage systems.

Starting from the end of 2024, a lot of work has been implemented to improve the engineering infrastructure of the MICC. The required upgrade of the equipment providing power supply and cooling of the computing complex was carried out. This stage of modernization was carried out without stopping the operation of all active equipment and at the same time, it was carried out promptly, with little loss of time and availability. It was necessary to design and develop our own power distribution modules (PDM). This complex and voluminous work was successfully carried out by MLIT senior engineer Andrey Evlanov. Assembly, installation and connection of the PDMs for the MICC modules No.1 and No. 2 were carried out at the end of 2024. Guaranteed power supply cabinets from Galaxy 7000 uninterruptible power supplies were installed and put into operation, eight PDMs were mounted in each module. Preparations were started to connect three-phase power to the PDMs. Later, in March-April, the modules No.1 and No. 2 were totally switched to three-phase power supply in the first turn.

At the same time, during the modernization, it was decided to significantly increase the size of module No. 2. A total of 10 new equipment cabinets were added to the module. To ensure efficient cooling of the upgraded module configuration, 10 inter-row air conditioners were installed. In the future, during the next phase of modernization, similar activities are scheduled for module No. 1.

This work was accompanied by the physical relocation of five cabinets with servers involved in the operation of such services as Tier-2 dCache, Tier-2 WN, Main EOS and cvmfs. For this reason, the laboratory had to announce in advance the downtime in the global WLCG (Worldwide LHC Computing Grid) infrastructure during the period from 27 March to 31 March in order to avoid the loss of running user tasks and to ensure the safety of data.

"The capacities of both power supply and cooling have increased by about 30%, we managed to significantly improve their efficiency. Expansion of the number of rack cabinets gave extra opportunities to install new equipment and to increase the computing power of Tier-2 and the development of JINR MIICC as a whole," Andrey Dolbilov said.

Improvement of the MICC engineering infrastructure has been carried out in accordance with the Seven-Year Plan for Development of JINR in 2024-2030 that in particular, implies that the data storage capacity and computing power for the WLCG project that is aimed at meeting tasks related to JINR participation in CERN experiments should be increased annually by 10-20%. It will allow maintaining the required data processing speed. The modernization provides the possibility to install new equipment and to extensively upgrade the system that will ensure the most reliable operation of the entire MICC complex.

"I would like to highlight the huge amount of work on disconnection and removal of obsolete equipment that has been carried out by the MICC staff. Our staff of young engineers and on-duty personnel work with great dedication and commitment. They ensure the smooth operation of the MICC all year round in a 24x7 mode and make a huge contribution to its development. It is also thanks to these people that our computing centre plays a prominent role in the international scientific community," Alexey Vorontsov emphasized.

Recall that in January 2025, the multilevel data storage and processing system of the Govorun supercomputer was expanded by two new servers based on Ruler technology with a total capacity of 2 petabytes. By now, these servers have been put into operation and are widely used for computational tasks of the NICA Megascience Project. After commissioning, the total volume of the ultra-fast part of the hierarchical data storage system is 10.6 petabytes that will significantly reduce the total time of the Govorun supercomputer tasks.

As reported by MLIT