Scribd Logo
Upload

Welcome to Scribd!

  • 3 views

    Pratyush

    Uploaded by

    Nilesh Nirale

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Pratyush

    Uploaded by

    Nilesh Nirale
    3 views18 pages

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    3 views18 pages

    Pratyush

    Uploaded by

    Nilesh Nirale

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 18

    Ministry of Earth Sciences, Govt.

    of India
    Indian Institute of Tropical Meteorology, Pune
    ➢ It is part of the MoES endeavor to provide
    world class services to the people of
    India.
    ➢ With 4.0 Peta Flops HPC facility, India is
    expected to rise from the 368th position
    to around top 30 in the Top500 list of HPC
    facilities in the world
    Cray XC40 System System Configuration

    Cray XC40 18
    Cabinets
    Blower Cabinets 12
    Interconnect Cray Aries with Dragonfly
    Network topology
    Total Peak 4006 TFLOPS
    Performance
    Compute Node
    Number of Nodes 3315
    Processor Intel Xeon Broadwell E5-
    2695 v4 CPU
    (18 core, 2.1 GHz)
    Memory Per Node 128 GB
    Total Memory 414 TB
    Cray XC40 System System Configuration

    Accelerator Node (1%)


    Number of Nodes 16
    Accelerator Intel KNL 7210, self-hosted
    mode
    Memory Per Node 96 GB
    (GB)
    Total Peak 42.56 TFLOPS
    Performance
    Additional Nodes
    External Login 5
    Nodes
    Utility Servers 8
    Utility Racks 1
    Cray XC40 System System Configuration

    Storage System Configuration


    Parallel File System Lustre
    Lnet Nodes 39
    Storage Array Sonexion 3000
    Storage Racks 7 racks + 6 Tape library
    frames
    PFS – Home
    Usable Storage 9.7 PB @ 246 GB/s
    Sonexion One MMU + 26 SSU
    Configuration
    Lustre I/O Nodes 2 MDS + 52 OSS
    (Embedded) (Active/Active config)
    HDD Data 6 TB – 7.2K RPM on
    GridRAID (RAID 6)
    Cray XC40 System System Configuration
    PFS – Scratch
    Usable Storage 986 TB @ 54 GB/s
    Sonexion Config One MMU + 4 SSU
    Lustre I/O Nodes 2 MDS + 8 OSS (Active/Active
    (Embedded) config)
    HDD Data 4 TB – 10K RPM on GridRAID
    (RAID 6)
    Cray XC40 System System Configuration

    HSM / Archive Tier

    Archive Useable 30 PiB @ 12 GB/s

    Storage Array Disk NetApp E5600 series (233 TiB


    Cache for HSM @ 20 GB/s)

    Tape Library SpectraLogic TFinity

    Number of Drives LTO-7 48


    Cray XC40 System System Configuration
    Software
    Cray Linux Environment 6.x
    Cray Programming
    Environment (CPE) 10 Seats
    Intel Parallel Studio XE
    Professional Edition 2 Seats
    Workload Manager PBS Pro
    Additional Debugger –
    Allinea DDT & Total View 2048 core
    PV-View License 1
    Capacity 22KV/433 Volts
    Total Capacity 7500 KVA

    No of 2500 KVA - 3
    Transformer Nos
    Purpose For redundancy back

    Total
    5k KVA
    Capacity

    No. of DG 1750 KVA - 3 Nos.


    Purpose For redundancy back

    Total
    375O KVA
    Capacity

    No. of UPS 1250 KVA - 3 Nos.

    Battery
    15 min.
    backup
    In the unlikely event
    Purpose
    of Fire

    SYSTEM NOVEC Gas 1230


    To cool the Heat
    Purpose Generated by the
    System

    Total
    1320TR
    Capacity
    No. Of
    220 TR - 3Nos.
    Chillers
    Building Monitoring System

    To monitor the HPC


    Purpose infrastructure health
    and functioning
    Programming Programming Optimized Scientific
    Languages Compilers Tools I/O Libraries
    models Libraries

    Distributed Environment
    Memory LAPACK
    setup NetCDF
    Fortran (Cray MPT) Cray Compiling
    Environment
    • MPI (CCE) Modules
    • SHMEM ScaLAPACK

    Debuggers HDF5
    BLAS (libgoto)
    C
    GNU Allinea (DDT)
    Shared
    Memory
    lgdb Iterative
    • OpenMP 4.0 Refinement
    • Toolkit
    Debugging
    C++ 3rd Party Support Tools
    Compilers
    PGAS & Global
    View Intel Composer
    • Abnormal
    • UPC (CCE) PGI Termination
    • CAF (CCE) Processing
    Python • Chapel FFTW

    STAT
    Cray PETSc
    (with CASK)
    Performance
    R Analysis
    Cray Trilinos
    (with CASK)
    •CrayPat
    • Cray
    Apprentice2

    Cray developed Scoping Analysis


    Licensed ISV SW
    3rd party packaging Reveal
    Cray added value to 3rd party
    Service Nodes
    Compute nodes
    elogin Executing the where the jobs
    qsub aprun
    batch script are executed

    Login Servers

    Pratyush System includes service nodes, network nodes


    and many compute nodes

    You might also like