Compiling and Executing Applications on BlueGene Subba r bodda and Ravi k komanduri



Yüklə 0,51 Mb.
tarix23.06.2017
ölçüsü0,51 Mb.


Compiling and Executing Applications on BlueGene

  • Subba R Bodda and Ravi K Komanduri

  • IBM India Systems and Technology Laboratory


Agenda

  • BlueGene setup

  • Compiling applications on BlueGene

  • Executing applications on BlueGene



Redbooks

  • Link: http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=blue+AND+gene

  • Blue Gene System Administration, SG24-7417-00

  • IBM System Blue Gene Solution: System Administration, SG24-7178-04

  • IBM System Blue Gene Solution: Configuring and Maintaining Your Environment, SG24-7352-00

  • Blue Gene/L: Hardware Overview and Planning, SG24-6796-02

  • IBM System Blue Gene Solution: Hardware Installation and Serviceability, SG24-6743-00

  • Blue Gene Safety Considerations, REDP-4257-00

  • Blue Gene/L: Safety Considerations, REDP-3983-01

  • Blue Gene/L: Performance Analysis Tools, SG24-7278-00

  • IBM System Blue Gene Solution: Application Development, SG24-7179-04

  • Unfolding the IBM eServer Blue Gene Solution, SG24-6686-00

  • GPFS Multicluster with the IBM System Blue Gene Solution and eHPS Clusters, REDP-4168-00

  • IBM System Blue Gene Solution Problem Determination Guide, SG24-7211-00

  • Evolution of the IBM System Blue Gene Solution, REDP-4247-00



BlueGene setup

  • bglfen01(10.16.28.30),bglfen02 is the front

  • end node

  • bglsn is the service node

  • Login to bglfen and compile your code

  • Use mpirun or LL to submit your jobs

  • bglsn has the service daemons which

  • would spawn your jobs on actual BG nodes



IBM Compilers for Blue Gene

  • IBM compilers (Fortran, C, C++) for Blue Gene are located on the frontend node in /opt/ibmcmp

  • Fortran:

  • /opt/ibmcmp/xlf/bg/10.1/bin/blrts_xlf

  • /opt/ibmcmp/xlf/bg/10.1/bin/blrts_xlf90

  • /opt/ibmcmp/xlf/bg/10.1/bin/blrts_xlf95

  • C:

  • /opt/ibmcmp/vac/bg/8.0/bin/blrts_xlc

  • C++:

  • /opt/ibmcmp/vacpp/bg/8.0/bin/blrts_xlC

  • Note : Users can add the respective directories to their PATH.



GNU Compilers for Blue Gene

  • GNU compilers (Fortran, C, C++) for Blue Gene are located in /opt/blrts-gnu/

  • Fortran:

  • /opt/blrts-gnu/bin/ powerpc-bgl-blrts-gnu-g77

  • C:

  • /opt/blrts-gnu/bin/ powerpc-bgl-blrts-gnu-gcc

  • C++:

  • /opt/blrts-gnu/bin/ powerpc-bgl-blrts-gnu-g++



MPI on Blue Gene

  • MPI implementation on Blue Gene is based on MPICH-2 from Argonne

  • National Laboratory.

  • Include files mpi.h and mpif.h are at the location:

  • -I/bgl/BlueLight/ppcfloor/bglsys/include

  • Libraries to link for MPI are:

  • -L/bgl/BlueLight/ppcfloorbglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts

  • Note : The linking order for the MPI libraries should be as mentioned above.



Compiler Optimizations



Compiler Optimizations



IBM ESSL libraries

  • The IBM ESSL (Engineering Scientific Subroutine Library) is located in

  • /opt/ibmmath/essl/4.2/



Executing jobs using mpirun

  • cd /gpfs/fcHome/scratch/userid

  • mpirun –np 32 –partition R000 -cwd /gpfs/fcHome/scratch/ -exe /gpfs/fcHome/scratch/hello_c

  • Common options:

  • -args : List of arguments to the executables in double quotes

  • -env : List of environment variables in double quotes. “VARIABLE=value”

  • -mode : CO or VN (coprocessor/virtual-node)

  • -only_test_protocol : This options is used only to test communication protocol

  • coprocessor mode : one MPI process per node, 1 GB limit per process.

  • Default mode

  • virtual-node mode : two MPI processes per node, 512 MB limit per process; L3 cache, memory, networks, are shared

  • For more details type the following at a command prompt : mpirun -h

  • Note : A user can have only one job per Blue Gene partition



Executing jobs using submit_job

  • Login to bglsn

  • cd /bgl/BlueLight/ppcfloor/bglsys/bin

  • ./mmcs_db_console

  • mmcs$ allocate M03B_J214_64

  • mmcs$ submitjob M03B_J214_64 /bgl/hello.rts /bgl/

  • OK

  • jobId=25

  • mmcs$ waitjob M03B_J214_64 25

  • OK

  • mmcs$ free M03B_J214_64

  • OK

  • mmcs$ quit



Environment Variables

  • Environment variables can be specified using –env “XYZ=123 ABC=456”

  • Some of the environment variables are

    • BGLMPI_EAGER
    • BGL_APP_L1_WRITE_THROUGH
    • BGL_APP_L1_SWOA
    • BGLMPI_COPRO_SENDS
    • BGLMPI_INTERRUPT


Redbooks

  • Link: http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=blue+AND+gene

  • Blue Gene System Administration, SG24-7417-00

  • IBM System Blue Gene Solution: System Administration, SG24-7178-04

  • IBM System Blue Gene Solution: Configuring and Maintaining Your Environment, SG24-7352-00

  • Blue Gene/L: Hardware Overview and Planning, SG24-6796-02

  • IBM System Blue Gene Solution: Hardware Installation and Serviceability, SG24-6743-00

  • Blue Gene Safety Considerations, REDP-4257-00

  • Blue Gene/L: Safety Considerations, REDP-3983-01

  • Blue Gene/L: Performance Analysis Tools, SG24-7278-00

  • IBM System Blue Gene Solution: Application Development, SG24-7179-04

  • Unfolding the IBM eServer Blue Gene Solution, SG24-6686-00

  • GPFS Multicluster with the IBM System Blue Gene Solution and eHPS Clusters, REDP-4168-00

  • IBM System Blue Gene Solution Problem Determination Guide, SG24-7211-00

  • Evolution of the IBM System Blue Gene Solution, REDP-4247-00



Summary

  • Compilation

  • IBM XLF/XLC and GNU Compilers

  • ESSL Libraries

  • submit_job

  • mpirun

  • LL

  • Environment Variables




Yüklə 0,51 Mb.

Dostları ilə paylaş:




Verilənlər bazası müəlliflik hüququ ilə müdafiə olunur ©azkurs.org 2020
rəhbərliyinə müraciət

    Ana səhifə