Man Linux: Main Page and Category List


       mpi - LAM/MPI implementation


       LAM features a full implementation of Message Passing Interface (MPI) 1
       standard with many features included from the MPI-2 standard.

       Compliant applications are source code portable  between  LAM  and  any
       other  implementation of MPI.  In addition to meeting the standard in a
       high quality manner, LAM offers extensive  monitoring  capabilities  to
       support  debugging.   Monitoring  happens  on  two levels.  LAM has the
       hooks to allow a snapshot of process and message status to be taken  at
       any time during an application run.  The status includes all aspects of
       synchronization plus  datatype  map  /  signature,  communicator  group
       membership  and message contents.  On the second level, the MPI library
       is instrumented to produce a cumulative record of communication,  which
       can be visualized either at runtime or post-mortem.

       LAM/MPI features the ability to change its underlying message transport
       mechanism, as well as pass tunable parameters to  different  components
       in  LAM/MPI at run-time -- without the need to recompile or relink user
       MPI applications.  This provides great flexibility for both  developers
       of  MPI  software as well as researchers investigating MPI performance.
       This modual framework is called the System  Services  Interface  (SSI),
       and is responsible for much of the back-end functionality in LAM/MPI.

       Another  strength  of  this  MPI implementation is the movement of non-
       blocking communication  requests,  including  those  that  result  from
       buffered  sends.   This  is  the  real  challenge  of implementing MPI;
       everything else is mostly a straight forward wrapping of an  underlying
       communication  mechanism.   LAM  allows  messages to be buffered on the
       source end in any state of progression, including partially transmitted
       packets.  This capability leads to great portability and robustness.

   User Information
       Users  are strongly encouraged to read the LAM/MPI User's Guide that is
       included with the LAM/MPI distribution, and is  provided  on  the  main
       LAM/MPI web site (

   Up-to-Date Information
       The   LAM   home   page  can  be  found  on  the  World  Wide  Web  at:  It should be consulted for the  most  current
       information about LAM, as well as updates, patches, etc.

   MPI Communication
       The  sophisticated  message  advancing  engine  at the heart of the MPI
       library uses only  a  handful  of  routines  to  drive  the  underlying
       communication  system.   Runtime  flags  decide  which  message passing
       engine module is  used,  so  recompilation  of  user  programs  is  not

       The  different  message  passing  engines  are  commonly referred to as
       "Request   Progression   Interface"   (RPI)   modules.    The   LAM/MPI
       distribution  includes  multiple RPI modules; see the lamssi_rpi(7) man
       page for more details.

       One  notable  module  uses  LAM's  network  message-passing  subsystem,
       including  its  buffer  daemon.  In this "daemon" mode, LAM's extensive
       monitoring features are fully available.  Although  the  "daemon"  mode
       typically  incurrs  higher  latency  than  the  "native"  RPI  modules,
       applications that can utilize latency-hiding techniques may  experience
       greater  performance  due  to the daemon-mode's ability to exhibit true
       asynchronous message passing.

   Guaranteed Envelope Resources
       Applications may fail, legitimately, on some  implementations  but  not
       others  due  to  an  escape  hatch in the MPI Standard called "resource
       limitations".  Most resources are managed locally and it is easy for an
       implementation  to  provide  a helpful error code and/or message when a
       resource is exhausted.  Buffer space for message envelopes is  often  a
       remote  resource (as in LAM) which is difficult to manage.  An overflow
       may not be reported (as in some other implementations) to  the  process
       that   caused  the  overflow.   Moreover,  interpretation  of  the  MPI
       guarantee  on  message  progress  may  confuse  the  debugging  of   an
       application that actually died on envelope overflow.

       LAM  has  a property called "Guaranteed Envelope Resources" (GER) which
       serves two purposes.  It is a promise from the  implementation  to  the
       application  that  a  minimum  amount  of  envelope  buffering  will be
       available to each process pair.  Secondly, it ensures that the producer
       of  messages  that  overflows  this resource will be throttled or cited
       with an error as necessary.

       A minimum GER is configured when LAM is built.  The MPI library uses  a
       protocol  to  ensure  GER when running in daemon mode.  The default C2C
       mode (TCP/IP) does not use a protocol, because process-pair  protection
       is  provided  by  TCP/IP  itself.   Errors  are  only  reported  to the
       receiving process in C2C mode.  An option to mpirun(1) disables GER.

   Input and Output
       The MPI standard does not specify standard I/O functionality.  LAM does
       not interfere with the I/O capabilities of the underlying system but it
       does  make  special  provisions  for  remote  terminal  I/O  using  the
       ANSI/POSIX routines.  See mpirun(1) and tstdio(3).

       LAM  now  includes  the  ROMIO  distribution  for  MPI-2 file input and
       output.  If ROMIO support is compiled into LAM, the functionality  from
       Chapter 9 of the MPI-2 standard is provided.

       ROMIO has some important limitations under LAM; the User's Guide in the
       LAM distribution should be consulted before writing MPI  programs  that
       use MPI I/O.

   Dynamic Processes
       LAM includes an implementation of MPI-2 dynamic process creation.

LAM Extensions to MPI

   Debugging Aids
       LAM  includes the MPI-2 functionality for naming opaque types.  Support
       for the Etnus TotalView parallel debugger is  also  provided;  see  the
       User's Guide for more details.

       Additionally, LAM provides the capability to launch non-MPI programs on
       remote nodes.  This includes shell scripts, debuggers, etc.  As long as
       an  MPI  program is eventually launched (as a child, grandchild, etc.),
       LAM can handle executing as many intermediate  programs  as  necessary.
       This can greatly help debugging and logging of user programs.

   Trace Generation
       To avoid being swamped with trace data from a long running application,
       LAM supplies collective operations to turn the tap  on  and  off.   See
       MPIL_Trace_on(2) and MPIL_Trace_off(2).

   Asynchronous Signals
       LAM has an signal handling package which mirrors but does not interfere
       with POSIX signal handling.  An MPI extension routine delivers a signal
       to a process.  See MPIL_Signal(2).


   Overview of Commands and Libraries
       introu(1), introc(2), INTROF(2)

   System Services Interface (SSI)
       lamssi(7), lamssi_boot(7), lamssi_coll(7), lamssi_rpi(7)

   Starting / Stopping LAM
       recon(1),  lamboot(1),  lamhalt(1),  lamnodes(1), lamwipe(1), tping(1),
       lamgrow(1), lamshrink(1)

   Compiling MPI Applications
       mpicc(1), mpiCC(1), mpif77(1)

   Running MPI Applications
       mpirun(1), lamclean(1)

   Running Non-MPI Applications

   Monitoring MPI Applications

   Unloading MPI Trace Data

   Reference Documents
       "LAM/MPI Installation Guide"
              included  in  the  LAM/MPI   distribution   and   available   on

       "LAM/MPI User's Guide"
              included   in   the   LAM/MPI   distribution  and  available  on

       "LAM Frequently Asked Questions"

       "MPI Primer / Developing with LAM", Ohio Supercomputer Center

       "MPI: A Message-Passing Interface Standard", Message-Passing  Interface
              Forum, version 1.1

       "MPI-2:  Extensions  to the Message Passing Interface", Message Passing
              Interface Forum, version 2.0

   MPI Quick Tutorials
       "LAM/MPI ND User Guide / Introduction"

       "MPI: It's Easy to Get Started"

       "MPI: Everyday Datatypes"

       "MPI: Everyday Collective Communication"

   Guaranteed Envelope Resources
       "Robust  MPI  Message  Delivery  Through  Guaranteed  Resources",   MPI
              Developer's Conference, 1995