Man Linux: Main Page and Category List


       corosync_overview - Corosync overview


       The  corosync  project  is  a project to implement a production quality
       "Revised BSD" licensed implementation of the  most  recent  SA  Forum’s
       Application   Interface   Specification.    The  Application  Interface
       Specification is a software API and policies which are used to  develop
       applications  that maintain service during faults.  The API consists of
       Availability Management  Framework  (AMF)  which  provides  application
       failover,  Cluster  Membership  (CLM),  Checkpointing  (CKPT), Eventing
       (EVT), Messaging (MSG), and Distributed Locking (DLOCK).

       Currently Messaging is unimplemented.

       Faults occur for various reasons:

       * Application Faults

       * Middleware Faults

       * Operating System Faults

       * Hardware Faults

       The major focus of high availability in  the  past  has  been  to  mask
       hardware  faults.  Faults  in  other components of the system have gone
       unsolved until AIS. AIS can mask many types of faults in  applications,
       middleware,  operating  systems, or even hardware by providing a simple
       framework for allowing developers  to  create  redundant  applications.
       These  redundant  applications  can  be distributed over multiple nodes
       such that if any one node faults, another node can recover.

       Application programmers develop  applications  to  periodically  record
       their state using the checkpointing service. When an active application
       fails, a standby application recovers the  state  of  the  application.
       This  technique,  called  stateful  application  failover, provides the
       fundamental difference between corosync and  other  systems  that  have
       come   before   it.   With  stateful  application  failover,  the  end-
       application user doesn’t have to reload the  application  or  redial  a
       telephone. The full state is recorded, so the end-application user sees
       no interruption in service.

       Because programmers can now  distribute  applications  across  multiple
       processes  or  nodes,  a  mechanism must exist for them to communicate.
       This mechanism is provided by two services. The event service  provides
       a  publish/subscribe  model  for events. The messaging service provides
       end to end messaging. Finally a  mechanism  to  synchronize  access  is
       provided by the distributed lock service.

       The  corosync  project  also  provides a group messaging toolkit called
       EVS.  The EVS service implements a messaging model  known  as  Extended
       Virtual  Synchrony.   This  model allows one sender to transmit to many
       receivers.  Certain guarantees are provided for message and  membership
       delivery  which make virtual synchrony ideal for developing distributed


       The corosync executive must be configured.  In the  directory  conf  in
       the  source  distribution  are several files that must be copied to the
       /etc/corosync directory.  If corosync is packaged by a distro, this may
       be complete.

       The  directory  contains  the  file  corosync.conf.   Please  read  the
       corosync.conf(5) man page for details  on  the  configuration  options.
       The  corosync  project  will  work  out  of  the  box  with the default
       configuration options, although the administrator may desire  different

       The   corosync   executive  uses  cryptographic  techniques  to  ensure
       authenticity and privacy of the messages.  In order for corosync to  be
       secure  and  operate, a private key must be generated and shared to all

       First generate the key on one of the nodes:

       unix# ais-keygen corosync authentication key generator.

       Gathering 1024 bits for key from /dev/random.

       Writing corosync key to /etc/ais/authkey.

       After  this  operation,  a  private   key   will   be   in   the   file
       /etc/ais/authkey.   This  private key must be copied to every processor
       in the cluster.  If the private key isn’t  the  same  for  every  node,
       those  nodes with nonmatching private keys will not be able to join the
       same configuration.

       Copy the key to some security  transportable  storage  or  use  ssh  to
       transmit  the  key  from  node  to node.  Then install the key with the

       unix#:     install     -D     --group=0      --owner=0      --mode=0400
       /path_to_authkey/authkey /etc/ais/authkey

       If  a message "Invalid digest" appears from the corosync executive, the
       keys are not consistent between processors.

       Finally run the corosync executive.  If corosync  is  packaged  from  a
       distro,  it may be set to start on system start.  It may also be turned
       off by default in which case the  init  script  for  corosync  must  be

       After  running  aisexec,  a list of all processors IP addresses running
       the ais executive and configured on the  same  multicast  address  will
       appear.  If they don’t appear, there may be a problem with multicast in
       the distro or hardware.  If this happens, participation in the corosync
       mailing  list  may  help  solve  the  problem.   The  email  address is


       The corosync AIS libraries have header files which must be included  in
       the  developer’s  application.   Once  the header file is included, the
       developer can reference the AIS interfaces.

       The corosync project recommends to distros to place  include  files  in
       /usr/include/corosync.   The  following  include lines must be added to
       the application to use each of the following services:

       #include <corosync/saClm.h> For the Cluster Membership B.01.01 service.

       #include <corosync/saCkpt.h> For the Checkpointing B.01.01 service.

       #include <corosync/saEvt.h> For the Eventing B.01.01 service.

       #include <corosync/ais_amf.h> For the AMF A.01.01 service.

       The  corosync  project  recommends to distros to place library files in
       /usr/lib.  The following link  lines  must  be  added  to  the  LDFLAGS
       section of the makefile.

       -lsaClm For the Cluster Membership B.01.01 service

       -lsaCkpt For the Checkpointing B.01.01 service

       -lsaEvt For the Eventing B.01.01 service

       -lsaAmf For the AMF A.01.01 service

       -lais  Specify  this  to  get  access  to  all  AIS  libraries  without
       specifying each library individually.


       The corosync project supports both IPv4  and  IPv6  network  addresses.
       The  entire  cluster  must  use  either  IPv4  or  IPv6 for the cluster
       communication mechanism.  In order to use IPv6, IPv6 addresses must  be
       specified  in the bindnetaddr and mcastaddr fields in the configuration
       file.  The nodeid field must also be set.

       An example of this is: nodeid: 2 bindnetaddr:  fec0::1:a800:4ff:fe00:20
       mcastaddr: ff05::1

       To  configure  a  host  for  IPv6,  use  the  ifconfig  program  to add
       interfaces: box20: ifconfig eth0 add fec0::1:a800:4ff:fe00:20/64 box30:
       ifconfig eth0 add fec0::1:a800:4ff:fe00:30/64

       If  the  /64 is not specified, a route for the IPv6 network will not be
       configured which will cause significant problems.  Make sure a route is
       available for IPv6 traffic.


       The  AIS  libraries are a thin IPC interface to the corosync executive.
       The corosync executive provides services for the SA Forum AIS libraries
       as well as the EVS and CPG libraries.

       The  corosync  executive  uses  the  Totem  extended  virtual synchrony
       protocol.  The advantage to  the  end  user  is  excellent  performance
       characteristics and a proven protocol with excellent reliability.  This
       protocol connects the processors in a configuration  together  so  they
       may communicate.


       The  corosync  executive process uses four environment variables during
       startup.  If these environment variables are not set, defaults will  be

              This   specifies  the  fully  qualified  path  to  the  corosync
              configuration file.

              The default is /etc/ais/corosync.conf.

              This  specifies  the  fully  qualified  path  to  the   corosync
              Availability Management Framework configuration file.

              The default is /etc/ais/amf.conf.

              This specifies the LCRSO that is used to parse the configuration
              file.  This  allows  other  configuration  file  parsers  to  be
              implemented within the system.

              The  default  is  to use the default corosync configuration file
              parser which parses the format specified in corosync.conf (5).

              This specifies the fully qualified path to the shared  key  used
              to authenticate and encrypt data used within the Totem protocol.

              The default is /etc/ais/authkey.


       The corosync executive optionally encrypts all messages sent  over  the
       network using the SOBER-128 stream cipher.  The corosync executive uses
       HMAC and SHA1 to authenticate all  messages.   The  corosync  executive
       library  uses  SOBER-128  as a pseudo random number generator.  The EVS
       library feeds the PRNG using the /dev/random Linux device.

       If membership messages can be captured by intruders, it is possible  to
       execute  a  denial of service attack on the cluster.  In this scenario,
       the cluster is likely already compromised and a DOS attack is the least
       of the administration’s worries.

       The security in corosync does not offer perfect forward secrecy because
       the keys are reused.  It may be possible for an intruder  by  capturing
       packets  in  an automated fashion to determine the shared key.  No such
       automated attack has been published as of yet.  In this  scenario,  the
       cluster is likely already compromised to allow the long-term capture of
       transmitted data.

       For security reasons, the  corosync  executive  binary  aisexec  should
       NEVER be setuid or setgid in the filesystem.


       The  corosync  libraries  are now nearly compliant with every aspect of
       the SA Forum’s AIS specification.  The AMF  service,  however,  is  not
       compliant  with the B.01.01 specification.  The remaining services pass
       most  of  the  tests  of  the  saftest  suite   against   the   B.01.01


       The  messaging  service  is  partially implemented and not suitable for
       deployment.  The distributed locking service is buggy and not  suitable
       for   deployment.   The  Availability  Management  Framework  is  under
       development and not suitable for deployment..


       corosync.conf(5), evs_overview(8)