Man Linux: Main Page and Category List

NAME

       openais.conf - openais executive configuration file

SYNOPSIS

       /etc/ais/openais.conf

DESCRIPTION

       The   openais.conf   instructs  the  openais  executive  about  various
       parameters needed to control the openais executive.  The  configuration
       file   consists  of  bracketed  top  level  directives.   The  possible
       directive choices are totem  { } , logging { } , event { } , and amf  {
       }.
        These directives are described below.

       totem { }
              This  top level directive contains configuration options for the
              totem protocol.

       logging { }
              This top level  directive  contains  configuration  options  for
              logging.

       event { }
              This  top level directive contains configuration options for the
              event service.

       amf { }
              This top level directive contains configuration options for  the
              AMF service.

       Within  the totem directive, an interface directive is required.  There
       is also one configuration option which is required:

       Within the interface sub-directive of totem there are  four  parameters
       which are required:

       ringnumber
              This  specifies  the  ring number for the interface.  When using
              the redundant  ring  protocol,  each  interface  should  specify
              separate  ring  numbers  to  uniquely identify to the membership
              protocol which interface to use for which redundant ring.

       bindnetaddr
              This specifies the address which the  openais  executive  should
              bind.   This  address  should  always end in zero.  If the totem
              traffic should be routed over 192.168.5.92, set  bindnetaddr  to
              192.168.5.0.

              This  may also be an IPV6 address, in which case IPV6 networking
              will be used.  In this case, the full address must be  specified
              and  there  is  no  automatic selection of the network interface
              within a specific subnet as with IPv4.

              If IPv6 networking is used, the nodeid field must be  specified.

       mcastaddr
              This  is  the  multicast address used by openais executive.  The
              default  should  work  for  most  networks,  but   the   network
              administrator  should  be  queried  about a multicast address to
              use.  Avoid 224.x.x.x  because  this  is  a  "config"  multicast
              address.

              This  may  also be an IPV6 multicast address, in which case IPV6
              networking will be used.  If IPv6 networking is used, the nodeid
              field must be specified.

       mcastport
              This  specifies  the UDP port number.  It is possible to use the
              same multicast address on a network with  the  openais  services
              configured for different UDP ports.

       Within  the  totem  directive, there are seven configuration options of
       which one is required, five are optional, and one is required when IPV6
       is  configured  in  the interface subdirective.  The required directive
       controls the version of the totem configuration.  The  optional  option
       unless  using  IPV6 directive controls identification of the processor.
       The optional options control secrecy and authentication, the  redundant
       ring  mode  of  operation,  maximum  network MTU, and number of sending
       threads, and the nodeid field.

       version
              This specifies the version of the configuration file.  Currently
              the only valid version for this directive is 2.

       nodeid This  configuration  option  is  optional  when  using  IPv4 and
              required when using IPv6.  This is a 32 bit value specifying the
              node identifier delivered to the cluster membership service.  If
              this is not specified with IPv4, the node id will be  determined
              from  the  32  bit  IP address the system to which the system is
              bound with ring identifier of 0.  The node identifier  value  of
              zero is reserved and should not be used.

       secauth
              This  specifies  that HMAC/SHA1 authentication should be used to
              authenticate all messages.  It further specifies that  all  data
              should  be  encrypted  with the sober128 encryption algorithm to
              protect data from eavesdropping.

              Enabling this option adds a 36 byte header to every message sent
              by   totem  which  reduces  total  throughput.   Encryption  and
              authentication consume 75% of CPU cycles in aisexec as  measured
              with gprof when enabled.

              For  100mbit  networks  with  1500  MTU  frame  transmissions: A
              throughput of 9mb/sec is possible with 100% cpu utilization when
              this  option  is enabled on 3ghz cpus.  A throughput of 10mb/sec
              is possible wth 20% cpu utilization when this optin is  disabled
              on 3ghz cpus.

              For  gig-e networks with large frame transmissions: A throughput
              of 20mb/sec is possible when this  option  is  enabled  on  3ghz
              cpus.   A throughput of 60mb/sec is possible when this option is
              disabled on 3ghz cpus.

              The default is on.

       rrp_mode
              This specifies the mode of redundant ring, which  may  be  none,
              active,  or  passive.   Active replication offers slightly lower
              latency from transmit to delivery in faulty network environments
              but  with  less  performance.   Passive  replication  may nearly
              double the speed of the totem protocol if the  protocol  doesn't
              become  cpu bound.  The final option is none, in which case only
              one  network  interface  will  be  used  to  operate  the  totem
              protocol.

              If   only   one   interface  directive  is  specified,  none  is
              automatically chosen.   If  multiple  interface  directives  are
              specified, only active or passive may be chosen.

       netmtu This  specifies  the network maximum transmit unit.  To set this
              value beyond 1500, the  regular  frame  MTU,  requires  ethernet
              devices  that  support  large, or also called jumbo, frames.  If
              any device in the network  doesn't  support  large  frames,  the
              protocol  will  not  operate properly.  The hosts must also have
              their mtu size set from 1500 to whatever frame size is specified
              here.

              Please  note  while  some  NICs  or  switches  claim large frame
              support, they  support  9000  MTU  as  the  maximum  frame  size
              including  the  IP  header.  Setting the netmtu and host MTUs to
              9000 will cause totem to use the full 9000 bytes of  the  frame.
              Then  Linux will add a 18 byte header moving the full frame size
              to 9018.  As a result some hardware will  not  operate  properly
              with  this size of data.  A netmtu of 8982 seems to work for the
              few  large  frame  devices  that   have   been   tested.    Some
              manufacturers  claim  large  frame  support  when  in  fact they
              support frame sizes of 4500 bytes.

              Increasing  the  MTU  from  1500  to  8982  doubles   throughput
              performance  from 30MB/sec to 60MB/sec as measured with evsbench
              with 175000 byte messages with the secauth directive set to off.

              When  sending  multicast  traffic,  if  the  network  frequently
              reconfigures, chances  are  that  some  device  in  the  network
              doesn't support large frames.

              Choose  hardware  carefully  if  intending  to  use  large frame
              support.

              The default is 1500.

       threads
              This directive controls how many threads are used to encrypt and
              send  multicast  messages.  If secauth is off, the protocol will
              never use threaded sending.  If secauth is  on,  this  directive
              allows  systems  to  be  configured  to  use multiple threads to
              encrypt and send multicast messages.

              A thread directive of 0 indicates that no threaded  send  should
              be used.  This mode offers best performance for non-SMP systems.

              The default is 0.

       vsftype
              This directive controls the virtual synchrony filter  type  used
              to  identify  a  primary component.  The preferred choice is YKD
              dynamic linear voting, however,  for  clusters  larger  then  32
              nodes  YKD  consumes  alot  of memory.  For large scale clusters
              that are created by changing the MAX_PROCESSORS_COUNT #define in
              the  C code totem.h file, the virtual synchrony filter "none" is
              recommended but then AMF and DLCK services (which are  currently
              experimental) are not safe for use.

              The default is ykd.  The vsftype can also be set to none.

              Within  the  totem  directive,  there  are several configuration
              options which are used to control the operation of the protocol.
              It  is  generally  not recommended to change any of these values
              without proper guidance and sufficient testing.   Some  networks
              may   require   larger   values   if   suffering  from  frequent
              reconfigurations.  Some applications may require faster  failure
              detection  times  which  can  be  achieved by reducing the token
              timeout.

       token  This timeout specifies in milliseconds until  a  token  loss  is
              declared  after  not  receiving a token.  This is the time spent
              detecting a failure of a processor in the current configuration.
              Reforming  a  new  configuration  takes about 50 milliseconds in
              addition to this timeout.

              The default is 1000 milliseconds.

       token_retransmit
              This timeout specifies in milliseconds  after  how  long  before
              receiving  a  token  the  token  is retransmitted.  This will be
              automatically calculated  if  token  is  modified.   It  is  not
              recommended  to  alter  this  value  without  guidance  from the
              openais community.

              The default is 238 milliseconds.

       hold   This timeout specifies in milliseconds how long the token should
              be  held  by  the  representative when the protocol is under low
              utilization.   It is not recommended to alter this value without
              guidance from the openais community.

              The default is 180 milliseconds.

       retransmits_before_loss
              This  value  identifies  how  many  token  retransmits should be
              attempted before forming a new configuration.  If this value  is
              set,  retransmit  and hold will be automatically calculated from
              retransmits_before_loss and token.

              The default is 4 retransmissions.

       join   This timeout specifies in milliseconds how long to wait for join
              messages in the membership protocol.

              The default is 100 milliseconds.

       send_join
              This  timeout specifies in milliseconds an upper range between 0
              and send_join to  wait  before  sending  a  join  message.   For
              configurations  with  less  then 32 nodes, this parameter is not
              necessary.  For larger rings, this  parameter  is  necessary  to
              ensure the NIC is not overflowed with join messages on formation
              of a new ring.  A reasonable value for large rings  (128  nodes)
              would  be  80msec.   Other timer values must also change if this
              value is changed.  Seek advice from the openais mailing list  if
              trying to run larger configurations.

              The default is 0 milliseconds.

       consensus
              This  timeout  specifies  in  milliseconds  how long to wait for
              consensus  to  be  achieved  before  starting  a  new  round  of
              membership configuration.

              The default is 200 milliseconds.

       merge  This  timeout  specifies in milliseconds how long to wait before
              checking for a partition when  no  multicast  traffic  is  being
              sent.   If  multicast traffic is being sent, the merge detection
              happens automatically as a function of the protocol.

              The default is 200 milliseconds.

       downcheck
              This timeout specifies in milliseconds how long to  wait  before
              checking  that  a network interface is back up after it has been
              downed.

              The default is 1000 millseconds.

       fail_to_recv_const
              This constant specifies how many rotations of the token  without
              receiving  any  of the messages when messages should be received
              may occur before a new configuration is formed.

              The default is 50 failures to receive a message.

       seqno_unchanged_const
              This constant specifies how many rotations of the token  without
              any  multicast  traffic  should occur before the merge detection
              timeout is started.

              The default is 30 rotations.

       heartbeat_failures_allowed
              [HeartBeating mechanism] Configures  the  optional  HeartBeating
              mechanism  for  faster  failure  detection.  Keep  in  mind that
              engaging this mechanism in lossy  networks  could  cause  faulty
              loss  declaration  as  the  mechanism  relies on the network for
              heartbeating.

              So as a rule of thumb use this mechanism if you require improved
              failure in low to medium utilized networks.

              This  constant  specifies  the  number of heartbeat failures the
              system should tolerate before declaring heartbeat failure e.g 3.
              Also  if  this  value  is  not  set  or  is 0 then the heartbeat
              mechanism is not engaged in the system and token rotation is the
              method of failure detection

              The default is 0 (disabled).

       max_network_delay
              [HeartBeating mechanism] This constant specifies in milliseconds
              the approximate delay that your network takes to  transport  one
              packet  from  one machine to another. This value is to be set by
              system engineers and please dont change  if  not  sure  as  this
              effects the failure detection mechanism using heartbeat.

              The default is 50 milliseconds.

       window_size
              This  constant specifies the maximum number of messages that may
              be sent on  one  token  rotation.   If  all  processors  perform
              equally  well,  this  value  could  be  large (300), which would
              introduce higher latency from origination to delivery  for  very
              large  rings.   To  reduce  latency  in  large  rings(16+),  the
              defaults are a safe compromise.  If 1 or more slow  processor(s)
              are  present  among  fast  processors,  window_size should be no
              larger then 256000 / netmtu to  avoid  overflow  of  the  kernel
              receive buffers.  The user is notified of this by the display of
              a retransmit list in the notification logs.  There is no loss of
              data, but performance is reduced when these errors occur.

              The default is 50 messages.

       max_messages
              This  constant specifies the maximum number of messages that may
              be  sent  by  one  processor  on  receipt  of  the  token.   The
              max_messages  parameter is limited to 256000 / netmtu to prevent
              overflow of the kernel transmit buffers.

              The default is 17 messages.

       rrp_problem_count_timeout
              This  specifies  the  time  in  milliseconds  to   wait   before
              decrementing  the  problem  count  by 1 for a particular ring to
              ensure a  link  is  not  marked  faulty  for  transient  network
              failures.

              The default is 1000 milliseconds.

       rrp_problem_count_threshold
              This  specifies the number of times a problem is detected with a
              link before setting the link faulty.  Once a link is set faulty,
              no  more data is transmitted upon it.  Also, the problem counter
              is no longer decremented when the problem count timeout expires.

              A  problem  is  detected whenever all tokens from the proceeding
              processor    have    not    been     received     within     the
              rrp_token_expired_timeout.   The  rrp_problem_count_threshold  *
              rrp_token_expired_timeout should be atleast 50 milliseconds less
              then the token timeout, or a complete reconfiguration may occur.

              The default is 20 problem counts.

       rrp_token_expired_timeout
              This specifies the time in milliseconds to increment the problem
              counter  for  the  redundant  ring  protocol  after  not  having
              received a token from all rings for a particular processor.

              This value will  automatically  be  calculated  from  the  token
              timeout  and  problem_count_threshold but may be overridden.  It
              is not recommended to override this value without guidance  from
              the openais community.

              The default is 47 milliseconds.

       Within  the  logging  directive,  there are seven configuration options
       which are all optional:

       to_stderr

       to_file

       to_syslog
              These specify the destination of logging output. Any combination
              of these options may be specified. Valid options are yes and no.

              The default is syslog and stderr.

       logfile
              If the to_file directive is set to yes , this  option  specifies
              the pathname of the log file.

              No default.

       debug  This  specifies whether debug output is logged for all services.
              This is generally a bad idea, unless there is some specific  bug
              or problem that must be found in the executive. Set the value to
              on to debug, off to turn off debugging. If  enabled,  individual
              loggers can be disabled using a logger_subsys directive.

              The default is off.

       timestamp
              This specifies that a timestamp is placed on all log messages.

              The default is off.

       fileline
              This  specifies  that file and line should be printed instead of
              logger name.

              The default is off.

       syslog_facility
              This specifies the syslog facility type that will  be  used  for
              any messages sent to syslog. options are daemon, local0, local1,
              local2, local3, local4, local5, local6 & local7.

              The default is daemon.

       Within the logging directive, logger directives are optional.

       Within the logger_subsys  sub-directive  of  logging  there  are  three
       configuration options:

       subsys This  specifies  the subsystem identity (name) for which logging
              is specified. This is the name used by a service in the log_init
              () call. E.g. 'CKPT'. This directive is required.

       debug  This   specifies   whether  debug  output  is  logged  for  this
              particular logger.

              The default is off.

       syslog_level
              This specifies the syslog level for this  particular  subsystem.
              Ignored if debug is on.  Possible values are: alert, crit, debug
              (same as debug = on), emerg, err, info, notice, warning.

              The default is: info.

       tags   This specifies which tags should be traced for  this  particular
              logger.   Set  debug  directive to on in order to enable tracing
              using tags.  Values are specified using  a  vertical  bar  as  a
              logical OR separator:

              enter|leave|trace1|trace2|trace3|...

              The default is none.

       Within  the  event directive, there are two configuration options which
       are all optional:

       delivery_queue_size
              This directive describes the full size of the outgoing  delivery
              queue  to  the application.  If applications are slow to process
              messages, they  will  be  delivered  event  loss  messages.   By
              increasing   this   value,  the  applications  that  are  slowly
              processing messages may have an opportunity to catch up.

       delivery_queue_resume
              This directive describes when new events can be accepted by  the
              event  service when the delivery queue count of pending messages
              has reached this value.  Please note this is not cluster wide.

       Within the amf directive, there is one configuration  option  which  is
       optional:

       mode   This  can  either  contain  the value enabled or disabled.  When
              enabled, AMF  will  start  the  applications  specified  in  the
              /etc/ais/amf.conf file.  The default is disabled.

FILES

       /etc/ais/openais.conf
              The openais executive configuration file.

       /etc/ais/amf.conf
              The openais AMF configuration file.

SEE ALSO

       openais_overview(8),README.amf