D Using the Event Manager

The Event Manager is a comprehensive event management system providing traditional event handling facilities. The Event Manager includes an event viewer and a full set of command line tools. It is integrated into the AlphaServer Management Station application.

The following topics are covered in this chapter:

An overview of Event Manager (Section D.1)

How to set up and customize Event Manager (Section D.2)

How to use Event Manager to assist in the administration of your system (Section D.3)

Troubleshooting common Event Manager problems (Section D.4)

D.1 Event Manager Overview

A critical function of the AlphaServer Management Station is to monitor the state of the platforms and systems being managed, and to inform the administrator when certain unusual conditions occur. Examples include errors like those reported by the platform firmware (fan failure) and by the operating system console (disk full). Such conditions are known as system events.

D.1.1 Features of the Event Manager

Event Manager provides the following features:

Facilities for users and applications to post and monitor events

Integration of an event viewer with the SPM and PCM user interfaces

A choice of summary or detailed event data

A full set of command-line utilities that you can use to monitor and manage events from shell scripts and from the command line

Configurable event logger that allows full control over which events are logged and optimizes storage space used by identical events

Configurable event forwarding that enables you to automatically notify other system entities of selected events

Log file management that automatically archives and purges log files daily

Centralized access to event information

D.1.2 Understanding Event Manager Events

An Event Manager event is a binary package of data that contains a set of standard data items, including a name, a timestamp, and information about the poster. An event may contain variable data, which is named and supplied by the poster. For example, an event reporting the failure of a device may hold variables containing the path name and type of the device.

Events are created and posted by an Event Manager posting client, and distributed to other clients by the Event Manager daemon. Then, a receiving process can extract and process the information contained in the event.

Figure D-1 shows a graphical representation of an event. The Event Contents box shows items, such as the process identifier (PID) and the name of the host system on which the event was generated, that may be included in the event. The Event Actions box shows some of the possible actions performed on any event.

Figure D-1: Event Model

The Event Manager includes command-line utilities that understand the format of the event, and which you use to perform basic operations at the command prompt or in shell scripts; you cannot view an event directly with a text viewer (for example, more ) because an event is a package of binary data. You can use Event Manager commands to:

Retrieve events from storage, sort them into a preferred order, and format them for display

Watch for new events being posted

The Event Manager command-line utilities are designed to be used together in pipelines. For example, you may pipe a set of events from a file into the sort utility, pipe the output into the formatting utility, then pipe the output of that command into the more command, or redirect it to a file. Section D.3 provides examples of using Event Manager commands to monitor and review event activity.

After the event file is converted to text form, you can use other standard utilities to analyze it. For example, you may display just the event names, and then pipe the display into the sort -u and wc -l commands to determine how many different types of events are in the file.

D.1.3 Event Manager Command-Line Utilities

Event Manager provides a number of command-line utilities both for administering the Event Manager system itself and for obtaining events. Table D-1 describes the general user commands. Detailed information is available from the reference pages. See Section D.3 for examples of how to use these commands to monitor and review event activity.

Table D-1: Event Manager Command-Line Utilities

Command	Description
`evmget`	Retrieves stored events from a configured set of log files and event channels, using channel-specific retrieval functions
`evmshow`	Accepts one or more Event Manager events and outputs them in the requested format
`evmsort`	Reads a stream of events and sorts them according to supplied criteria
`evmwatch`	Subscribes to events specified and outputs them as they arrive

Table D-2 lists the Event Manager administrative commands, which are usually invoked during system initialization. The individual command reference pages discuss other conditions under which the command is used.

Table D-2: Event Manager Administrative Utilities

Command	Description
`evmchmgr`	The Event Manager daemon automatically starts the Event Manager channel manager. It executes the periodic functions defined for any channel.
`evmd`	The Event Manager daemon receives events from posting clients and distributes them to subscribing clients, that is, clients that have indicated they want to receive the events. The daemon is a critical system facility that starts automatically at system boot. Do not terminate it.
`evmlogger`	The Event Manager daemon automatically starts the Event Manager logger. The logger receives events from the daemon and writes them to each of the logs whose filter string they match. The `evmlogger` also serves as an event forwarding agent that you can configure to take an action when required.
`evmreload`	This command posts control events, which instruct the Event Manager components to reload their configuration files. When you modify an Event Manager configuration file you must use this command to load the new configuration.
`evmstart`	This command starts the Event Manager daemon. It is intended for use by the system startup scripts, but you can also use it to restart Event Manager should it terminate for any reason.
`evmstop`	This command stops the Event Manager daemon, preventing entities from posting or subscribing for events. It is intended for use by the system shutdown scripts. Do not use this command under normal circumstances, because Event Manager is required for many system functions to operate correctly.

D.1.4 Event Manager System Files

Event Manager creates or uses the system files described in the following sections.

Executable Files

Executable files for Event Manager administrative commands are located in the /usr/sbin directory.

General (that is, user) command executable files are located in the /usr/bin directory.

Initialization files are located as follows:

In /sbin/init.d for Tru64 UNIX.

In /etc/init.d for Linux

Configuration Files

Base Event Manager configuration files are located in the /etc directory; they are listed here.

/etc/evmdaemon.conf: This file is a text file that contains commands used to configure and start the Event Manager.
/etc/evmchannel.conf: The event channel configuration file, which is read by the channel manager, evmchmgr, and the evmshow command. This file describes all the channels through which events can be posted and retrieved.
/etc/evmlogger.conf: The configuration file for the logger, evmlogger. It contains commands used to direct the display, forwarding, or storage of events. See Section D.2.2 and evmlogger.conf(4) for a complete description of this file.
/etc/evm.auth: This file is used to control access to events and event services. See Section D.2.3.2 and evm.auth(4) for a complete description of this file.

Log Files, Working Files, and Local Installation Files

Log files, working files, and local installation files are located in the following subdirectories of /var/evm.

/var/evm/sockets

This directory contains a domain socket node, evmd, and a related lock file, evmd.lck. Local clients use this socket for connection.

/var/evm/evmlog

This directory contains the event logs created by the default Event Manager logger configuration. Log files in this directory have names in the format evmlog.yyyymmdd[_nn], where yyyymmdd is the date of the log, and _nn is a sequential generation number. A new log generation starts if the log reaches its configured maximum size during the course of the day, or if the logger finds an error in the current file. The day's first log file has no generation number. A new log file is started automatically when it receives the first event after midnight, system time.

This directory also contains a lock file, evmlog.dated.lck, and a generation control file, evmlog.dated.gen, the latter containing information about the current generation number. See Section D.2.4 for more information on managing log files.

/var/evm/adm/logfiles

This directory contains output message logs created by the resident components of Event Manager: the daemon, logger, and channel manager. New files are created each time Event Manager starts. Old files are renamed by appending the suffix ".old" to their names, overwriting any previous old files. These message logs are encapsulated by Event Manager's misclog event channel, so their contents are visible through evmget and the event viewer.

/var/evm/shared

This directory is a work directory that holds temporary files required for client authentication.

/var/evm/adm/templates

The directory is provided for installation of local and third-party event template subdirectories. This directory is connected to the system template directory by a symbolic link.

/var/evm/config

This directory and its subdirectories contain secondary configuration files for various Event Manager components. In this release, only the logger supports secondary configuration files; see evmlogger.conf(4) for more information.

/var/evm/adm/filters

The directory is provided for installation of local event filter files.

/var/run/evmd.pid

This file contains the daemon process identifier (PID), that is saved by the evmd daemon for future actions, such as stopping Event Manager.

/var/run/evmlogger.info

This file contains the logger's PID and information about the log files being managed. The evmlog channel retrieval and daily cleanup functions use this information.

System-Supplied Definition Files

System-supplied definition files for templates, channels, and filters are located in the following subdirectories of the /usr/share/evm directory.

Do not modify these files.

/usr/share/evm/channels: This directory contains a subdirectory for event channels such as evmlog. Each subdirectory contains scripts that define the services available for that channel.
/usr/share/evm/filters: This directory contains system filter files.
/usr/share/evm/templates: This directory contains system event template files and subdirectories.

D.2 Administering Event Manager

The role of the administrator in running Event Manager involves the following principal activities:

Starting and stopping Event Manager, described in Section D.2.1

Configuring the evmlogger files, described in Section D.2.2

Controlling who is allowed to post or access events, described in Section D.2.3

Managing log files, described in Section D.2.4

For information on using the Event Manager, see Section D.3.

D.2.1 Starting and Stopping Event Manager

The Event Manager is started automatically at system startup and is stopped when the system is shut down.

Use the evmstop command to stop Event Manager:

# /usr/sbin/evmstop

Use the evmstart command to start Event Manager:

# /usr/sbin/evmstart

You do not need to stop and start Event Manager when you want to change the Event Manager configuration. In this instance, change the configuration, then issue the evmreload command. See evmreload(8) for more information.

D.2.2 Configuring the Event Manager Logger

The Event Manager logger handles storage and forwarding of events, according to entries in the /etc/evmlogger.conf configuration file. For a complete description of the contents and syntax of this file, see evmlogger.conf(4).

Example D-1 shows sample entries in a logger configuration file — an example of possible customization of the logger is to have events e-mailed to specific users.

Note

The syslog templates referenced in the following examples are not included in the AMS software kit and are used only for example purposes.

Note that

Example D-1: Sample Event Manager Logger Configuration File Entries

# Main log file:
eventlog {    [1]
    name        evmlog   [2]
    logfile     /var/evm/evmlog/evmlog.dated   [3]
    type        binary   [4]
    maxsize     512  # Kbytes    [5]
 
    # Uncomment the following "alternate" line and set the 
    # logfile path to specify an alternate logfile in case
    # of write failures.
    # The path must specify an existing directory.
    #alternate  /your_alternate_fs/evmlog/evmlog.dated   [6]
 
    # Log all events with priority >= 200, except syslog events:
    filter       "[prio >= 200] & (! [name @SYS_VP@.syslog])"  [7]
 
    # Suppress logging of duplicate events:
    suppress    [8] 
    {   filter      "[name *]"
        period      30    # minutes 
        threshold   3     # No. of duplicates before suppression
    } 
}
# Forward details of high-priority events to root:
forward {    [9]
    name      priority_alert    [10] 
    maxqueue  200   [11]
 
    # Don't forward mail events through mail
    filter  "[prio >= 600] & ![name @SYS_VP@.syslog.mail]"   [12]
 
    suppress    [13] 
    {   filter  "[name *]"
        period  120    # minutes
        threshold   1  # No. of duplicates before suppression
    }
 
    # This evmshow command writes a subject line as the first
    # line of output, followed by a detailed display of the 
    # contents of the event. 
    # The resulting message is distributed by mail(1).
    command "evmshow -d -t 'Subject: EVM ALERT [@priority]: @@' |
    mail root"    [14]
 
    # Limit the number of events that can be queued for this
    # command:
    maxqueue        100
}
# Secondary configuration files can be placed in the following
# directory.  See the evmlogger.conf(5) reference page for
# information about secondary configuration files.
configdir       /var/evm/adm/config/logger

This line begins an event log configuration group. [Return to example]

This line provides a name for the event log. Other portions of the configuration file may reference this name. [Return to example]

This line specifies that the log files are stored in the /var/evm/evmlog directory. Each day, when the log for that day is first written, the dated suffix is replaced by the date in the format yyyymmdd. [Return to example]

This line specifies that the type of events written to this log are binary Event Manager events, rather than formatted (ASCII text) events. [Return to example]

This line specifies the maximum size of the log file in kilobytes (KB). In this case, if the size of the current log file exceeds 512 KB the logger closes it and begins a new log file, with a sequentially numbered suffix (for example, _2) appended to the file name. [Return to example]

If this line is not commented out (by #) and the sample path is replaced by the path name of an existing write-enabled directory, an alternate log file is opened in this directory if the primary directory becomes write-disabled. [Return to example]

This line establishes the filtering conditions for events, determining which events are logged by this event log. See EvmFilter(5) for details of Event Manager filter syntax. The @SYS_VP@ entry is a macro that is replaced with sys.unix when the file is read. [Return to example]

These statements define the suppression parameters for this event log. In this case, suppression of a particular event begins if three or more duplicate events are received within 30 minutes. Suppression of duplicate events saves space in the log file. See evmlogger.conf(4) for a detailed description of event suppression. [Return to example]

This line establishes conditions for forwarding events to the root user. An event forwarder executes a specified command string when selected events occur. It is useful for notifying the system administrator when a significant error occurs. [Return to example]

In this line, name identifies the forwarder. [Return to example]

The maxqueue queue_limit keyword limits the number of events that a forwarder can queue while a previous event is being handled. If the maximum number of events is already queued when a new event arrives, the new event is ignored by this forwarder. If not specified, this keyword has a default value of 100 events. If you specify a value greater than 1000 events, the logger automatically limits it to 1000 events. [Return to example]

This line establishes filtering for the events. As with an event log definition, the filter string specifies the set of events that are handled by this forwarder. To prevent an event loop from occurring if the mailer posts high-priority events, signifying a possible problem in the mail subsystem, mail events are explicitly excluded from this forwarder. [Return to example]

These lines suppress multiple forwarding of events. The suppression mechanism for a forwarder is similar to that for an event log. Here, the purpose is to prevent the command from being sent multiple times in a short period because of the same event being posted repeatedly. In the example, a particular event is forwarded once every two hours at most. [Return to example]

This line defines the command that executes when an event is handled by the forwarder. The event is piped into the command's stdin stream. The result of this command is shown in the comments preceding the command line. [Return to example]

If you make any changes to the logger configuration file you must run the evmreload command to make the changes known to the logger; see evmreload(8) for more information.

Secondary Logger Configuration Files

Secondary logger configuration files enable you to add event logs or forwarders without modifying the primary configuration file, /etc/evmlogger.conf. This feature ensures that any problems with secondary files do not affect the primary configuration. It enables you to safely experiment with different logger configurations.

Should the logger encounter a syntax error in a secondary configuration file, it displays an error message and rejects the file. The primary configuration file and any additional (and correct) secondary files are processed and Event Manager functions correctly.

The secondary configuration directory feature also allows individual system components, products and applications to install or change log files and forwarders by installing or replacing files, rather than having to insert or maintain lines in the primary configuration file. You can uninstall entries by removing the file.

The default and recommended location of secondary configuration files is the /var/evm/adm/config/logger directory, or a subdirectory of that directory. Your secondary configuration files must have file name suffix .conf and the file syntax must follow the rules described in Example D-1.

It is important that you give appropriate permissions to the secondary logger configuration files and directories. The logger runs with superuser privileges and can execute commands specified in any secondary configuration file. For this reason, the logger rejects any configuration files that do not have the correct permissions and posts a warning event. See evmlogger.conf(4) for the correct file permissions.

D.2.3 Security Considerations

Security is an important consideration when dealing with events, for the following reasons:

Uncontrolled access to certain event information may provide an unauthorized user with sensitive information about system operation.

Posting certain events may cause critical system actions, for example, application failover or system shut down, to occur.

Traditionally, event information security is maintained by restricting read access to log files and limiting certain operations to the root user. Because the Event Manager daemon and event retrieval facilities provide alternate means of access to all events, both as they are posted and after they are logged, the daemons also provide a way to limit access, so that events are seen only by authorized users. You can enable access control by providing authorization facilities and using authentication techniques.

D.2.3.1 User Authentication

The Event Manager daemon authenticates the identities of all local system users before accepting any connection request.

D.2.3.2 User Authorization

Access to events is controlled by the Event Manager authorization file, /etc/evm.auth.

When AMS is installed, members of the amsuser group are automatically provided EVM access rights. It is critical to the proper operation of AMS that these privileges remain intact. In addition, the root user can authorize additional individual users or groups of users to do the following:

Access (subscribe to or retrieve from storage) selected events

Execute selected services

Event rights are granted by supplying, for each event class, a list of users who have the specified right or who are explicitly denied rights.

A plus sign (+) that is not followed by a user list implicitly grants the right to all users. A minus sign (-) that is not followed by a user list implicitly denies the right to all users.

The root user has implicit posting and access rights to all events unless explicitly denied them. Example D-2 shows sample entries in an authorization file. See evm.auth(4) for more information.

Example D-2: Sample Event Manager Authorization File Entries

# ===================
#      EVENTS
# ===================
 
event_rights {    [1]
   class        @SYS_VP@.evm.control   # EVM control events
   post         root
   access       +
}
 
event_rights {    [2]
   class        @SYS_VP@.evm.msg.admin  # EVM admin message
   post         root
   access       "root, group=adm"
}
 
event_rights {    [3]
   class        @SYS_VP@.evm.msg.user   # EVM user message
   post         +
   access       +
}
 
# ===================
#      SERVICES
# ===================
 
service_rights {    [4]
   service      event_get
   execute      +
}

Only the root user can post the class of events that have names beginning with sys.unix.evm.control. Such events are accessible by all users. The @SYS_VP@ entry is a macro that is replaced with sys.unix when the file is read. [Return to example]

Only the root user can post the class of events that have names beginning with sys.unix.evm.msg.admin. Such events can be accessed by root or other users in the admin group. [Return to example]

All users can post or access the class of events that have names beginning with sys.unix.evm.msg.user. [Return to example]

All users can execute the event_get service. [Return to example]

If you make any changes to the authorization file you must run the evmreload command to make the Event Manager daemon aware of the changes.

D.2.4 Managing Log Files

The Event Manager channel manager, evmchmgr, provides log management capability through the channel fn_cleanup function. You can define this capability for any channel through the channel configuration file, evmchannel.conf.

By default, channel cleanup functions run when Event Manager starts. You can change the time of day by editing the cleanup_time value in the channel configuration file. When a cleanup is scheduled, the channel manager scans the event channel list, and executes the fn_cleanup command for each channel identified in the file.

The evmlog cleanup function, evmlog_cleanup, takes two arguments:

The archive period, which has a default value of 7 days.

The delete period, which has a default value of 31 days.

The function uses the find utility to locate and compress (zip) all logs older than the archive period, and to delete any archived files older than the delete period. You can change the period values by editing the function definition in the channel configuration file. Setting either of these values to zero disables the corresponding function.

The evmget command does not retrieve evmlog events that are stored in archived (zipped) logs. To retrieve events from archived logs you must first uncompress them with the gunzip command; see gunzip(1) for information on unzipping archive files.

D.3 Using Event Manager in System Administration

The following sections illustrate the commands you can use to monitor and review event activity. As you become familiar with the Event Manager command set, you build up a set of favorite commands, shell scripts, and filters that help you to keep track of what is happening on your system.

D.3.1 Displaying Events Using evmshow

Because an Event Manager event is a binary data package, it must be converted to text before you can display it on a terminal. The evmshow command reads binary Event Manager events from its stdin stream or from a named file, and outputs the same events in text form to stdout. For example, you can display the contents of a file containing Event Manager events by using the following command:

# cat my_events | evmshow | more

This command displays the events from the log file in the default manner, that is, it takes the format data item from each event, expands it with the values of any variables it references, and displays it. References to variables are identified by a dollar sign ($). Therefore, if the my_events file contains an event with a format data item of ext3: ext3 domain $domain is full, and the event also contains a variable named domain with a value of root_domain, the corresponding line of the output is:

ext3: ext3 domain root_domain is full

This information tells you what happened, but not when it happened, or the importance of the event. You can modify the output of the evmshow command to include any data items in the event, including its timestamp and priority, by using the -t option to specify a show-template. A show-template is a text string that indicates which data items you want to be displayed for an event, and how you want them to be displayed.

The following example illustrates the use of a show-template to display an event with a timestamp, a priority, and the formatted event message. In the show-template, the names of the items to be displayed are each preceded by an at sign (@) . Two at signs (@@) indicate that the event's format item should be expanded and displayed. The second line shows the output for the domain full event. In the output, the event priority is surrounded by brackets, and there are two spaces before the message text, exactly as specified in the show-template:

# cat my_events | evmshow -t "@timestamp [@priority]  @@" | more
22-Jun-2000 11:22:27 [600] ext3: ext3 domain root_domain is full

You can set up your own show-template to display the items that are important to you, in any format you want. See EvmEvent(5) for a list of all the data items. After you determine your preferred style you can set a default show-template in the environment variable EVM_SHOW_TEMPLATE and use fewer keystrokes at the command line. The following Korn shell (ksh) commands are equivalent to those in the previous example:

# export EVM_SHOW_TEMPLATE="@timestamp [@priority]  @@"
# cat my_events | evmshow | more

If you want more information about an event, you can request a detailed display, including an explanation and a full dump of its contents, by using the evmshow command with the -d option. The following example shows a detailed display of the domain full event:

# cat my_events | evmshow -d | more
 ============================ EVM Log event ======================= 
EVM event name: sys.unix.fs.ext3.fdmn.full
 
     This event is posted by the ext3 filesystem to provide
     notification that the specified ext3 domain is full.
     No more     space is available for writing.  [1] =
=================================================================  
Formatted Message:     
     ext3: ext3 domain root_domain is full       [2] 
 
Event Data Items:      [3]
     Event Name        : sys.unix.fs.ext3.fdmn.full
     Cluster Event     : True
     Priority          : 600
     PID               : 1177
     PPID              : 724
     Timestamp         : 22-Jun-2000 11:22:27
     Host IP address   : 0.0.0.0
     Host Name         : x.x.example.com
     User Name         : root
     Format            : ext3: ext3 domain $domain is full      [4]
     Reference         : cat:evmexp.cat:450  
 
Variable Items:       [5]   
     domain (STRING) = "root_domain"  
======================================================================

The explanation of the event. In some cases, this data field contains a recommended action to rectify a problem. [Return to example]

The Formatted Message section. [Return to example]

The Event Data Items section, which lists all the standard data items contained in the event. See EvmEvent(5) for a description of each of these items.
The items shown here are typical of many events, but sometimes some of these are missing, and occasionally you may see additional items. [Return to example]

The Format data item is almost the same as the content of the Formatted Message data item, but it includes a reference to a variable called domain, indicated by the $ symbol preceding it. [Return to example]

The Variable Items section, which contains the value of the domain variable. [Return to example]

See Section D.3.9.2 for information on how to select events for detailed display.

You can use the evmshow -x command to display the explanation alone. Alternatively, use the -x and -t options together to provide a summary of the event followed immediately by its explanation. For example:


 #cat my_events | evmshow -x -t "@timestamp [@priority]  @@" | more \
21-Jun-2002 11:22:27 [600] ext3: ext3 domain root_domain is
full
 This event is posted by the ext3 filesystem to provide
 notification that the specified ext3 domain is full.
 No more space is available for writing.

The examples in this section show how to display Event Manager events that are contained in a single log file. You can display events that are stored in the various system log files, or monitor them as they occur by using the evmget and evmwatch commands, which are introduced in Section D.3.3 and Section D.3.6.

Some systems produce a large number of events, many of which report normal operation. Use event filters to limit the display to a set of events that you consider interesting. Section D.3.2 introduces the Event Manager filtering facilities.

Regardless where the events come from, you use the evmshow command to format them for display. See evmshow(1) for more details of the show-template.

D.3.2 Introducing Event Filters

This section introduces event filters and relates them to the evmshow command examples from the previous section. Filtering is used more extensively in later sections, which describe event retrieval and monitoring techniques. The full filter syntax is defined in EvmFilter(5).

An Event Manager event filter is a text string that tells Event Manager which events you want to retrieve. For example, the filter string [priority >= 600] selects events that have a priority of 600 or higher. A filter can be very simple, but the filter language is powerful, and with some practice you can easily build and store a filter expression that defines precisely the set of events that you want to monitor. Filters are used by several of the Event Manager command line utilities, by the Event Manager logger, and by system daemons and client applications.

The evmshow, evmget and evmwatch commands support the -f option which you use to specify a filter string. You can select the events to be displayed from the my_events file, as shown in the following example:

# export EVM_SHOW_TEMPLATE="@timestamp [@priority]  @@"
# cat my_events | evmshow -f "[priority >= 600]" | more

(The preceding example was introduced in Section D.3.1.) In this example, the -f option specifies the filter, and selects events that have a priority of 600 or higher. The command reads all events from the file, but returns only those events that match the filter string.

If you know the names of the events you want to retrieve, you can specify them in a filter, as shown in the following example:

# cat my_events | evmshow -f "[name sys.unix.fs.ext3.fdmn.full]" | more

You can use wildcard characters in place of name components as follows:

An asterisk (*) character matches zero or more complete components

A question mark (?) matches exactly one complete component

For example, use the following command to shorten the preceding example command:

# cat my_events | evmshow -f '[name *.ext3.fdmn.full]' | more

The wildcard asterisk matches the components sys.unix.fs. To avoid any possibility that the shell expand the wildcard character with file names, enclose the filter string in single quotes instead of the double quotes. This is always a wise precaution when special characters are used in shell commands.

When you filter by name, Event Manager assumes that there is a wildcard .* at the end of the name string, even if it is not included in the command. Therefore, you may receive events with more name components than you specify. The following two commands are equivalent to each other, but the final wildcard (.*) in the first command is unnecessary:

# cat my_events | evmshow -f '[name *.ext3.*]'
# cat my_events | evmshow -f '[name *.ext3]'

You can find the names of events by specifying @name as one of the items in your show-template when you run the evmshow command.

Use the filter syntax to combine multiple conditions into a single filter with the AND, OR and NOT keywords, and you can use parentheses to group conditions. The following example command selects all events whose names include the component ext3, and that have a priority of 600 or higher:

# cat my_events | evmshow -f '[name *.ext3] and [priority >= 600]'

The following command also selects events with the name component syslog, regardless of their priority. In the following example, the keyword priority is abbreviated to pri, and name is abbreviated to na. Most filter keywords can be abbreviated as described in EvmFilter(5).

# cat my_events | evmshow -f '([na *.ext3] and [pri >= 600]) or [na *.syslog]'

The examples in this section illustrate the most commonly used filter keywords. When you are familiar with applying filters to the evmshow command and the Event Manager commands described in the following sections, you can use the more advanced filter features to create and save useful filters, and to increase your ability to select the events that are most interesting. Advanced filter techniques are described in Section D.3.9, and the full syntax is given in EvmFilter(5).

D.3.3 Retrieving Stored Events Using evmget

System log files store events in many different formats and with different levels of detail, making it difficult to produce an ordered view of all events by using traditional system utilities.

You can use the evmget command to produce an ordered view by retrieving events from each of the various log files, converting them to Event Manager events if they are not already in that form, and returning a single stream of Event Manager events. Using the evmshow command, you can then turn the Event Manager event stream into a display format.

The following command pipeline uses the evmget command to retrieve all system events, and passes them to the evmshow command for display:

# evmget | evmshow -t "@timestamp [@priority]  @@" | more

The evmget command makes a service connection to the Event Manager daemon, which starts a new copy of the get-server program, /usr/sbin/evm_getsrv. The get-server program reads the channel configuration file, and runs the get function, usually a shell script, for each channel configured in the channel configuration file, /etc/evmchannel.conf.

The get function does the following:

Reads the channel's log file

Converts the events into EVM format

Feeds events back to the evmget command which writes them to its stdout stream

After all the channel get functions run and all the events are returned, the get-server daemon and the evmget command both terminate.

Note

Even though events may be stored in log files as lines of text, or in a special binary format, the evmget command returns all events in the form of binary Event Manager events, which can be passed to evmshow for display. If you send the output of evmget directly to your terminal, the command displays an error message because the binary output cannot be displayed properly and could affect the settings of your terminal. If you pipe the output into another command, such as more or less, the evmget command is unable to detect the error, and random characters are displayed.

Like the evmshow command, the evmget command supports a filter option to allow you to limit the events it returns. For example, the following command displays only high-priority events:

# evmget -f '[pri >= 600]' | evmshow | more

It is more efficient to specify a filter with the evmget command than with the evmshow command. This is because the evmget command passes its filter string to the event channel's get function, which only returns events that match the filter. Fewer events are passed back through the get-server daemon to the evmget command, and the commands operate faster because they transfer and process fewer events.

If you want to save retrieved events for later analysis, or to copy them to another system, you can redirect the output of the evmget command into a file. For example:

# evmget -f '[pri >= 600]' > my_events

Saving the binary output of the evmget command provides greater flexibility than saving the text output of the evmshow command. At a later time you can sort and filter the binary file and pass it to the evmshow command to view it in any format you like.

As you experiment with evmget, you will see that the events appear in batches, usually ordered chronologically. Each get function feeds its events back to the evmget command, which then outputs them in the order in which it received them. Because you usually want to see events in some order (often, but not always, chronological), you need to pipe the events through the evmsort command, which is described in Section D.3.4. Section D.3.5 introduces using the evmget -A command, which lets you retrieve, sort, and display events without building a pipeline.

Depending on the size and type of your system and the number of events being logged, event retrieval may take a noticeably long time. This is because each retrieval operation requires every channel's get function to read through its log files, convert its events to Event Manager events, and then apply the filter string (if any) to determine whether the event is passed back to the evmget command. The larger the log files, the longer this process takes.

Careful log file management helps to speed up the process. If you know that you want to display events that belong to a particular event channel, you can shorten the process by using the evmget -C command to display only the specified channel. For example:

# evmget -f '[pri >= 600]' -C syslog | evmshow | more

In this example, the get function runs only on the syslog channel, so the command completes its task quickly. A filter string is specified to return events that have a priority greater than 600. You can determine what channels are configured by using the evminfo -lc command, or by examining the channel configuration file. See evminfo(1) for more information.

D.3.4 Sorting Events Using evmsort

The evmsort command takes a stream of Event Manager events as input, sorts them into the requested order, and writes them to its stdout stream. The command is most useful in sorting the output from the evmget command, but it can be used to sort Event Manager events from any source. See evmsort(1) for more information.

Section D.3.3 explained that the events retrieved by the evmget command are output in batches, corresponding to the event channel configuration. You can use the evmsort command to sort the events into a preferred order, before passing them to the evmshow command for display. The following example shows a typical command sequence:

# export EVM_SHOW_TEMPLATE="@timestamp [@priority]  @@"
# evmget -f '[pri >= 600]' | evmsort | evmshow | more

By default, the evmsort command sorts events into chronological order, so the previous command is suitable for most cases. You can use the -s option to declare a sort specification if you want the events sorted differently. A sort specification is a text string that defines one or more sort keys, which are the data items on which you want to sort the events. The specification is a list of data item names, separated by colons (:). For example:

priority:timestamp

The preceding specification sorts events by timestamp within priority, so the first group of events that are returned are those with the lowest priority, sorted in their order of occurrence. You may use this specification as follows:

# evmget -f '[pri >= 600]' | evmsort -s "priority:timestamp" | evmshow | more

The default sort order is ascending, but you can change it to descending for an individual item specifier by appending a minus sign (-). You can explicitly request ascending order by specifying a plus sign (+). For example, the following command displays the highest priority events first (descending order), but within each priority range the events are sorted oldest first (ascending order):

# evmget -f '[pri >= 600]' | evmsort -s "priority-:timestamp+" | evmshow | more

For consistency with the show-template syntax, the evmsort command allows you to precede each item specifier with an at (@) character, as described in Section D.3.1. There is no requirement to do this, and it does not affect the operation.

When you establish your sorting preferences, you can create a new default sort sequence by setting the environment variable EVM_SORT_SPEC. The following Korn shell (ksh) commands are equivalent to the previous example:

# export EVM_SORT_SPEC="priority-:timestamp+"
# evmget -f '[pri >= 600]' | evmsort | evmshow | more

You can override the value of the EVM_SORT_SPEC variable at any time by supplying a different sort specification with the -s option.

D.3.5 Using the -A Option to Simplify the Command String

The Event Manager commands are designed to be building blocks, with each command doing one specific operation. This gives you great flexibility in developing shell scripts to manipulate event information. When you enter commands from the command line you may prefer to simplify the command.

The most common command sequence for event retrieval is the evmget command, piped into the evmsort command, piped into the evmshow command. You can then pipe the text output into the more command to display the output. Consider the following example:

# evmget -f '[pri >= 600]' | evmsort -s "priority-:timestamp+" | 
evmshow | more

You can simplify the preceding command by using the evmget -A command option, which automatically pipes the command output to other Event Manager commands. For example, you can use the -A option to simplify the previous command example as follows:

# evmget -A -f '[pri >= 600]' -s "priority-:timestamp+" | more

When the evmget -A command starts, it automatically runs the evmsort -A command, and pipes its output into that command. When the evmsort command starts, the -A option causes it to start the evmshow command, piping events into it for display. You can supply a sort specification with the -s option and a show-template with the -t option. These options are passed along to the evmsort command and evmget commands respectively.

The evmwatch command supports the -A described in Section D.3.6.

D.3.6 Monitoring Events Using evmwatch

You can use the evmwatch command to monitor event activity through a terminal window. This command is an Event Manager subscribing client. It makes a connection to the Event Manager daemon, sends it a subscription request, and waits to receive events. As events arrive, the evmwatch command writes them to the standard out stream (stdout) as binary Event Manager events.

You cannot display the output of the evmwatch command because it is a stream of binary events. You must use the evmshow command to format the events. The following example monitors all events, and displays them on your terminal as they occur:

evmwatch | evmshow -t "@timestamp [@priority]  @@"

Depending on your system type, and the level of event activity, this command may run for a while before any events are displayed. The command continues to run until you terminate it to regain control of your terminal, usually by pressing [Ctrl/c].

When a system is operating correctly, many of the events posted are low-priority informational events. You may want to filter these events out, particularly if your system has a high level of event activity. You can do this by supplying a filter to the evmwatch command:

# evmwatch -f "[priority >= 400]" | 
evmshow -t "@timestamp [@priority]  @@"

This example watches for events with a priority of error or higher. You can change the filter string to exclude any set of events that occur regularly and are uninteresting. Alternatively, you may need to watch for a particular set of events.

The preceding examples do not show the output of evmshow piped into more for display, because evmwatch is a realtime monitor. The evmwatch command displays events as they occur, rather than displaying them from a file. A command like pg or more may wait for the operator to intervene before reading more data from its input pipe; over time, this could lead to congestion in the pipeline. The Event Manager daemon cannot wait for its client (the evmwatch command) to clear its backlog; this results in the evmwatch command missing events. You should display the output from the evmwatch command directly on a terminal window, instead of using of piping commands to more or pg; also use the scroll bar to review the event list.

Avoid piping the output of the evmwatch command into the evmsort command because the evmsort command cannot sort events until it reads to the end of its input. As a monitoring program, the evmwatch command usually waits for input until it is killed explicitly. As a result, if you pipe the output of the evmwatch command directly into the evmsort command, there is no output from the evmsort command.

The -A option simplifies the command string by running the evmsort command and the evmshow command automatically. The evmwatch command also supports the -A option and automatically runs the evmshow command when you use it. You can specify a show-template as an option to the evmwatch command as follows:

# evmwatch -A -f "[priority >= 400]" -t \"@timestamp \
[@priority]  @@"

As with the evmget command, you can capture a set of interesting events in a file, to review later. It is more useful to store events in binary form than in text form, so you should send the output of the evmwatch command directly to a file, as shown in the following example, rather than piping it into the evmshow command first.

# evmwatch -f "[priority >= 400]" > my_events

The evmwatch command supports additional options that are useful for monitoring events from within a shell script. See evmwatch(1) for more information.

D.3.7 Understanding the Event Manager Mark Event

When you review or monitor event activity, you observe the following event that occurs every 15 minutes:

26-Jun-2000 08:57:45 [200] EVM: Mark event

The evmlog event channel posts this event to ensure that there is periodic event activity. If your system has a problem and you need to determine when it was last operational, you can look for mark commands in the system log by using the following command:

# evmget -f "[name *.evm.mark]" | evmshow -t "@timestamp @last_timestamp  @@"
26-Jun-2000 00:57:35 26-Jun-2000 04:42:40  [16 times] EVM: Mark event
26-Jun-2000 04:57:41 -  EVM: Mark event
26-Jun-2000 05:12:41 -  EVM: Mark event
26-Jun-2000 05:27:41 -  EVM: Mark event
26-Jun-2000 05:42:41 26-Jun-2000 09:12:45  [15 times] EVM: Mark event

If the default logger configuration file is in use, you usually see three individual mark events, followed by a single event preceded by [n times], where n is a number up to 16. This is the result of the logger's suppression facility, which minimizes wasted space by combining multiple events over a period of up to four hours. The normal timestamp value shows the first occurrence of a combined event, and the last_timestamp data item shows the time of the last occurrence. The example includes the last_timestamp data item in the show-template, which displays the last mark event, posted at 09:12:45. This mark event tells you that the system was operational at that time.

To disable mark event posting, edit the channel configuration file to make either of the following changes:

Comment out the evmlog channel's fn_monitor entry to disable it completely

Change the mon_period value for the channel to change the frequency with which the event is posted

See Section D.2.2 and evmlogger.conf(4) for more information about event suppression.

D.3.8 Viewing Events Using the Event Viewer

The graphical event viewer provides a simple and convenient interface to the system event logs. The event viewer is an integral part of the AMS system management suite; you can use it in a character cell terminal or from a Web browser.

To launch the event viewer from SPM refer to Section 2.6.4; to launch from PCM see Section 5.8.4.

When you run the event viewer for the first time a warning message may indicate that events are filtered to show only high priority events. If your system is operating normally it is likely that no events are displayed in the event summary window.

To choose the events you want to see, select Filter... at the bottom of the window, and change the filter criteria in the Filter window. If you want to see all stored events, make sure that all the check boxes at the left side of the window are in the unchecked state, and select OK.

If your system produces a high level of event activity you can reduce the number of events shown, and the time taken to display them, by checking the Priority box and adjusting the priority range. Setting the range to 400-700 displays all events with a priority of error and higher. Setting the low end of the range to 300 includes warning events in the display.

You can check any of the buttons at the left of the Filter window to include additional criteria in the display filter. Each time you make a change you must select Apply to apply the change to the event list, or select OK to apply the change and return to the main viewer window.

The Filter dialog window offers an intuitive and convenient way for you to build an event filter string without having to type it. If you are familiar with the filter syntax and you want to make better use of its power, you can enter a filter string through the Advanced Filter dialog box, which you access by selecting Options... at the bottom of the main event window. You can also save a filter string and reuse it later. For more information about the filter syntax, see EvmFilter(5).

One of the most important features of the viewer is the ease with which you can display a detailed view of any event. Simply select the event in the summary window and select Details... to see all the information available. From the Event Details window you can browse through the event list without returning to the main window.

You can change the viewer display by selecting Customize... and Options.... To change the order in which events are displayed, select Sort.... Select Help... from any window for detailed information about the viewer and its facilities.

Note

The event viewer does not monitor event activity in real time. To display an updated view of the event list, select Refresh from the main window.

D.3.9 Advanced Selection and Filtering Techniques

The following section describes some additional filtering techniques that you can use to further improve event selection, so that you receive only the events in which you are interested.

How to filter events according to their time of posting (Section D.3.9.1)

How to filter using the event-id identifier (Section D.3.9.2)

How to filter using reserved component names (Section D.3.9.3)

How to use filter files (Section D.3.9.4)

D.3.9.1 Filtering By Time

You can filter for events according to the time at which they were posted by using the timestamp, before, since, and age keywords. You may find that the age keyword is the easiest of these keywords to use, and the most useful for everyday operation.

When you use the timestamp keyword, you must supply a string that defines a time range in the following way:

year:month-of-year:day-of-month:day-of-week:hours:minutes:seconds

You can use an asterisk (*) as a wildcard character for any of the components, so to select events that occurred on July 6, 2002 you may use the following commands:

# export EVM_SHOW_TEMPLATE="@timestamp [@priority] @@"
# evmget -A -f '[timestamp 2002:7:6:*:*:*:*]' | more

The asterisks (*) in the final four components indicate that you are interested in all events that occurred on that day, no matter what time they occurred. Also, you can specify one or more ranges in any position, as shown in the following command:

# evmget -A -f '[timestamp 2002:*:*:1-3,5:*:*:*]' | more

The fourth component specifies the day of the week. Searching for events with posting times in the range 1-3 or 5 yields all events that were posted on a Monday, Tuesday, Wednesday or Friday in the year 2002.

The before and since keywords use similar specifier strings, but you cannot use wildcard characters and there is no day of the week indicator. For example, the following command finds events that were posted after 3:00p.m. on July 6, 2002:

# evmget -A -f '[since 2002:7:6:15:0:0]' | more

The age keyword provides a more convenient and intuitive way to select events according to their timestamps. As a system administrator you may be most interested in recent events that indicate a system problem. You can combine the event filter's priority and age keywords to find such events. For example, the following command sequence shows all events with a priority of error (400) or higher, that occurred either yesterday or today (the age of the event is less than 2 days):

# evmget -A -f '[pri >= 400] and [age < 2d]' | more

In the preceding example, 2d specifies events that are less than 2 days old. You can specify an age in seconds (s), minutes (m), hours (h), days (d), or weeks (w). See EvmFilter(5) for information about how each specifier is used in calculating an event's age.

You can use a more complex filter to return events that occurred within a more specific period. The following example finds error events that occurred more than 3 days ago, but less than 6 days:

# evmget -A -f '[pri >= 400] and ([age < 6d] and [age > 3d])' | more

See EvmFilter(5) for detailed information on selecting events according to their timestamps, and the full filter syntax.

D.3.9.2 Using the event-id to Select Events for Detailed Display

Using the evmshow -d command option to display events can result in a large amount of output and you may want to limit the number of displayed events. Events that are posted through Event Manager contain a sequential identifier known as the event-id. You can use the event-id to select a specific event or a range of events for detailed display.

The event-id is not guaranteed to be unique within any particular set of events because the daemon's counter is set to zero each time it is restarted. To ensure that an event is unique, you must also use the timestamp when selecting events as shown in the following example:

# evmget -A -f '[age < 1d]' -t "@timestamp @event_id @@" | more

15-Apr-1999 14:19:06 0  EVM daemon: Configuration completed
15-Apr-1999 14:19:06 1  EVM daemon: Initialization completed
15-Apr-1999 14:19:06 2  EVM logger: Logger started
15-Apr-1999 14:19:06 3  EVM: Mark event - initial
15-Apr-1999 14:19:06 5  EVM logger: Started eventlog /var/evm/evmlog/evmlog.19990415
[1]               [2]
.
.
.

The age filter keyword selects all events that have occurred today, as indicated by the timestamp in the first column of data. [Return to example]

The @event_id specifier in the show template instructs the evmshow command to display the event-id for each retrieved event, which is shown in the second column of data. [Return to example]

When the event-ids are displayed, you can select the interesting events. For example, use the following command to display details of the initial mark event, which has an event-id of 3 in the preceding example output:

# evmget -f '[age < 1d] and [event_id = 3]' | evmshow -d | more

You can select a range of events by using a more complex filter as shown in the following example:

# evmget -f '[age < 1d] and [event_id >= 1] and [event_id <= 3]'|
 evmshow -d | more

Choose the time range carefully to select the right set of events. If you recently rebooted your system, specify a filter of [age < 2h] to select events occurring within the preceding 2 hours.

The most convenient way to select events for detailed display is to use the event viewer described in Section D.3.8.

D.3.9.3 Searching for Reserved Component Names

Some event names include reserved component names as name extensions. These components begin with an underscore character (_), and usually are followed by a component that identifies the item for which the event is being posted. For example, the names of many hardware-related events include the component _hwid, followed by the numeric hardware identifier of the item. Reserved component names are appended automatically as an extension to the event name. The name is appended, followed by the value for the named variable. This is done for every reserved component name. For example, an event with the name @SYS_VP@.temperature_high and the variable _degrees with the value 212 would be observed as an event with the name @SYS_VP@.temperature_high._degrees.212.

You can search for all such events by using the following command:

# evmget -A -f '[name *._hwid]' | more

If you know the hardware identifier of a specific device, you can narrow the search for events related to that device by using a command similar to the following:

# evmget -A -f '[name *._hwid.4]' | more

D.3.9.4 Using Filter Files

You can save a useful filter in a file and recall it by using the Event Manager's indirect filter facility. Filter files have names with the suffix .evf, and can contain any number of named filters. For example, the following filter file entry selects all syslog events that refer to SCSI devices:

filter {
    name "scsi"
    value "[name @SYS_VP@.syslog.hw.scsi]"
    title "Syslog SCSI events"
}

In this example, the @SYS_VP@ is a standard Event Manager macro that is replaced by sys.unix when the filter is used.

To use indirect filtering, specify the at sign (@), followed by the name of the file containing the filter instead of a filter string, as shown in the following example:

# evmget -A -f @syslog

You do not need to include the .evf suffix when you specify a filter file name in such commands.

The previous example uses the first filter in the file, but you can choose a different filter by specifying its name as follows:

# evmget -A -f @syslog:scsi

You can include as many filters as you like in a single file, or you can keep each filter in its own file. The preceding example specifies the syslog filter, which is included in Event Manager. Other filters are provided in the /usr/share/evm/filters directory. Use these files as examples for establishing your own filter library.

The evmshow -F command option provides an easy way for you to see the contents of a stored filter. The -F option causes the evmshow command to display the filter string and then exit without reading any events. In the following example, the evmshow command displays the contents of the filter named scsi, stored in the syslog.evf file:

# evmshow -f @syslog:scsi -F
( [name sys.unix.syslog.hw.scsi] )

See evmfilterfile(4) for complete information about the syntax of filter files, and where to locate your files.

Note

Do not edit the filter files provided in the /usr/share/evm/filters directory. Your changes may be overwritten without warning by a future installation update.

D.3.10 Logging and Forwarding Events

The response to an event is any action determined by your site-specific needs and conditions. This response can range from activating alarms or paging responsible personnel, to making a log entry or ignoring an expected occurrence of a regular activity.

You can configure the event processing sequence to perform a series of dependent tasks, by using an event output by one task as the trigger to activate the next process. Event Manager provides an interface to the response activity through the logging facility. The available options are event storage and event forwarding.

The Event Manager logger, evmlogger, started automatically by the Event Manager daemon, is responsible for the following:

Displaying selected events on the system console or other device
If a terminal device is indicated as the logfile in the configuration file, all events meeting the filter specifications of an eventlog statement are formatted for display on the terminal. (See Section D.2.2 for a discussion of the configuration file.)

Storing selected events in one or more log files

Forwarding selected events to interested parties in some other form

By default, the logger handles events posted through its local daemon, but you can also configure it to handle events posted on remote systems.

The logger is an ordinary Event Manager client that is controlled through a configuration file. The default is the /etc/evmlogger.conf file, described in Section D.2.2. See evmlogger.conf(4) for more information on this file and evmlogger(8) for more information on the command.

D.3.10.1 Logging Events

All events meeting the specifications of an eventlog group in the configuration file are written to the event log. See Section D.1.4 for the default location of this file and the naming conventions.

As shown in Example D-1, you can include a suppress group specification in an eventlog statement in the configuration file. When you include such a statement, events meeting the suppression criteria are not entered in the log. One instance of the event is stored, with additional data indicating the number of events and the time of the first and last occurrence of the event. See evmlogger.conf(4) for the explanation of this criterion.

D.3.10.2 Using Forwarding to Handle Events Automatically

If you want to automate the handling of selected events, you can configure the Event Manager logger to forward the event by executing a command. For example, you can mail the event information to a paging service, or invoke an event-handling application program.

By default, the logger is configured to mail high priority events to the root user. You can use that default forwarding command as an example for developing your own actions. See Section D.2.2 and evmlogger.conf(4) for more information.

All events meeting the filter specifications of a forward statement in the configuration file are written to the standard input (stdin) of the command specified in the statement. The command is the name of a shell script, a single command, a series of commands (pipeline), or any other executable statement. The following operations are typically specified as a forwarding action:

Specifying the mail command or mailx command, or another command line mail processor, to send a mail message to a responsible person or paging service

Invoking additional software that causes emergency shutdown procedures to commence

Invoking a dependent process that is waiting for the event to occur

When configuring the logger to forward an event, note the following:

The event selected for forwarding is piped into the configured forwarding command. If your commands need to deal with text information, the evmshow command must be the first command in the pipeline so that the event is converted to text form.

The logger executes the forwarding command asynchronously, meaning that it starts the command and then continues with its normal operation without waiting for the command to finish. The following behaviors are normal:
- If multiple forwarders are specified in the logger's configuration file, and the same event is to be handled by more than one forwarder, the logger starts each forwarding command without waiting for the others to finish, so the commands may execute simultaneously.
- If the logger receives another event to be processed by a forwarding command, and the command is still processing the previous event, the logger queues the new event. When the command finishes, the logger restarts it, passing it the new event. By default, the logger queues up to 100 events for each forwarding command. You can increase this limit by specifying a MAXQUEUE keyword in the forwarder's configuration.
See evmlogger.conf(4) for more information.

Event text may include characters such as quotes, which have special meaning to the shell. Be sure to post test versions of the event to verify that your command executes correctly under realistic conditions.

You must take care that the forwarding command does not itself result in the posting of events which would cause an event loop. For example, if you use mail to forward events, the forwarder's filter must exclude mail events.

Use the logger's secondary configuration file facility for adding forwarders or other configuration items as described in EXPLICIT_ANCHOR.

D.4 Troubleshooting Event Manager

The following list describes actions you can take if you encounter specific problems:

A subscribing application fails to receive expected events
Verify that the user is a member of amsgroup by checking that the user's name is present in /etc/group.
```
# more /etc/evm.auth
```
Verify that the event is registered by using the following command:
```
# evmwatch -i -f `[nameevent_name]' |
  evmshow -t "@name"
```
If the events are still not shown, run evmreload and examine it again. If they are still not visible, verify that the template files are correctly installed.
Verify that the subscriber is authorized to access these events, by using the following command:
```
# more /etc/evm.auth
```
Verify that the expected events are actually being posted by using the following command:
```
# evmwatch | evmshow -t "@name @@"
```
Run the program that posts the event, and verify that the preceding evmwatch command displays them correctly.

Event retrieval through evmget or the event viewer is slow
Examine the sizes of all log files, particularly the evmlog files (/var/evm/evmlog).
Use the ls -l command when listing file sizes to ensure that you see the file itself and not a symbolic link.

Expected events are not being logged
Examine the event priority from the PCM user interface. Only events with a priority of 200 or higher are logged by the Event Manager logger.

evmlogger: Missed receipt of number events
This error occurs when events overflow the receive buffer, whose size is set to the default system socket buffer maximum. You can alter this value by following the Linux kernel tuning instructions provided with your Linux distribution.