Skip to content
CA Application Performance Management - 10.0
Documentation powered by DocOps

apm-events-thresholds-config.xml

Last update September 30, 2016


Properties defined in one of three Enterprise Manager properties files control a number of Enterprise Manager behaviors. This section describes the properties in apm-events-thresholds-config.xml. (The others are IntroscopeEnterpriseMangager.properties and APMEnterpriseManager.properties).

The apm-events-thresholds-config.xml file, located in the <EM_Home>/config directory, contains properties for Introscope and CA CEM clamps and CA APM Important Events. 

  • When these clamps meet or exceed the configured threshold values, the Enterprise Manager notifies the APM Status console Clamps tab.
  • When the Important Events meet or exceed the configured threshold values, the Enterprise Manager notifies the APM Status console Important Events table view.
Important! Configure only the threshold values in the threshold value fields. Do not change any other fields such as clamp ID, importantEvent ID, and description because Introscope uses these fields for system monitoring.

You do not need to restart the Enterprise Manager or monitored application to put into effect any apm-events-thresholds-config.xml configuration changes. These properties use the Introscope hot configuration (hot config) service. If a property uses hot config, then you do not need to restart the Enterprise Manager after making configuration changes.

Important! Back up the apm-events-thresholds-config.xml file before making any configuration changes. If configuration introduces an error into the apm-events-thresholds-config.xml file, the Enterprise Manager cannot read the file and puts default clamp threshold settings into effect. For example, if a close bracket is missing in apm-events-thresholds-config.xml, the Enterprise Manager cannot read the file. If new configurations or the updated file do not work properly, perform one of these actions:
  • Troubleshoot the last-changed file, preferably using an XML editor.
  • Reinstate a previous backup that did work properly.

Introscope Clamp Properties

These properties define the limits set for Introscope-specific clamp properties. When the property thresholds are met or exceeded, the Enterprise Manager notifies the APM Status console.

These properties are hot configurable, meaning the values are applied when the Enterprise Manager checks the property file during a hot deploy. A hot deploy occurs about once every 60 seconds.

Find additional Enterprise Manager clamps in the IntroscopeEnterpriseManager.properties file in the <EM_Home>/config directory.

introscope.clw.max.users

The property limits the number of the Command Line Workstations (CLW) that are allowed to be connected to the Enterprise Manager at any given time.

Use this clamp if the CLW connections tend to slow down the Enterprise Manager. The CLW connections can overload the Enterprise Manager by trying to extract too many metrics.

  • Notes

You can use this clamp for a standalone Enterprise Manager, MOM, and CDV. However, the Workstation connections are not typically made to Collectors.

When a user runs a CLW command, the Enterprise Manager makes two connections for each query. One connection determines if the Enterprise Manager is available and the second is for the query. This means that each CLW command increases the introscope.clw.max.users count by two, not one. In addition, some CLWs make more than two connections to the Enterprise Manager. Consider these factors when setting the introscope.clw.max.users property threshold value.

  • Default
    500

introscope.enterprisemanager.agent.connection.limit

The introscope.enterprisemanager.agent.connection.limit property limits the number of agents that can connect to an Enterprise Manager.

This property applies to both standalone Enterprise Managers and Collectors.

When the clamp threshold is met and a new agent tries to connect to the Enterprise Manager, then these actions take place:

  • Enterprise Manager logs this clamp activation.
  • Enterprise Manager does not accept the new connection.
  • Notes

The Enterprise Manager uses the maxThreads property in the em-jetty-config.xml file to limit the number of HTTPS agent connections that the Enterprise Manager can service. The Enterprise Manager does not recognize the connected agents or process metrics from them if there are not enough threads.

When using HTTPS, the Enterprise Manager services only the number of agents that are configured in the maxThreads property. Be sure that the maxThreads value is greater than or equal to the introscope.enterprisemanager.agent.connection.limit value. When agents are configured to connect over HTTPS, if there are not enough threads some HTTPS agents may not appear in clients or report any metrics.

When the maxThreads property limits the number of agents an Enterprise Manager services, the Enterprise Manager and agent logs contain no errors, warnings, or messages about the non-serviced agents.

By default agents in on-premise installations do not connect using HTTPS.

  • Default
    400
  • Example
    introscope.enterprisemanager.agent.connection.limit=400

introscope.enterprisemanager.agent.error.limit

The introscope.enterprisemanager.agent.error.limit property clamps error snapshots. The property defines the limit for number of error snapshots that the Enterprise Manager can handle for each agent per interval.

This property applies to both standalone Enterprise Managers and Collectors.

If the number of error snapshots that the agent reports exceeds the specified limit, all the rest of the errors are dropped by the Enterprise Manager.

  • Default
    10
  • Example
    introscope.enterprisemanager.agent.error.limit=10

introscope.enterprisemanager.agent.metrics.limit

Set this property with the value for which agent reporting is clamped when the value is exceeded.

For example, if the default number of metrics, 50,000, is met, then the following occur:

  • The agent reports only the 50,000 metrics.
  • The agent does not send new metrics to the Enterprise Manager.

An inappropriately configured agent can create thousands of metrics in quick succession, overloading the Enterprise Manager.

Use this property to prevent metric overload. If an agent generates more than the specified number of metrics, the Enterprise Manager drops them. If this situation occurs, modify the agent configuration to reduce the volume of metrics reported.

  • Default
    50000
  • Note

This clamp property works with the introscope.agent.metricClamp property located in the Introscope.Agent.profile file.

If introscope.enterprisemanager.agent.metrics.limit clamp value is triggered before the introscope.agent.metricClamp value, then the Enterprise Manager reads agent metrics but does not report them in the Investigator metric browser tree.

If the introscope.agent.metricClamp clamp value is triggered before the introscope.enterprisemanager.agent.metrics.limit clamp value, the agent stops sending metrics to the Enterprise Manager.

introscope.enterprisemanager.agent.trace.limit

The introscope.enterprisemanager.agent.trace.limit property clamps Transaction Traces. The property defines the limit for number of transaction events that the Enterprise Manager can handle for each agent per interval.

This property applies to both standalone Enterprise Managers and Collectors.

If the number of Transaction Traces that the agent reports exceeds the specified limit, all the rest of the traces are dropped by the Enterprise Manager.

  • Default
    1000
  • Example
    introscope.enterprisemanager.agent.trace.limit=1000

introscope.enterprisemanager.collector.cdv.max

This property limits the number of Cross-cluster Data Viewers (CDVs) that can connect to a single Collector. The Cross-cluster Data Viewer usually connects to multiple Collectors across two or more clusters.

A CDV connection can add to Collector performance load by extracting metrics. Use this clamp to restrict the number of CDV connections that a Collector can accept.

  • Default
    5

introscope.enterprisemanager.disconnected.historical.agent.limit

The property limits the number of disconnected but still mounted historical agents. This clamp minimizes the use of Enterprise Manager run-time memory.

This clamp limits both historical agents that:

  • The Enterprise Manager can unmount automatically.
  • CA APM users must unmount manually.

For information about mounting and unmounting agents, see the CA APM Workstation User Guide.

When the clamp threshold value is hit, the Enterprise Manager tries to unmount the oldest disconnected historical agent that can be unmounted automatically. If there is no historical agent that the Enterprise Manager can automatically unmount, this means that CA APM users mounted manually all the disconnected historical agents. The Enterprise Manager never tries to unmount a disconnected historical agent that a CA APM user mounted manually.

  • Example

introscope.enterprisemanager.disconnected.historical.agent.limit=400

The clamp threshold is set to 400. An agent named Tomcat and 399 agents named Other are connected to an Enterprise Manager. Of the 400 agents, the Tomcat agent is the oldest agent that that can be unmounted automatically. The Enterprise Manager disconnects all 400 agents. As the agents reconnect, the Enterprise Manager evaluates whether the disconnected but still mounted agent count is below the 400 agent threshold. All 399 agents named Other reconnect, then the Tomcat agent attempts to reconnect. The Enterprise Manager performs these actions:

  • Finds that the Tomcat agent is the oldest agent that can be automatically unmounted among the 400 disconnected agents.
  • Unmounts the Tomcat agent.

After the Tomcat agent is unmounted the number of disconnected agents is 399, again under the 400 agent clamp threshold value.

  • Notes

The Enterprise Manager notifies the APM Status console when the following occur:

  • Disconnected mounted agent is unmounted due to the clamp.
  • Manual agent mount fails.

This clamp does not prune disconnected mounted agents when the following occur:

  • An Enterprise Manager is low on memory.
  • A disconnected historical agent times out and is automatically unmounted.

A disconnected historical agent stays mounted when the following occur:

  • The historical agent becomes disconnected and can be automatically unmounted.
  • The disconnected historical agent count is below the introscope.enterprisemanager.disconnected.historical.agent.limit value.

The Enterprise Manager mounts a disconnected historical agent when the following occurs:

  • A CA APM user tries to mount manually a disconnected historical agent.
  • The disconnected historical agent count is below the introscope.enterprisemanager.disconnected.historical.agent.limit value.

The Enterprise Manager searches for mounted disconnected agents that can be automatically unmounted when the following occur:

  • A historical agent that can be automatically unmounted becomes disconnected.
  • A CA APM user tries to mount a historical agent manually.
  • The disconnected historical agent count is equal to the introscope.enterprisemanager.disconnected.historical.agent.limit value.

If the Enterprise Manager finds a mounted disconnected agent, then it unmounts the oldest disconnected historical agent and keeps newly disconnected agent mounted. If there is no historical agent that the Enterprise Manager can unmount automatically, then the Enterprise Manager immediately unmounts the newly disconnected historical agent. If a CA APM user then tries to mount manually a historical agent, then the:

  • Workstation displays an error message instructing the user to unmount some historical agents to make room to mount new historical agents.
  • Enterprise Manager sends notification to the APM Status console.
  • Default
    400

introscope.enterprisemanager.events.limit

The introscope.enterprisemanager.events.limit property clamps transaction traces. The property defines the limit for number of transaction events processed by the Enterprise Manager for each interval.

This property applies to standalone Enterprise Managers, Collectors, and the MOM.

This clamp is applied to the total number of all transaction events generated by all agents. When the clamp is hit, all the rest of the transaction events reported by the agent are dropped for the remainder of that interval.

  • Default
    1250
  • Example
    introscope.enterprisemanager.events.limit=1250

introscope.enterprisemanager.metrics.historical.limit

This property limits the number of historical metrics in the Enterprise Manager. The limit is checked every 15 seconds, so the Enterprise Manager may go above this value before the clamp is applied. When the Enterprise Manager detects that it has exceeded this limit, the Enterprise Manager stops all agents from registering new metrics. Old metrics continue to report.

Default: 1200000

Note: Even when you increase the default value to 5 million, there is no significant impact on the performance.

introscope.enterprisemanager.metrics.live.limit

This property limits live metrics currently reporting from agents. When the Enterprise Manager detects that it has exceeded the limit, the Enterprise Manager will stop all agents from registering new metrics. The limit is checked every 15 seconds, so the Enterprise Manager may go above this value before the clamp is applied. Old metrics will continue to report. Metrics for which we have historical data will never be clamped.

  • Example

If an agent has 60k metrics it has reported to an Enterprise Manager before clamping, and then the clamp of 50k is applied; all 60k old metrics can still register with the Enterprise Manager. If however, any new metrics attempt to register for which we historical data is not already collected, then they will not be allowed to register.

  • Default
    500000

introscope.enterprisemanager.transactionevents.storage.max.disk.usage

Determines the maximum desired disk usage in MB for transaction trace storage. If this maximum is exceeded, then the daily historical trace stores are deleted starting with the oldest first until the total historical trace storage size is below this property value. Even if the size of this property value is exceeded, stored traces may not be deleted because the size of the trace index is not considered when determining what historical trace stores to delete.

A negative or zero value indicates that all historical trace values stored should be deleted.

If the Transaction Events database is larger than this value, historical storage files are deleted until the total database size is less than this value, starting with the oldest storage file.

This might delete historical data even if it has not reached the fully aged date. The current live storage is not deleted, even if its size exceeds the max database size.

If you determine that the Transaction Events database is too large, you can delete database files manually, when the Enterprise Manager is shut down. When the Enterprise Manager is restarted, the Transaction Trace storage system recognizes that the files have been deleted manually and adjusts the trace index accordingly.

  • Notes

If you delete files manually, delete only files in the traces directory that have this format: traces_yyyyMMdd.db, where yyyyMMdd represents the year, month, and day of the traces in that file.
Do not delete any other files in that directory or subdirectory.

  • Default
    1024

introscope.workstation.max.users

The property limits the number of Workstations that are allowed to be connected to the Enterprise Manager at any given time.

Use this clamp if Workstation connections tend to slow down the Enterprise Manager. Workstation connections can overload the Enterprise Manager by trying to extract too many metrics.

  • Note

You can use this clamp for a standalone Enterprise Manager, MOM and CDV. However, Workstation connections are not typically made to Collectors.

  • Default
    40

Customer Experience Clamp Properties

These properties define the limits set for CA CEM-specific clamp properties. When these property thresholds are met or exceeded, the Enterprise Manager notifies the APM Status console.

These properties are hot configurable, meaning the values are applied when the Enterprise Manager checks the property file during a hot deploy. A hot deploy occurs about once every 60 seconds.

introscope.enterprisemanager.max.application.user.rows

The maximum number of application users retrieved.

Too many application users can negatively affect performance.

  • Options
    numeric value
  • Default
    1000
  • Example
    introscope.enterprisemanager.max.application.user.rows =1000
  • Notes

When the maximum is reached, the CA CEM console limits what is displayed on the web page that lists the application users.

Limiting the number of changes listed since last synchronization

Before synchronizing monitors, you can review configuration changes that have made since the last synchronization. This is described in Viewing configuration changes since last synchronization.

The configuration changes appear in the Domain Configuration Change History table on the Monitors page. The introscope.enterprisemanager.max.number.domain.configuration.changes property determines the maximum number of rows in this table.

introscope.enterprisemanager.max.number.domain.configuration.changes

Sets the maximum number of changes that appear in the Domain Configuration Change History table when View Changes Since Last Synchronization is clicked (on the Setup > Monitors).

If the value is -1, then all the changes appear.

  • Options
    numeric value
  • Default
    0 (unlimited)
  • Example
    introscope.enterprisemanager.max.number.domain.configuration.changes=1000
  • Notes
    See Synchronize Monitors.

introscope.enterprisemanager.max.transaction.user.groups

The maximum number of automatically created user groups.

This property works in conjunction with automatically created user groups via request attribute or by IP subnet.

Too many user groups, even groups without any users, can negatively affect performance.

  • Options
    numeric value
    Maximum allowed value is 10,000.
  • Default
    5000

    We recommend you not to exceed the default value (5000) of the user groups to avoid performance issues.

  • Example
    introscope.enterprisemanager.max.transaction.user.groups=5000
  • Notes
    When the maximum is reached:
    • User groups are no longer created automatically.
    • Users are assigned to the New Users group instead of creating a new group.
    • The System > Events list displays a warning message.
    See Users and Groups on CEM.

introscope.enterprisemanager.transaction.discovery.max.nonidentifying.components

Maximum number of discovered non-identifying components to keep for a currently defined transaction. (This is in addition to any original non-identifying components.)

  • Options
    numerical value (a positive integer)
  • Default
    50
  • Example
    introscope.enterprisemanager.transaction.discovery.max.nonidentifying.component=60
  • Notes
    This limit is important for transactions where non-identifying components change frequently, for example, if a component is a GIF file that changes daily. Setting this limit prevents saving an excessive number (of probably obsolete) non-identifying components for a transaction.
    A CA CEM event called Limit exceeded adding new components is logged each time a new non-identifying component replaces an old one.
    For general information about automatically discovering non-identifying components, see CA APM Transaction Definition Guide.

Setting Important Event Thresholds

These properties define the limits used to notify the APM Status console about events important to Enterprise Manager health monitoring. They are hot configurable, meaning the values are applied when the Enterprise Manager checks the property file during a hot deploy. A hot deploy occurs about once every 60 seconds.

apm.events.thresholds.config.error

This property monitors the apm-events-thresholds-config.xml file for syntax errors. When the Enterprise Manager finds an error, it takes the following actions:

  • Notifies the APM Status console Important Events table view.
  • Reverts all the apm-events-thresholds-config.xml file clamp and important event properties values to their default values.

This property applies to all the Collectors in a cluster or to a Standalone Enterprise Manager.

  • Default
    Not applicable. This property is for notification purposes only and cannot be modified.

introscope.enterprisemanager.cpu.usage

This property defines the limit, in percent (%) at which Enterprise Manager CPU usage is considered an Important Event. When this threshold is met or exceeded, the Enterprise Manager notifies the APM Status console Important Events table view.

This property is based on Enterprise Manager monitoring of the CPU Capacity (%) metric. CPU usage greater than 60% indicates Enterprise Manager resource problems. For information about CPU usage and Enterprise Manager performance, see the CA APM Sizing and Performance Guide.

This property applies to all the Collectors in a cluster or to a Standalone Enterprise Manager.

  • Default
    60
  • Example
    introscope.enterprisemanager.cpu.usage=60

introscope.enterprisemanager.database.connectivity

This property monitors the connection between the Enterprise Manager and the APM database. When the connection is lost, the Enterprise Manager notifies the APM Status console Important Events table view.

This property applies to all the Collectors in a cluster or to a Standalone Enterprise Manager.

  • Default
    Not applicable. This property is for notification purposes only and cannot be modified.

introscope.enterprisemanager.harvest.cycle.diff

This property defines the limit, in number of time slices, that the Collector and MOM harvest cycles can be out of sync. When this threshold is met or exceeded, the Enterprise Manager notifies the APM Status console Important Events table view.

This property is based on comparing the Collector metrics harvest cycle to that of the MOM. During each 15-second harvest cycle, these actions take place:

  • Collector collects and aggregates agent metrics data.
  • Collector sends metrics data to the MOM.
  • MOM aggregates data from the Collector.
  • MOM reports the data to the Workstation.

A Collector does not send data to the MOM until its harvest cycle is complete. When a Collector harvest cycle does not complete within the MOM harvest cycle, data aggregation on the Collector and MOM are out of sync.

If the Collector and MOM are out of sync for more consecutive MOM harvest cycles than the property value, CA APM considers the cluster unhealthy.

  • Default
    4
  • Example
    introscope.enterprisemanager.harvest.cycle.diff=4
  • Notes
    Configure this property only on the MOM.

introscope.enterprisemanager.harvest.duration

This property defines the limit, in milliseconds (ms), at which Enterprise Manager harvest duration is considered an Important Event. When this threshold is met or exceeded, the Enterprise Manager notifies the APM Status console Important Events table view.

This property is based on Enterprise Manager monitoring of the Harvest Duration (ms) metric. A harvest duration longer than 3500 ms indicates Enterprise Manager resource problems. For information about harvest duration and Enterprise Manager performance, see the CA APM Sizing and Performance Guide.

This property applies to all the Collectors in a cluster or to a Standalone Enterprise Manager.

  • Default
    3500
  • Example
    introscope.enterprisemanager.harvest.duration=3500

introscope.enterprisemanager.smartstor.duration

This property defines the limit, in milliseconds (ms), at which Enterprise Manager SmartStor duration is considered an Important Event. When this threshold is met or exceeded, the Enterprise Manager notifies the APM Status console Important Events table view.

This property is based on Enterprise Manager monitoring of the SmartStor Duration (ms) metric. A SmartStor duration longer than 3500 ms indicates Enterprise Manager resource problems. For information about SmartStor duration and Enterprise Manager performance, see the CA APM Sizing and Performance Guide.

This property applies to all the Collectors in a cluster or to a Standalone Enterprise Manager.

  • Default
    3500
  • Example
    introscope.enterprisemanager.smarstor.duration=3500

Was this helpful?

Please log in to post comments.