Skip to content
CA Application Performance Management - 10.5
Documentation powered by DocOps

Triage Application and Infrastructure Issues Starting from CEM Console

Last update September 17, 2018

Contents

Network data on the CEM console provides the diagnoser with high-level network performance information related to the application being monitored. You can then determine if network issues are contributing to the problem being triaged and drill into the Multi-Port Monitor for session level data. If metrics indicate that the network is not a concern, you can connect to the APM WebView to investigate and triage the application.

The following diagram shows a possible triaging scenario.

Triage from CEM

As a diagnoser, perform the following steps:

  1. Determine your Triaging Options.
  2. Continue Triaging from Multi-Port Monitor.
  3. Continue Triaging from APM WebView.

Determine your Triaging Options

When you see infrastructure concerns related to your applications, determine some triaging options to help guide your next steps.

Follow these steps:

  1. Log onto the CEM console.
  2. Select CEM, Incident Management, Defect.
  3. Do one of the following:
    1. Select a defect from the list.
    2. Search for a specific defect.
  4. Review the network health metrics associated with the selected defect.
    For example, if you see high network round-trip time or packet loss, you might choose to connect to the Multi-Port Monitor. From there, you can investigate the network infrastructure in more detail.
    See Network Metrics on the Defect Details Page for information on the network health metrics.

Network Metrics on Defect Details Page

When CEM identifies a defect, it queries Multi-Port Monitor for the session-level TCP information and saves it as part of the defect details. When you drill into a defect, this TCP information is made available on the CEM console.

Each field displays the average value of all TCP sessions for the application between the web server and client IP for the 15 minutes leading up to the defect time. Use the link on the TCP Conversation Analysis field to see the metrics for the individual TCP sessions.

The following network information is available on the Defect Details page:

  • NCT Obs

Connection Time Observations. The number of monitored TCP connections occurring during the selected time interval. A good indication of usage levels and a gauge of metric significance. For example, many observations can indicate an event that can affect users.

  • DTT

Data Transfer Time. Elapsed time between when the server starts responding and when it finishes sending data. Several factors affect this value, such as response size, available bandwidth, and interaction between the application and the network. Excludes the initial server response time and includes only NRTT if there is more data to send than fits in the TCP window. This value is related to the number of network round trips required to deliver all data and the delay per round trip.

  • ENRTT

Effective Network Round-Trip Time. Includes NRTT and Retransmission Delay, which is the delay that retransmissions cause for a transaction. Reflects the latency that users actually experience and serves as an indicator of the performance degradation that retransmissions cause.

  • NCT

Network Connection Time. The amount of time that it takes the client to confirm the server connection acknowledgment. In general, network latency causes delay in connection times. NCT serves as a baseline for carrier latency and comparison to NRTT values.

  • NRTT

Network Round-Trip Time. The amount of time it takes for a packet to travel to and from the server and clients on a network, excluding latency from retransmissions. Application and server processing times are excluded from this value. This value is often useful when compared to the NCT value.

  • NRTT Obs
    Network Round Trip Time Observations. The number of round trip between the server and clients on a network during the selected time interval. A good indication of utilization levels, as well as a gage of metric significance. For example, a large number of observations indicates that an event might affect many users.
  • Retrans

Retransmission Delay. The additional delay in the NRTT that retransmission cause. Retransmissions are packets that are retransmitted after data loss. The data is expressed as an average across all observations, not the actual retransmission time for each transaction. The NRTT value increases when Retransmission Delay causes a delay in client acknowledgment. This metric does not reveal the impact of losses on the DTT because of TCP congestion. This metric reflects only data loss from the server to the clients, not from clients to the server.

  • SCT

Server Connection Time. The amount of time from when the server receives the SYN packet from the client until the server sends the first SYN/ACK.

Opening a TCP connection involves exchanging three packets: SYN, SYN/ACK, and ACK. The TCP header has SYN (synchronize) and ACK (acknowledge) bits. The first packet has the SYN bit set. The second packet has both bits set. The third packet has only the ACK bit set. This exchange establishes the initial sequence numbers of the connection.

SCT and NCT comprise the Connection Setup Time metric.

  • SRT

Server Response Time. The amount of time a server takes to respond to a client request. Server speed, application design, and volume of requests affect SRT.

  • TCP Bytes

TCP data volume in bytes. The total number of TCP bytes sent and received during the selected time period by the selected host or pair of hosts.

  • TCP Bytes From

TCP data volume in bytes. Total number of Application-Layer bytes that the selected server sent to or received from clients during the selected time period.

  • TCP Byte Loss Percent

Data loss, expressed as a percentage of TCP bytes sent and received.

  • TCP Byte Rate
    TCP data volume in bytes. The data rate in bytes per second during the selected time period.
  • TCP Byte Rate From

TCP throughput in bits. The data rate in bits per second (bytes per second x 8) between the selected server and clients during the selected time period.

  • TCP Byte Rate Retransmtd

Ratio of retransmitted data to total data, percentage of data that was lost on the monitored network, and loss rate in bits per second.

  • TCP Conversation Analysis
    Link to Multi-Port Monitor TCP Conversation report on the Analysis tab. The client network information on Multi-Port Monitor corresponds to the client network ID for the session requested by the CEM console. The initial view of the TCP Conversation report is sorted by Transaction Time. From this report, you can access the Server/Client Pair view to see all clients belonging to the same subnet as the client IP address that is talking to the given server.
  • Packet Loss Percent

Data loss, expressed as a percentage of TCP packets that were sent and received.

  • TCP Packet Rate

TCP throughput in packets. The data rate in packets per second during the selected time period. ADA reports use the term Data Rate.

  • TCP Packet Rate From

TCP throughput in packets. The data rate in packets per second from the selected server to clients, or from clients to the server, during the selected time period.

  • TCP Packet Rate Retransmtd

Ratio of retransmitted data to total data, percentage of data that was lost on the monitored network, and loss rate in packets per second.

  • TCP Packets

TCP data volume in packets. The total number of TCP packets that the selected host (or pair of hosts) sent and received during the selected time period.

  • TCP Retransmtd Bytes

The number of TCP bytes that were retransmitted due to data loss.

  • TCP Retransmtd Packets

The number of TCP packets that were retransmitted due to data loss.

  • Transaction Time

The amount of time from the moment a client sends the request (packet-level or transaction-level) to the moment the client receives the last packet in the response.

  • Transaction Time Observed

Transaction Time Observations. The number of monitored TCP transactions that occurred during the selected interval. A good indication of usage levels and a gauge of metric significance. For example, many observations can indicate an event that can affect many users.

Continue Triaging from Multi-Port Monitor

If the network metrics for the defect show degraded network/server performance (such as high network round-trip time or packet loss), one option is to connect to the Multi-Port Monitor via the TCP Conversation Analysis link. From the Multi-Port Monitor, you can investigate the infrastructure in more detail. Do one of the following to access the Multi-Port Monitor.

Follow these steps:

  1. Select the link associated with the TCP Conversation Analysis field from the Defect Details page on CEM.
    This link takes you to the TCP Conversation report in the Multi-Port Monitor. The initial view of the TCP Conversation report is sorted by Transaction Time. From this report, you can access the Server/Client Pair view to see all clients belonging to the same subnet as the client IP address that is talking to the given server.
    See the Multi-Port Monitor User Guide for more information.

Follow these steps:

  1. Log on to Multi-Port Monitor.
  2. Select the Analysis tab.
  3. Navigate to the relevant TCP Conversation report.
    See the Multi-Port Monitor User Guide for more information.

Continue Triaging from APM WebView

If the network metrics for the defect are within the expected range for your business, one option is to continue triaging the application from the APM WebView

Follow these steps:

  1. Log on to the APM WebView.
  2. Use the Console or Investigator to triage your issue.
Was this helpful?

Please log in to post comments.