Skip to content
CA Single Sign-On - 12.52 SP1
Documentation powered by DocOps

Run a Stress Test

Last update December 5, 2014

Contents

The Test Tool allows you to test performance when the Policy Server receives more than one request at a time. Using stress tests, you can simulate multiple agents talking to the Policy Server simultaneously or a single agent communicating with the Policy Server on multiple threads.

Stress tests are run in Advanced Playback mode. The Test Tool receives instructions from a Thread Control file that specifies which tests to run and how many times to run them. After executing the instructions in the thread control file, the results of the test are written to an output file.

Configure a Thread Control File

Configure a Thread Control file to define the Command Script files to run, number of repetitions, threads, and so on to automate complex tests for stress testing. A Thread Control file contains multiple instruction lines in the Test Tool’s own scripting language and comments, indicated by the # symbol at the beginning of a line.

The basic instructions are in the following format:

command_script_file, repetition_count, thread_count, max_time_in_seconds

  • command_script_file
    Specifies the pathname of the previously recorded command script file.
  • repetition_count
    Specifies the number of times that the Test Tool runs the Command Script.
  • thread_count
    Specifies how many simultaneous threads the Test Tool initiates to run the Command Script.
  • max_time_in_seconds
    (Optional) Specifies a limit (in seconds) for the duration of the test. If elapsed test time exceeds this limit then playback halts, regardless of whether the configured number of repetitions are complete.

For example:

# c:\temp\test_data.txt, 8, 6, 120

This line specifies that:

  • The input Command Script is c:\temp\test_data.txt
  • The Test Tool runs the script eight times
  • Six simultaneous threads run the script
  • The test ends after 120 seconds, regardless of whether eight repetitions are complete

The Test Tool writes the output of the test to a file. The output file name is the name of the input file with _out# appended to it, where # is the incremented thread number. For example, the output files for the test above are c:\temp\test_data.txt_out1 to c:\temp\test_data.txt_out6.

The Test Tool scripting language includes the commands in the following table to control the script file output. See the following figure for a sample thread control file.

Command Description
.report Generates a final report (as an output file) summarizing the test results. The report does not include the status of each server request. This is the default.
.reportspread Generates a file containing a response time spread report. The file name is command_script_stats_spread (for example, isprotected-record.txt_stats_spread ).
.output Generates a final report (as an output file) summarizing the overall results including the status of each server request.
.viewstats Displays final statistics in a text editor.
.verbose Generates output files containing the details of each server request.
.brief Generates output files containing the brief results of each server request. This option is only valid when used with the .output command.
.sleep Lets the Test Tool pause for a specified amount of time (in milliseconds). This simulates intermittent server requests.
.userselectionmode Determines whether usernames are used sequentially or selected randomly. Valid values are 0 (sequential) and 1 (random).
For more information, see (Optional) Configure How the Test Handles Usernames.
.randomseed Allows the pseudo-random sequence of names to be repeated (up to thread ordering) for each test.
For more information, see (Optional) Configure How the Test Handles Usernames.
.connect settings_file Initializes the Test Tool using information from the Test Tool Settings file to set up a multi-threaded test with one simulated Agent. The default multi-threaded test comprises multiple simulated Agents with one simulated Agent per thread. You can also set up a test with one simulated Agent and multiple threads by using this option.
.disconnect Un-initializes the Test Tool to indicate the end of one simulated Agent multi-threaded test.

Example Thread Control File

.output

.connect c:\test\smtest.ini

.brief

c:\temp\test_data1.txt, 2, 3

.verbose

.sleep 5000

c:\temp\test_data1.txt, 2, 2

.brief

c:\temp\test_data1.txt, 3, 4

.connect smtest.ini

c:\temp\test_data1.txt, 5, 6

.disconnect

Perform a Stress Test Using a Thread Control File

Use the Advanced Playback test mode to run a stress test defined in a Thread Control file.

Follow these steps:

  1. Select Advanced Playback in the Mode group box.
  2. Enter the name of the Thread Control file in the Control field.
  3. In the Command group box, click Run Script.
    The Test Tool runs the Thread Control file and creates an output script file.

(Optional) Configure How the Test Handles Usernames

The following three parameters configure how a test handles usernames.

  • 34 UserCount:
    (Defined in the Command Script file.) Specifies the number of users of the format AAAAAA, BAAAAA, through ZZZZZZ that the test uses for authentication and authorization. If the UserCount: parameter is set, the UserName: parameter is ignored.

    Note: Although it is ignored, the Username: parameter is still required. Do not delete it.

    Add the UserCount: parameter using the same format as the other parameters in the Command Script file. That is, enter the complete parameter name (including the "34 " prefix) and the parameter value starting at column 24 of the line.
    For example:

    34 UserCount:          1000

  • .userselectionmode
    (Defined in the Thread Control File) Determines whether usernames are selected sequentially or randomly. Valid values fare 1 and 2.
    If set to 1, the test steps sequentially step through usernames until the value specified by the UserCount parameter is reached. Usernames then wrap back to "AAAAAA" until all repetitions are completed or time has elapsed.
    If set to 2 (the default), users are selected randomly.
  • .randomseed
    (Defined in the Thread Control File) If configured with a non-zero integer value, causes a randomly ordered set of usernames in the range "AAAAAA" until the user name indexed by the value of the "UserCount" parameter. This allows the pseudorandom sequence of names to be repeated (up to thread ordering) for each test.
    If .randomseed is not specified, the current time is used.

(Optional) Simulate a Steady Load

To simulate a steady load in which requests are sent to policy server at a constant rate, configure the following parameter in the Thread Control File.

  • .sleepbetweenrequests
    Specifies the amount of sleep time (in milliseconds) between each request to the Policy Server on each thread.

For example, if the.sleepbetweenrequests parameter is set to 5 (milliseconds), then requests are sent to the Policy Server approximately 200 times per second on each thread.

Note: The actual request rate can be influenced by the following factors: time requests take to complete, amount of CPU time that each thread gets, and external factors.

(Optional) Write Playback Test Results to a CSV File

You can configure the Test Tool to write all playback test results to a comma-separated values (CSV) format file by setting the following system environment variable:

  • SMTESTCSVFILE
    Specifies the pathname of the CSV file to which the Test Tool should write playback test results. Playback test results are written in comma-separated format to the specified file with a header line at the beginning.

    If the file already exists, the file is appended. If the Thread Control file specifies playback of multiple recordings, the results are aggregated.

Report Viewing

When you run a stress test, the Test Tool generates a report summarizing the results. This report contains the following information:

  • Time the test started and finished
  • Total elapsed time
  • Minimum, maximum, and average request time
  • Total number of requests
  • Throughput
  • The number of tests run and their results

The report is saved in the directory where the thread control file is located. The name of the report is the name of the thread control file with _stats appended to it. For example, the thread control file, thread.txt, yields a report named thread.txt_stats.

Control File:   C:\temp\control.txt

Started at:     0:02:05.481

Finished at:    0:03:22.672

Total Elapsed:  0:00:01.396

Minimum Request Time:   0:00:00.400

Maximum Request Time:   0:00:05.498

Average Request Time:   0:00:01.396

Total Requests          234

Throughput (Req/Sec):   3.241

Request         Count   Yes     No     Timeout   Error

--------------- ------  ------  -----  --------  -----

IsProtected     78       72     0      0         6

IsAuthenticated 78       78     0      0         0

IsAuthorized    78       72     0      6         0

------------------      ------  -----  -------   ----- 

Total:          234     222     0      6         6

Was this helpful?

Please log in to post comments.