As previously mentioned, we use the Jasig CAS authentication server at KTH and occasionally, at least nowadays, actually update it. In order to verify that any update is likely to meet the requirements of our production environment, we do some simple performance testing using Apache JMeter which we also use to verify performance of some other systems.
About Apache JMeter
JMeter is a free, open-source tool to automate functional behavior and measure performance, and it can be used to automate scenarios such as a user logging in, causing a couple of service authentications and logging out again. It has a reasonable user interface and can also be used without the user interface for automatic batch tests. If you use Jenkins, you can combine JMeter with the Performance Plugin for continuous performance monitoring of your builds.
It can be somewhat tricky to set up the test scenarios, but the documentation, especially the Component reference which details the behavior of the different controllers, samplers, assertions etc. you can use is quite good.
For our CAS verification, we have a reference environment with an authentication backend (an Active Directory as it happens) with known users and passwords. The JMeter setup consist of a JMX-file with the scenario, a file with username and password combinations separated by tabs and a file with properties containing configuration parameters for the JMX-file, including the name of file containing the users. A complete set of example files, GenericCasPerfTest.jmx, users.csv and localhost.properties are available in this GitHub repository and they are reasonably self explanatory.
JMeter is then launched with:
jmeter -p localhost.properties -t GenericCasPerfTest.jmx
The example files should work for a regular CAS server though I haven’t verified this, since our setup is slightly different. E.g., our server does not respond with the same username the user logins with but a site-wide internal identifier, which means that the test scenario I use is slightly modified to handle this.
Our test setup generates on average not quite two authentication transactions per second, which about the same, or slightly less than, the load our moderate size production site typically peaks at. JMeter provides counters and graphs during the test, in particular the Aggregate Report is useful, and the new statistics view in CAS provides nice graphs of how the system is performing, as well as counters from the ticket registry (provided your backend supports it) to monitor it’s behavior.
Our new Couchbase backend discussed previously also provides an interface with extensive counters for its performance.
Together these reports gives me a rough verification that it will handle the actual load in production.
You can always improve
Still, the setup I use could do with improvements. E.g., to better simulate actual behavior, I would need many more but much less active sessions. It is perfectly possible to tweak the provided setup this way, but this is left as an exercise for the reader.
More on Jasig CAS Wiki
Just as I wrote this I realize that there is a page with more test cases on the CAS Wiki, https://wiki.jasig.org/display/CASUM/Apache+JMeter