Stressing SignalR by Crank-ing it up

In our previous article we saw how to use the Windows PerfMon.exe utility along with custom SignalR performance counters to monitor performance of the SignalR Test Harness sample. However we had only three clients via three web browser clients. Today we’ll see a utility called Crank - again a part of the SignalR codebase that the team uses to test limits of a SignalR app on a given server.

Thanks to Sumit Maitra for sharing his expertise in this article

Types of SignalR Load

Before we get hands on, it’s worth reviewing the types of load that a SignalR application can handle. Quoting from the recent Build talk by Damian Edwards, SignalR can be subjected to four types of loads

1. Server Broadcast – In this scenario, the server broadcasts a message to all clients. Here the server is sending out only one message, but it’s going out to all connected clients. So this relatively less stressful thing to do.

2. Server Push – What’s the difference with Server Broadcast? In case of Server Push, a message is sent to one or a few connections (owned by the same user preferably). So if a server is going to push lots of messages to lots of different people, it increases the load on the server, specially if you have a lot of connections and lot of messages. However, this is server controlled and can be managed with respect to load.

3. User Event Driven – Chats or Collaboration apps are the perfect example. Load on the server depends on both number of connections and the amount of chatter/communication between the clients or client groups. Increasing number of connections will increase load as will increased communication between the connected people. This is a scenario where it becomes difficult to control load on the server only because of the dynamics of load; communication frequency and number of connections are both dynamic.

4. High Frequency – This scenario is typical for realtime interactive games (e.g. http://shootr.signalr.com). Here the server sends out messages at a very high frequency (SignalR team strongly recommends it be < 25Hz). Also messages sent out to each client varies based on their gameplay. So increase in number of simultaneous users increases load proportionally. Due to the high fixed rate of unique messages, this scenario can put a lot of load on the system quickly.

In our previous article we saw the ‘Server Broadcast’ scenario but for a very limited number of clients. Today we will try it out for a much higher number of clients. The other three scenarios will need creative test harness creation from us, we’ll leave that for another day.

Using Crank

As with our previous article, we’ll start off with the SignalR codebase (can download the zip here, or fork on Github from here).

The Crank is a command line tool and the executable is under src\Microsoft.AspNet.SignalR.Crank. So the executable after a successful build is two folders down under bin\Debug

Crank Options

Crank has the following options

So if we setup Micrsoft.AspNet.SignalR.LoadTestHarness as the default project and run the application from Visual Studio, it will by default run at http://localhost:29573/. To setup Crank such that it creates 100 clients that connect to this app, we type the following command in

crank /Clients:100 /Url:http://localhost:29573/TestConnection /BatchSize:10 /Duration:120

This starts Crank with 100 clients pointing to the URL specified (Note: TestConnection is the name of the PersistentConnection in the LoadTestHarness sample). It creates the 100 clients in batches of 10 and the clients remain connected for 120 seconds as set by the Duration flag. But before we start Crank, let’s setup our performance counters.

Start perfmon.exe as Administrator and add all the SignalR performance counters. If you don’t see the loadtestharness object instance, close your Visual Studio, reopen in Administrator mode, reload project and run. It should appear in the Instance Object list.

Once the Performance counters are selected, we are ready for our test. The perf counter at the beginning of the test looks like the following.

Generating Clients and Broadcasting Data

We are ready to run Crank now, so let’s start with 100 clients for two minutes and see how loaded our server gets.

As we can see below, maintaining 100 connections is pretty easy for SignalR on this machine as it roughly uses about 15% CPU. But at this point, there are no messages being exchanged.

Let’s see what happens when we start exchanging messages.

At 6 messages per second we are almost at 80% load

When we push it to 12 messages per second we hit 100% CPU and the frame rate is unable to keep up as it sticks to 11 FPS.

On this rather low powered machine we were able to sustain 100 connections at 11 messages per second where message size was 32 bytes. This can serve as a baseline and when we are deploying to production, we could run a similar test using the Web Application and Crank to approximately determine its performance.

Limitations of Crank

My buddy Sumit told me that Crank can only connect to PersistentConnection based apps and not Hub based apps. Though it makes it unsuitable straight off the bat, the sample can serve as a base for your custom stress testing harness. In future we will see how we can build a custom testing harness using the .NET client object to call our Hub API and stress it out using multiple connections.

Conclusion

We looked at the Crank utility and saw how we could exercise the Test Harness SignalR Application. Crank is not a tool you use out of the box for your SignalR apps, but something we can use as a guideline to build our own test harnesses.

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

Suprotim Agarwal, MCSD, MCAD, MCDBA, MCSE, is the founder of DotNetCurry, DNC Magazine for Developers, SQLServerCurry and DevCurry. He has also authored a couple of books 51 Recipes using jQuery with ASP.NET Controls and The Absolutely Awesome jQuery CookBook.

Suprotim has received the prestigious Microsoft MVP award for Sixteen consecutive years. In a professional capacity, he is the CEO of A2Z Knowledge Visuals Pvt Ltd, a digital group that offers Digital Marketing and Branding services to businesses, both in a start-up and enterprise environment.

Get in touch with him on Twitter @suprotimagarwal or at LinkedIn