Handle reconnections to signalR host

This article was originally published at tech.blinemedical.com

SignalR does a great job of dealing with reconnecting to a host when either the client disconnects or the server disconnects. This is pretty handy since it handles all the intricacies of a persistent http connection for you. But what it doesn’t deal with is the initial negotiation to a server. If that fails you are stuck retrying yourself. I wrote a simple reconnection function that leverages the scheduling functionality of Rx to continuously try to reconnect to the server.

For our SignalR usage (version 0.5.2), I’m using the exposed Hub functionality, not the persistent connections since I liked the encapsulation that Hub’s gave us. In the following example, we have a local member variable called Connection which is a HubConnection type created with this code.

HubConnection Connection = new HubConnection(Url);

HubConnection has a Start method that you use to initialize connections to the Url.  Connection.Start() internally creates an asynchronous task that looks like this, after unwrapping the nicely packaged methods:

private Task Negotiate(IClientTransport transport)
{
    var negotiateTcs = new TaskCompletionSource<object>();

    transport.Negotiate(this).Then(negotiationResponse =>
    {
        VerifyProtocolVersion(negotiationResponse.ProtocolVersion);

        ConnectionId = negotiationResponse.ConnectionId;

        var data = OnSending();
        StartTransport(data).ContinueWith(negotiateTcs);
    })
    .ContinueWithNotComplete(negotiateTcs);

    var tcs = new TaskCompletionSource<object>();
    negotiateTcs.Task.ContinueWith(task =>
    {
        try
        {
            // If there's any errors starting then Stop the connection
            if (task.IsFaulted)
            {
                Stop();
                tcs.SetException(task.Exception);
            }
            else if (task.IsCanceled)
            {
                Stop();
                tcs.SetCanceled();
            }
            else
            {
                tcs.SetResult(null);
            }
        }
        catch (Exception ex)
        {
            tcs.SetException(ex);
        }
    },
    TaskContinuationOptions.ExecuteSynchronously);

    return tcs.Task;
}

The comment above the IsFaulted check says that if the server fails to connect, an exception is set and the transport is closed. Since SignalR utilizes the task parallel library we can just call Start() again and get a new task.

Here is the snippet we use to continuously reconnect:

/// <summary>
/// Handles if the connection start task fails and retries every 5 seconds until
/// it succeeds
/// </summary>
/// <param name="startTask"></param>
/// <param name="connectionSucessAction"></param>
private void HandleConnectionStart(Task startTask)
{
    startTask.ContinueWith(task =>
    {
        try
        {
            if (task.IsFaulted)
            {
                // make sure to observe the exception or we can get an aggregate exception
                foreach (var e in task.Exception.Flatten().InnerExceptions)
                {
                    Log.WarnOnce(this, "Observed exception trying to handle connection start: " + e.Message);
                }

                Log.WarnOnce(this, "Unable to connect to url {0}, retrying every 5 seconds", Url);
                RetryConnectionStart();
            }
            else
            {
                // do success actions
            }
        }
        catch(Exception ex)
        {
            Log.ErrorOnce(this, "Error handling connection start, retrying", ex);
            RetryConnectionStartRescheduler();
        }
    });
}

private void RetryConnectionStartRescheduler()
{
    ThreadUtil.ScheduleToThreadPool(TimeSpan.FromSeconds(5),
        () =>
        {
            try
            {
                HandleConnectionStart(Connection.Start());
            }
            catch(Exception ex)
            {
                Log.ErrorOnce(this, "Error retrying connection start, retrying", ex);
                RetryConnectionStartRescheduler();
            }
        });

}

ThreadUtil.ScheduleToThreadPool is a wrapper we have on top of the Rx framework’s threadpool scheduler. Internally it looks like this

public static void ScheduleToThreadPool(TimeSpan executeTime, Action action)
{
    Scheduler.ThreadPool.Schedule(DateTime.Now.Add(executeTime), action);
}

It’s important to note that you have to touch the exception object of a faulted task or use the exception Handle method in order to avoid an UnobservedTaskExceptions. Those happen to unobserved exceptions which are then rethrown on the finalizer thread.

In conclusion, by leveraging tasks and a couple simple scheduling utilities, we can cleanly and asynchronously schedule a new task to connect periodically. When we finally connect we can continue with our initialization logic. At this point the remaining signalR reconnection logic is handled by the hub.

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>