Download ProtocolFramework.zip - 41.8 KB

Introduction

Welcome. Here is an effort towards an abstraction that developers can use to build concrete implementations of network protocols. It is provided as a set of classes in a C++ library that evryone can build upon.

The major benefit of such abstraction is the ease of integration of protocol implementations into applications where we do not want to waste time caring about how to implement a desired protocol or adapting a third party implementation of it.

Additionally, the abstraction helps people easily join their efforts together, reuse their works and even open the door for other ideas. For example, while some developers can work on protocol implementation, others can focus on protocol testing tools.

Goals

An abstraction is good as long as it solves all current problems and needs, and stays valid for a long time before it is changed. Additionally it should be natural and concise. Meaning, all differences are brought to one model. When the developer, looks at the abstraction he should feel it natural and he should feel like the design is guiding him easily through the implementation.

Those goals speak about the protocol developer. From the application developer point of view, the abstraction should make it easy to integrate implementations into its application, and most importantly, make it easy to wire protocol on top of each others, For example, deciding to add security should be as easy as wiring two protocols implementations with few calls. Or, if the application is transfering huge data and we find someone coming out with a compression protocol, it should be easy to stack that implementation to the top. Following diagrams shows an example where network data enter the lower layer which is TLS, which decrypt it into Websocket frames of which payload is then decompressed into discrete units of buffers that get deserialized into structured message objects that application reacts to. When application sends a message object, that gets serialized into a buffer of bytes, that get compressed, then framed as Websocket frames, then encrypted and sent to TCP.

Implementation

Trying to imagine what the abstraction will be, requires one to have looked at many protocol implementations, and have deduced common requirements. From my own experience, I write down the list of most important requirements, that I've see. I am excited to see what other people think, approve or criticize.

Encoding Vs Serialization: Protocols should never impose a way of representation of the content to be carried. In a layered stack of protocol, the upper layer will still hand the received data as just byte content. It is not the business of protocol to deserialize bytes into specific structured types, or expect input to be inform of particular type before doing serialization. Protocol will encode a bytes buffer into a bytes buffer, and will decode a bytes buffer into a bytes buffer. I think any protocol implemntation that goes beyond that, is doing some extra stuff that just makes it difficult to reuse the implementation, difficult to layer protocols together, etc.
Many protocols needs a way to store per-connection data. encoding and decoding may depend on the evolution of that data. (In ProtocolFramework, this is ProtocolContext class)
When data is received, evry protocol layer will try to decode content and pass it to upper layer. This process should be repeated untill no layer is able to advance new content to its upper layer. This also means that evry layer needs to buffer the data that require more read.
When data is received, it should not be buffered directly. Some protocols apply transformation on the content. So we should give them a chance to do this transformation, before appending the new result with the previous data from previous reads which are still not given to upper layer.
When a protocol decodes data, it may either return content, or protocol bytes that should go back to the peer without requiring intervention of application code. This will serv the needs of many protocols that require what they name as control messages. Most easy example, when a layer is sending regular heart beat messages and the other must reply back.
The abstraction should provide a way for a protocol layer to enter an initialization stage, during which upper layers and application layer should still not send any data untill the stage finishes. Example, if we layer Websocket and TLS, then when a connection is established, we need TLS layer to finish its handshake stage before giving the OK to the Websocket layer to start exchanging data. The application code itself does still not start anything untill Websocket finishes its handshake stage.

Use cases:

If you want to see a complete working case of this framework, you can look at Websocket solution in Push Framework repository at CodePlex:

http://pushframework.codeplex.com/SourceControl/latest (Go to version 3.0, then to WebsocketServer directory).

WebsocketProtocol is a library that implements Websocket Protocol (RFC 6455) using ProtocolFramework abstraction. And DemoServer is an example server that just echos what ever it receives. It uses WebsocketProtocol as protocol and I used it to stablize the protocol implementation thanks to the fuzzing client found in Autobahn Websocket test suite.

Thanks to ProtocolFramework, I could now easily switch between multiple protocols, layer protocols together, and even use multiple listening ports and each ports talks to a different category of clients that understand a specific stack of protocols:

MyServer theServer;
WebsocketProtocol webProtocol;

//Create a listening port using unsecured port.
ListenerOptions lOptions;
lOptions.pProtocol = &webProtocol;
theServer.createListener(10010, &lOptions);

//Create a second listening port, for secured clients:
SSLProtocol sslProtocol;
if (!sslProtocol.initializeAsServer("E:\\certificates\\server.crt",
                                    "E:\\certificates\\server.key",
                                     "pass"))
{
    cout << "ssl initialization failed"<< endl;
    return -1;
}
WebsocketProtocol webProtocol2;
webProtocol2.addLowerProtocolLayer(&sslProtocol);
ListenerOptions lOptions2;
lOptions2.pProtocol = &webProtocol2;
theServer.createListener(10010, &lOptions2);
theServer.start(true)

A C++ framework for Protocol development