While translating reliable.io’s endpoints to C++ I quickly came to the realization that I will need to properly implement the messaging system. Right now it’s kind of spread out all over the place (in the socket thread, the main loop, and inside of flecs systems). To make the system cleaner and easier to understand it makes sense to refactor the netcode and keep it all within the socket thread.
To do that, the Reliable Endpoints will be in the net namespace and created once we have a valid encryption key for a client. Here’s the new flow:
I’ll continue to use In/Out LockFreeQueues (LFQ) for passing messages between the main and socket threads. The major change comes from no longer allowing the main thread to send messages over the socket. Now it will have to build the Server/Client commands, use flatbuffers to serialize them, then pass the blob of serialized bytes over to the LFQ.
Once in the Socket Thread, the messages will be popped off the queue, then sent to the Reliable Endpoint for that user. It will take the serialized message, continuing to treat it as a blob of bytes, but now check if it needs to be fragmented (> 1024 bytes). If it does, each individual fragment will be encrypted with it’s own nonce (can’t be going around re-using nonces, that’s bad!). Each individual fragment (or packet) will then be serialized again using the MessageType of EncryptedData.
This ‘header’ message type of EncryptedData also contains the userid/time/sequence ids and other necessary bits of information needed for reliability. Basically I’m serializing these messages twice, once with the encrypted data, and once with the metadata needed to process it. The second time it’s serialized it will be sent over the Client/Server socketclass.
On the recieving side, we will only process it IFF (if and only if) it decrypts correctly given the provided UserId taken from the top level flatbuffers message!
This design will allow the game code to not care about reliability or how big a message will be, it just needs to serialize it’s messages and let the reliable endpoint and socket thread deal with fragmenting, encrypting, decrypting, and or reassembling!
Now that I have my design in place, I’m going to have to rewrite a bunch of stuff, see you in a week or two!
a place to jot 