a place to jot

Wow windows networking is.. not straight forward. For those of you who have never had the “joy” of working with them, I recommend you keep it that way.

For Windows I only really need the client netcode to work, which means I don’t really need high performance socket handling code. This is because the client will be sending/receiving packets at a set interval.

But silly me, always likes to look into high performance stuff and started looking at the Registered Input/Output (RIO) API. It sounds very fast and if I were going to run windows servers (why you would, I do not know) I’d definitely take a closer look at it. For Linux folk, think of it like io_uring… Great, now I kind of want to switch from epoll from the server… UGH, maybe later.

Anyways, I found a great resource for implementing various RIO UDP Servers and quickly cloned their sample code repository. However, after reviewing the code for a bit, I realized it was extremely complicated and not worth doing just for a client.

I then decided to go back to I/O Completion Ports (IOCP) which was my original intent for getting client code working. I got half way through implementing it when I realized, I don’t need all of this crap, and there’s a lot of it! The MSDN documentation on how exactly one should use IOCP is as clear as mud, so I ended up doing a lot of github searches but started to realize, I only really need a very simple select style polling loop for clients.

So, out goes IOCP.

Have you ever worked with Winsock2? It is such a cluster. There are a million different ways to do the same thing, all the sample code is very old and gross looking and it doesn’t actually give you a working server/client for UDP. You basically have to guess how to poll a socket. So, that’s what I did!

Here’s the (apparent??) flow:

1. Call WSAStartup because ???
2. Call ioctlsocket to set the socket to be non-blocking.
3. Create an event object with WSACreateEvent.
4. Call WSAEventSelect with the socket, the event object from WSACreateEvent and the events you want to select on ( FD_READ / FD_WRITE ).
5. Enter your “select” loop. (In my case the network thread while loop).
6. Call WSAWaitForMultipleEvents with a bunch of garbage windows-y things one of which is a timeout (optional).
7. Call WSAResetEvent because you have to, or the Wait call in step 6 will always immediately return.
8. Call WSAEnumNetworkEvents to fill out a WSANETWORKEVENTS object with the event type (FD_READ or FD_WRITE).
9. Got a FD_READ? Ok NOW you can call recvfrom (or WSARecvFrom) on the socket. I have no idea if it’s better or not to use the WSA form.
10. Die from typing WSA all those times, and making your code look like shit.

Is this process documented anywhere on MSDN? haha nope. Also you can apparently use things like WaitForSingleObject and CreateEvent or a bunch of other strange esoteric methods, but this is the only one I actually got working.

At last, I have a working Listen() method for Windows clients! (and servers, but I only use that for functional tests).

Here it is, in all it’s glory:

#if defined(_WIN32)
    void Listener::Listen()
    {
        auto NetEvent = WSACreateEvent();   
        if (NetEvent == nullptr)
        {
            fmt::print("Listen: Failed to WSACreateEvent: {}\n", WSAGetLastError());
            return;
        }
        auto Ret = WSAEventSelect(Socket.GetFD(), NetEvent, FD_READ | FD_WRITE);
        if (Ret != 0) 
        {
            fmt::print("Listen: Failed to WSAEventSelect: {}\n", WSAGetLastError());
            return;
        }

        while (Running.load())
        {
            // we only have 1 event
            const DWORD EventTotal = 1;
            WSAEVENT EventArray[WSA_MAXIMUM_WAIT_EVENTS] = {NetEvent};
            WSANETWORKEVENTS wsaEvents;
            // Wait for events
            auto Index = ::WSAWaitForMultipleEvents(EventTotal, EventArray, FALSE, TimeoutMs, FALSE);

            // Reset our event handle
            auto ResetResult = WSAResetEvent(NetEvent);
            if (ResetResult == 0) 
            {
                fmt::print("Listen: WSAResetEvent failed with error = {}\n", WSAGetLastError());
                return;
            }

            // Figure out what kind of event we have
            if (WSAEnumNetworkEvents(Socket.GetFD(), NetEvent, &wsaEvents) != 0)
            {
                fmt::printf("Listen: WSAEnumNetworkEvents failed.  Error code %08x", WSAGetLastError() );
                continue;
            }
            
            // Not a read? BALEETED, or well, go back to WSAWaitForMultipleEvents
            if (!(wsaEvents.lNetworkEvents & FD_READ))
            {
                continue;
            }
            
            // OK we have an inbound packet, process it.
            struct sockaddr_in ClientAddr;
            std::memset((char *)&ClientAddr, 0, sizeof(ClientAddr));

            std::array<unsigned char, PMO_BUFFER_SIZE> PacketBuffer{0};
            
            // finally call recvfrom on our socket
            auto PacketLen = Socket.Receive(ClientAddr, PacketBuffer.data(), BufferSize);
            if (PacketLen <= 0)
            {
                continue;
            }
            
            // Process our packet
            ProcessPacket(PacketBuffer, PacketLen, ClientAddr);
        }
    }
#else 
	// ... linux implementation ...
#endif

All other socket code code is in socket.h/socket.cpp.

I tried to keep the preprocessor flags/conditions to a minimum and prefer duplicating code over inlining tons of little preprocessor ifs. I find it SUPER hard to read code where every other line is a bunch of preprocessor statements.

And that’s it! I now have client/server comms working in windows.

References:

• Interesting looking io_uring paper I have yet to read.
• Another great looking io_uring resource.
• MSDN introduction post on RIO
• ServerFramework.com’s awesome RIO series