Using CodeRunner to build Middleware Protocol

Tue, October 23 2018

Using CodeRunner to build Middleware Protocol

After spending weeks crawling through all of my old code - and not finding my old DXDataset and DXMiddleware components, I decided, why not build it from scratch. I own (registered) the ports 2874 and 2875 with IANA years ago for my DXMessageBase protocol - which is how I developed my first middleware solution in 1998/1999. It of course was my first attempt to develop a middleware solution. Now, after having experience with MOM (Message Object Middleware), and ORB (Object Request Broker) and hands on with ASTA 3, kbmMW, ICS MidWare, RabitMQ, MotoMQ, and many more. Each of these present some cool features and concepts in moving data, objects, etc.

However, a project I am working on requires a middleware solution that offers more of a "traffic optimized" design. Of course developing my own protocol is extremely easy due to the design of Modern Pascal's CodeRunner engine. It handles all of the communications, threading and memory management. I just have to focus on the functionality. CodeRunner can take my code and scale it to tens (10s) of thousands of connects (aka concurrent scripts).

To make sure I have all my bases covered, I searched google and found a couple articles on building a middleware search from scratch. Unfortuanetly, these were too basic for me - so, I started coding. Since Modern Pascal provides an object for storing variables to a common memory region for a connect, then chaining (running) another pieces of code, which can access the common memory region. I elected to main a very small front-end firewall script. Thinking like fail2ban, the initial script simply tracks valid and invalid connections. Of course, I want to throw the hackers off, so every new connection is greated with a standard XML header.
   Session.Writeln('<?xml version="1.0" encoding="UTF-8"?>');

The goal of that line of code is to imply the protocol on the server speaks XML, which it does not. My thining is set them up for failure, and when they start throwing XML streams at the server, there invalid authentication count goes up - and then the system blocked them from even connecting to the server. Now, the fun is I could update IPTABLES to deny connections from their IP, I can write code that upon accepting the connection I just keep the socket handle open, but, I do not read from it - allowing the stack for this connection to fill and add confusion to the hacker. Or, I can simply chain to a sandbox script and let them feel like they are getting somewhere - without it really producing any form of burden on the Middleware Server.

If the client passes along a value "Product" authentication string, the firewall script will store the information in the common memory region and chain to the main middleware script. These are scripts, as I am not wasting the overhead in compiling them, I am letting CodeRunner JIT Compile (Just-in-time Compiler) the code as connections are received. Modern Pascal is capable of compiling and executing coding in under 100ms (even thousands of lines of code).

Defining Functionality

Even though the primary design is MOM (Message Oriented Middleware), there will be times that the server will be executing code for the client RPC (Remove Procedure Call), let alone serializing database records ORB (Object Request Broker). The workflow I have decided will be to focus on MOM, then RPC then ORB.  What does all this mean? Think of MOM as my day to day protocol implementation, without needing to know the actual structure. Think of RPC as the ability to centralize certain calculations or business rules without releasing them to the client world where someone could steal your design. And think of ORB as the true database structure from database to client. Any of these three could be a successful Middleware server to achieve my goal, however, I also plan on releasing the source so people can see (a) why we use CodeRunner to prototype a new protocol, and (b) see where they could extend the design/framework to meet their need(s).

The MOM Layer in my design, sends and receives data in null terminated batches. This means the 0x00 (#0) character is reserved to denote the end of a transaction or command. This allowed me to focus on a less verbose structure to optimize command and control over the network (Intranet and Internet).

Product Authentication

     Session.WriteNull('LOGIN'+userKey+#32+StrToHex(UUID)+SHA1(StrToHex(UUID)+softwarePassword);

This approach is sending a 5 letter prefix "LOGIN", followed by a software to software UserKey. Using #32 (spacebar) as the delimiter and a session UUID converted from 16 Bytes to 32 Hexadecimal characters. I append to those 32 characters the SHA1 of those 32 characters plus the software to software Password. Sounds pretty straight forward, however, if there is a misunderstanding by a third-party developer who decides to use this API - they will get one of 3 errors.

Error 501
Denotes that the first 5 characters received from the client were not LOGIN.

Error 502
Denotes that the software UserKey is not what we expect.

Error 503
Denotes that the hash of (session UUID plus the software password) does not match the calculation the server generated. The server is capable of handling a wide varaiety of hash algorithms, contact whomever is hosting the middleware software to see if they are using MD5, SHA1, SHA256.