Recently we have completed phase one of a full stack MQTT framework. Client simulator, server, data repository and associated analytics stack.
While working on this it's been interesting to see the different trade-offs involved in the protocols, which can be sourced to the different user communities and their requirements.
SBE - FIXP - SOFH
Electronic trading - overall latency is key
Multiple hardware implementations - x64, FPGA, SmartNIC
Multiple OS implementations - *nix, Windows, Solaris etc.
Multiple implementation languages - Java, C++, C, VHDL
Upgrades are frequent and need to be supported - no re-implementation to patch the protocol
Bandwidth use is generally only reduced where that is a side-effect of another optimisation, it's not a core deliverable
MQTT
Internet-Of-Things - bandwidth usage is key and minimal client side resource constraints
Multiple hardware implementations - Set-top box, custom chip, low power ARM
Multiple OS implementations - QNX, RTOS, *nix, Windows, Solaris etc.
Multiple implementation languages - Java, C++, C, VHDL
Upgrades are infrequent and generally do not need to be supported
Bandwidth use is minimised as it is a core deliverable
SBE 32-bit unsigned integer versus MQTT Variable Byte integer
An SBE 32-bit unsigned integer is exactly what the name suggests, 4 bytes used to store a value between o and 2^32-1
So, even if you are sending a value of one, that still requires 32 bits over-the-wire.
An MQTT Variable Byte integer is more complex in implementation. It is also unsigned, but the number of bytes used varies with the size of the number transmitted.
This means that the number of bytes used on the wire representation is linked to the value of the number transmitted.
Depending on the distribution of the numbers transmitted you may see an improvement in bandwidth needed or not. And since the Variable Byte Integer only uses 7 bits for data and 1 as a 'continuation bit' the maximum value transmitted using the Variable Byte Integer is a 28 bit integer rather than 32.
Aside from low level numeric types, SBE offers standard mapping to common datatypes such as dates, timestamps, monetary amounts, and so forth.
ReplyDeleteHi Don!
ReplyDeleteAbsolutely, it's fair to say that SBE has a wealth of datatypes and the ability to add more pretty easily - such as the 256-bit Integer we discussed in the High Performance Working Group a while back.
This blog post is really part of a series that I am writing but I have been tied up with the whole process and other work that has appeared.
It's fair to say that someone from an SBE-FIXP-SOFH background with FIX knowledge (ie - me!) has something of a learning curve when working with MQTT - bit-twiddling and bit-by-bit wire representations are something that are not mandatory for SBE-FIXP-SOFH. For sure, those sorts of low-level operations can assist when really tuning the engine for performance but are not needed in a basic implementation. Whereas they are needed for MQTT.
It's horses for courses, when I get the time I do want to write-up (at a high-level, without commercially sensitive details) a more formal comparative analysis.
Just one thing for now - the MQTT working group senior membership are all employees of IBM and Microsoft. Whereas the world of FIX is much broader in membership...