Featured post

Fixed Income Trading: New venues ( How many Fixed Income trading venues are there? )

How many Fixed Income trading venues are there?  [153]  A simple question came up recently in a conversation – how many new Fixed Income t...

Thursday, 16 July 2020

MQTT versus SBE : Fight!

Recently we have completed phase one of a full stack MQTT framework. Client simulator, server, data repository and associated analytics stack.

While working on this it's been interesting to see the different trade-offs involved in the protocols, which can be sourced to the different user communities and their requirements.

Electronic trading - overall latency is key
Multiple hardware implementations - x64, FPGA, SmartNIC
Multiple OS implementations - *nix, Windows, Solaris etc.
Multiple implementation languages - Java, C++, C, VHDL
Upgrades are frequent and need to be supported - no re-implementation to patch the protocol
Bandwidth use is generally only reduced where that is a side-effect of another optimisation, it's not a core deliverable

Internet-Of-Things - bandwidth usage is key and minimal client side resource constraints   
Multiple hardware implementations - Set-top box, custom chip, low power ARM 
Multiple OS implementations - QNX, RTOS, *nix, Windows, Solaris etc.
Multiple implementation languages - Java, C++, C, VHDL
Upgrades are infrequent and generally do not need to be supported 
Bandwidth use is minimised as it is a core deliverable

SBE 32-bit unsigned integer versus MQTT Variable Byte integer

An SBE 32-bit unsigned integer is exactly what the name suggests, 4 bytes used to store a value between o and 2^32-1

So, even if you are sending a value of one, that still requires 32 bits over-the-wire.

An MQTT Variable Byte integer is more complex in implementation. It is also unsigned, but the number of bytes used varies with the size of the number transmitted.





0 (0x00)

127 (0x7F)


128 (0x80, 0x01)

16,383 (0xFF, 0x7F)


16,384 (0x80, 0x80, 0x01)

2,097,151 (0xFF, 0xFF, 0x7F)


2,097,152 (0x80, 0x80, 0x80, 0x01)

268,435,455 (0xFF, 0xFF, 0xFF, 0x7F)

Extract from OASIS website

This means that the number of bytes used on the wire representation is linked to the value of the number transmitted. 

Depending on the distribution of the numbers transmitted you may see an improvement in bandwidth needed or not.  And since the Variable Byte Integer only uses 7 bits for data and 1 as a 'continuation bit' the maximum value transmitted using the Variable Byte Integer is a 28 bit integer rather than 32.

No comments:

Post a comment