DevOps

The below picture ("Worked fine in Dev, Ops problem now) is one that I am sure many readers will have seen. 

During a conversation recently this topic came up - what is DevOps and what, if anything, is new and unusual?

Let's roll back a few years and look at how many firms in financial services operated.  Rigid silo based approaches, typically split by asset class.  Hierarchical management structures with considerable process overhead and a plan/build/operate structure. Or "run-the-bank" (RTB) and "change-the-bank" (CTB) if the firm followed that model.  Incentive structures often very narrowly aligned to actual corporate performance and more often linked to micro-goals rather than a holistic deliverable.

One consequence of this model was the famine and feast approach to project portfolio resource allocation - no money for a project that has delivered and instead pour money into failing projects because to admit failure is a career limiting move.

 

Or, in another way of viewing this - CTB would always want to deliver their projects to RTB as quickly as possible such that any issues became production outages which can then be blamed upon the failure of RTB to do their job properly.

 

The cultural problem of these older models can be encompassed in the picture above and a couple of phrases I have heard many times

• If it compiles, release it
• Cowboy developers 

These are symptoms of an often engrained cultural value set that fails to see the holistic nature of the operation of the firm - a client who receives poor service from a bank will not care in the slightest if the issue is within this division or that division of the bank.  The client sees a bank fail to deliver on their product offering and so will probably trade away.

 

In this older model of operation, many systems were what are now viewed as old fashioned batch job based processes.  So a system would take a finite amount of data, process it and generate output.  No real-time systems, just input==>process==>output.

 

One of the pleasant side effects of the Batch processing paradigm was that it allowed for monitoring by way of measuring effect - if a batch job for 125,000 accounts should create 125,000 lines of output in a file then simply measure the number of lines in the output file.  If not equal to 125,000 there's a problem. Now simply write up an operations manual to say what to do if (simplified) there are 0 lines, 0<n<125,000 or >125,000 lines

Digital Transformation, web 2.0, real-time delivery of application value and a true transition to real-time systems means that this sort of cause/effect based monitoring is no longer fit for purpose.

A typical use case - a distributed Java application in London processes large data feeds from multiple sources in real time and sends a stream of messages to a Singapore based hardware device that then fans out to multiple client systems.  The client systems then publish out to HTML5 applications using a web friendly AJAX protocol.  The Singapore user community complain that the system is running slow.  Where do you start looking?

This is where the cultural change needs to come into play - rather than the old model of a development group throwing code over the wall to an operations team that keeps the lights on, there has to be a partnership of common purpose.

 

That's where DevOps comes into play.  Unify the purpose of the Development and Operations groups and crucially allow for feedback loops to develop through the use of software that monitors the entire landscape in real-time on a functional and non-functional level.

 

This is a field that has exploded over the last few years with IPOs for Splunk and New Relic and I am sure we will see more from AppDynamics and others.  The market for this type of software has become increasingly crowded over the last few years with big players such as Tibco Loglogic and financial service niche firms such as ITRS and Velocimetrics all offering credible products.

 

The real challenge is not to merely implement the software but to follow through on the cultural shifts needed to maximise the value of the DevOps mindset.