Boost Signal, Reject Noise with Improved Observability

Focusing on what’s important … is important. IT Heroes have several jobs that rely very strongly on “signal boosting” – focusing on what’s important, investing thought and effort there, while rejecting “noise” (i.e., distractions).

These centers of effort include:

Observability

Ensuring that infrastructure monitoring, process management, project management, software development and other activities are consistently showing and sharing the right information for making technical and business decisions: calling the important metrics to deciders’ attention in consumable, actionable ways.

IT folks will, of course, already know that observability (short form: “o11y,” as formalized by distributed systems expert Cindy Sridharan in this 2017 tweet) is a control theory term, meaning that the state of a system can be inferred, black-box style, from its outputs. And the term has indeed been adopted to refer to the monitoring of complex, distributed, and ephemeral applications.

A more general, more easily consumable definition of “observability,” however, might go like this: “To make decisions, people need relevant information, presented at the right time, in forms mere mortals can understand. Observability is the characteristic of systems to reveal and explain themselves in this way: actionably, with empathy.”

Benefits management

Using a disciplined, mature process to discover, encode, negotiate about, measure, and optimize around your organization’s shared understanding of ROI expected from IT projects.

In a sense, all IT project management has, as its ultimate goal, upside maximization. It’s all too easy to fall into the habit of executing tactically and forgetting about strategy: delivering on a backlog at the right velocity across a set of sprints, while ceasing to wonder whether the features you’re delivering are actually useful (and being used). Sure, project managers ensure production of deliverables. But stakeholders pass ultimate judgment on the benefits provided by those deliverables, not the deliverables themselves.

IT Heroes will already have grasped that these two things are connected. Understanding real-world benefits and delivering them (not just features) is how IT makes itself strategic. Implementing observability (not just monitoring and logging and reporting) is how IT helps make sure that they, and the whole business, are keeping their collective eyes on the right balls.

Upsides for IT Heroes

If this sounds like hand-waving to you, think again. A report by Project Management Institute, based on several thousand surveys of IT leaders, worked out that projects lacking mature benefits management wasted an average of $112 million USD on each $1 billion USD invested. That’s more than 10% of spending just left on the table as unrealized upside.

Predictably, big fails from little acorns grow. According to PMI survey results:

Many projects begin without explicitly defining goals and business intent. Only 41% of responding organizations always do so.

Many projects fail to differentiate at all between concrete deliverables and benefits. Only 15% of responding orgs always make such distinction.

Many projects fail to differentiate between (or identify) shortand longer-term upsides. Only 11% of respondents always differentiate.

Maybe most telling: Benefits (as opposed to deliverables) are very rarely considered when planning projects for customers. (It’s done more often for internal projects, where obvious skin is in the game, and where responsibility for requirements-setting can’t be fobbed off on the customer.)

Conversely, according to the PMI report, when benefits are aligned with strategic goals, 80 percent more projects meet or exceed forecasted ROI. (I leave the cynical interpretation of this statistic as an exercise for the interested students.)

While there’s a whole science involved in determining and encoding project benefits, IT Heroes would do well to absorb a pair of key ideas:

A reasonable formula for “benefit” is “thing we truly value and can measure now,” increased by a meaningful percentage.

“Delighting customers” is not a benefit. “Effecting net revenue growth of 25%, year over year” is clearly a benefit, but may lie beyond the scope of IT initiatives alone to enable. The sweet spot may be asking Sales “So – what can IT do for you that might help you grow revenue by 25% over the next year?” The result of that conversation, suitably simplified, qualified, and encoded, can be one or more benefits, leading from © 2019 OPSVIEW. ALL RIGHTS RESERVED. 2 opsview.com deliverables, that everybody can agree on. 

Basic impact vs. difficulty analysis can reveal low-hanging fruit.

Imagine a Cartesian graph with ‘Impact’ on the positive X axis and ‘Difficulty’ on the positive Y axis. Things that are hard to do and won’t move the bar end up in the upper left. Stuff that’s easy to do, but also very impactful get sorted to the lower right. You’re welcome. 

What about O11y? (How Does this Connect with Monitoring?)

As noted above, “observability” is a property of systems. But systems are built by people, and people have empathy and pride in work, and want their creations to be understood, so they build observability in. And so should you. 

Enterprise monitoring is a tool for imposing observability on black-box apps and infrastructure. Part of the reason IT teams like it is that it plugs and plays: bringing to the table a certain amount of best practice about what’s being observed (e.g., the most important tuning metrics for MySQL clusters). But that’s not all it can do. The best enterprise monitoring products are highly customizable down to a very granular level, making it relatively easy to create and package new monitoring capabilities for consumption by your IT organization, or by endusers/stakeholders in your applications. That leads us to o11y take-away #1:

Observability is part of any IT deliverable.

Solve the metrics problem for your org, your colleagues, and your customers.

Take-away #2 climbs the value-chain, leveraging the fact that sophisticated enterprise monitoring offers numerous tools to help people visualize what’s really important to them, and not get distracted by noise. The point here is empathy: making your stuff explain itself in consumable, actionable ways, with the goal of empowering users, not stressing them out. Concretely:

Dashboards and BSM are parts of any IT deliverable.

Explore the facilities of IT monitoring that let you create and share custom views of infrastructure, clearly labeled and conveying simple messages, while suppressing unnecessary information that’s only relevant to IT. Your business users don’t need to see red boxes because one webserver in a tier of nine has gone South. They need to see green across the board, because if you’re doing your job right, the website is up and performing just fine with eight remaining servers online.