If you’re a technical manager, you’ve probably heard of business intelligence tools, and they probably sound like something for the business and executive teams. In fact, they’re much more than that. You may also not realize how well-integrated BI tools already are in your existing environment, especially if you’re an Office365 subscriber. The examples in this story use the excellent but rather unfortunately-named Microsoft PowerBI, but the same results can be achieved with other tools such as Tableau or Spotfire.
The whole domain of so-called business intelligence is a misnomer. It ought to be called data-driven intelligence. Anybody who makes decisions needs to be equipped with data, not just business types. Or more accurately, your decisions can only be as good as your knowledge and your available data, so business intelligence is really just the trendy term for “making good decisions”. The way you make better decisions is by understanding your world better, and the way you gain that understanding is through data.
Modern software engineering leaders and managers run teams that generate tons of data. From what we’ve seen, however, most haven’t leveraged that data nearly as much as they could. Thanks to the current crop of BI tools, it’s become much easier to organize and visualize this data. Managers who jump on this get control of their data, and consequently their operation, and gain a significant competitive advantage.
BI for software engineering
Let’s look at a concrete example: a dashboard that a software engineering manager might create. The one below is adapted from one that Essilen Research helped build for one of our clients.
Looks like a lot of information, so let’s break it down:
- An area plot and donut chart of work distribution in dollars. Both of these plots indicate the time spent on features versus the time spent on bugs. Bugs are further broken down to: new defects, debt-related or regressions. How did we do this? This particular team was an Agile team that used JIRA to track work. Each story in their sprint had hours logged. That made it easy to export the data to PowerBI, where we constructed the plots.
- Component map chart. The rectangles on the top right represent different components in the system. The size of the rectangle is the size of the component (lines of code, for example). Note the colors of the components range from green to red. The more red, the more bugs the component generates. On the bottom right is a bright red component in a small rectangle. That indicates that this small component is generating a lot of bugs and is a candidate for re-factoring or at least extra attention. This data is easily obtained by mining the bug database.
- Weekly feature rate. Feature rate is a great metric to track productivity. Think of your software operation as a manufacturing operation where the product is features. There are many factors affecting feature rate, so we’ll save that for another story.
- Spend and debt ratio. The big numbers in the middle of the dashboard represent the R&D dollars spent in the time period. Debt ratio is simply the fraction of time spent on fixing technical debt versus developing new features. This is another important efficiency metric: it tracks how much business utility you’re getting out of your team.
Some of these metrics may seem exotic. For example, many engineering managers don’t think of their efficiency in actual dollars. But we believe strongly that data is only compelling when it’s expressed in terms people can understand, and that’s usually dollars.
Remember, software companies are already generating this data all the time. The source control software, program management software and bug software systems alone can be used to generate the above visuals. There is no need to create new data streams through weekly reports or otherwise.
Once you are able to use the data to express the right metric, anecdotal problems in your team suddenly become quantifiable. For example, engineers might complain that they are spending too much time “fixing bad code,” but a product manager might be pushing for more feature work. A dashboard like the one above allows you to actually see where the time is going. Engineering and product managers might agree, for example, that a .250 debt ratio is sustainable. That way, engineering time can be partitioned accordingly. Going from qualitative “feels” to real data is a powerful transition. It’s the key to making great decisions.
Aside from the rich visual capabilities of BI tools, where they really shine is in their ability to mash-up data from multiple sources. PowerBI can take data from just about any source, including plug-ins for GitHub, Excel, JIRA, even social media feeds. Why is this so important? Your organization almost certainly stores data across multiple sources and intelligence is gained when you can create visuals that aggregate across those sources. Example: mix sales data with engineering data and you could create a feature-level return on investment (ROI) metric: the ratio of value per feature vs development cost of that feature. Wouldn’t you like to know where your engineering resources are used to actually generate revenue?
BI for technical management
Creating dashboards like the one we showed above is also effective as a management tool. Once you have actionable data, you can set quantitative goals. And once you have goals, you can share them broadly so people can have a sense of how the team is doing. For example, the customer support team might have a goal of reducing support calls in a particular problematic feature by 15%. The engineering team gets involved to help address the feature shortcomings. A dashboard of support tickets is then shared on the web and even on TV’s in the office. Suddenly everyone is getting real-time feedback on how their efforts are working. You automatically create engagement and alignment between groups.
Finally, data driven team performance management can be very powerful for managers. Consider the screen-shot below of real data from a client’s recent performance review cycle.
Each employee has an overall “performance rating” (solid, strong, or top), but is also rated in seven categories ranging from hard skills on the left (analytical, technical, productivity) to soft skills on the right (decision making, creativity, communication and relationships). When the data is presented visually you can see that this group has most of its high scores (gold color) on the left hand columns, meaning this group has good hard skills but needs to build more soft skills. Now you know where to focus your training.
Again, this is an example of a team that was already producing this data but had never visualized it in a way that made it clear to its management.
“You make what you measure” is a phrase that’s often repeated in technology teams and it’s absolutely true. Maybe you don’t want to use the specific example metrics we proposed, and have ideas for your own. The key insight is that BI tools are making it incredibly easy to create truly useful metrics of whatever you believe are your key engineering or business priorities. And once you measure it, you can make it.