The Need for Measuring Performance

As engineering managers, you are no stranger to metrics. From lines of code to uptime, the software development industry is almost too obsessed with numbers. But when it comes to measuring the performance of an engineering team, engineering managers often get stuck.

There are two types of people, those that don't believe in quantitive metrics and those who do. I belong to the second group. This does not mean I don't look at qualitative feedback but quantitive metrics can tell us a lot of stories and paint a great picture if you know what to measure in the first place.

In this article, I will talk about how you could think about what metrics to measure, based on the unique situation of your team and what steps to take in not only measuring but improving the performance of your engineering team. Because the ultimate reason why you want to measure it in the first place is so you could improve on it. Just like the saying goes, you can't improve what you don't measure.

Types of Metrics

There are two types of metrics you should look at when it comes to measuring the success of a software engineering team: qualitative metrics and quantitive metrics.

Qualitative Metrics

The first type of metrics is qualitative metrics. Usually, this can be achieved by talking to people - employees and customers.

Many software companies use NPS (Net Promoter Score) to measure customer satisfaction. Looking at customer-driven metrics ensures your software engineering team is delivering value to business and customers.

Under qualitative metrics, you may also want to get 360° feedback on individual team members to evaluate their performance.

When measuring the performance of a team qualitatively, it's important to pay attention to low performers within the team. Like the saying goes, one rotten apple spoils the whole barrel. Although it may just be one single person in a team of ten, that one person can have a significant impact on the performance and morale of the entire team if they are low performing. Being able to identify low-performing engineers as early as possible comes from experience.

There are multiple reasons why they may be performing poorly. Some of them are:

• Not engaged with work (bored, not interested in tech, not interested in company's goals)
• Other personal matters affecting them
• Not feeling like they belong in their team
• Do not have the required competency

After identifying the root cause, the software engineering manager can then come up with an appropriate plan.

Possible actions are:

• Move to a different team or project
• Give them time off to recharge
• Provide appropriate training/resources to up-skill them
• Pair them with other more senior engineers
• Provide them with a smaller scope of work

You will then observe the same feedback loop and metrics to keep track of progress. If none of the above works after a period of time, it is time to consider letting them go.

The most rewarding part about being a manager (at least for me) is to play a part in people's career growth through coaching and empowering them. I've found that the best way to keep people motivated and engaged at work is to give them training and professional development and stretch assignments.

However, it's not all fun and games. The difficulty of the job as a manager involves managing people when they're not performing up to the expectation. Continuous feedback is important to make sure that you are providing support and guidance to your people and they know where they stand.

Timely and actionable feedback is important and you should not be afraid to have tough conversations with them. You are not doing your job as a manager if you are sugarcoating performance or behavioural issues.

This is just one example of how important qualitative feedback is in identifying issues and working through them effectively.

Quantitive Metrics

The second type of metric is quantitive metrics. They can be represented numerically and obtained scientifically.

Examples of quantitive metrics that can be used to measure/manage quality, delivery, and productivity in software product development are:

Time to delivery
Cycle Time
Lead Time

Quality
Defect rate
Percentage of test coverage

Uptime (reliability)
Latency
Time to recovery
Incident detection time (Mean Time To Detection)
Incident recovery time (Mean Time To Repair)

Speed of delivery
Average age of closed bugs

Team velocity
Number of releases
Number of story points completed
Rate of story completion

As I've written in my book, The Engineering Manager's How-To Guide, it's important to measure what matters and use metrics that relate back to the business goals. Otherwise, software engineering teams become too distant from the why of the business.

How To Get Started

The following are recommended steps in putting together metrics for your engineering team.
Start with the goal - an example of your goal might be to deliver reliable features at an increased speed.

• Think about what metrics you want to measure that are aligned with the goal and how you want to measure them. Once you have defined the metrics that are aligned with the goal you have for your engineering team, it's important to monitor them and stay close to them. Without monitoring and keeping track of them, it's as good as not having any metrics at all.
• Assign people who will be the owners of the metrics and work with them. These people could be engineering managers, staff engineers, or software engineers who are passionate about particular metrics.
• Put together tools and mechanisms. Trust but verify. Automation and alerting, Slack, If This Then That (ITTT) tool to set up smart alerting.
• Set a cadence for reviewing metrics, e.g., forthright meetings for quantitative metrics and quarterly review of qualitative feedback.
• Keep monitoring, reviewing, and improving!