30 November 2022

Metrics a CTO should be monitoring

Metrics a CTO should be monitoring Nov 30, 2022  Brody, Daniel  COPYRIGHT 2022 YOM SHORE VENTURES LLC

Metrics a CTO should be monitoring

As a Business First CTO, there are several sides to the KPIs a CTO should be monitoring. In addition to the items below, a good CTO should look at the business KPIs in Sales and marketing to improve the product, lower the cost of customer acquisition and improve retention.

Technology Produced

The primary metrics are how quickly the organization can produce new features and solve problems (velocity), the frequency and severity of bugs (quality), and the cost of running the organization and delivering services. The CTO’s responsibility is to find the appropriate balance at the proper time. For example, a consumer social media product will have different constraints than an enterprise medical product. Similarly, a weekend warrior startup has an entirely different balance than an established company.

The People

Creating and managing technology is a human endeavor, and a product is only as good as the people and process that produces it.

You are raising some pretty interesting questions. For example, if you work in a VC-fueled startup in a highly competitive space… You may reach a point where the only way to deliver your product is to burn out your employees. Or, in a more favorable position, you’ve had a very successful quarter, and you’re trying to evaluate the most effective incentives and rewards for your team. Every person in your group will have significant personal tragedies and victories and different degrees and kinds of motivations.

The metrics here are soft, but you can check in periodically. How’s it going? What’s working well right now? What can we help you with?

Technology Consumed

As the CTO, one of the most common choices is to build or buy – and you will be surrounded by people eager to push and pull you in opposite directions.

When picking an external technology to bring into your product, you should understand what it will cost, who it will impact, and how much time it will save.

One way to quantify this is through defects:

  • How often do they happen?
  • What parts of your product cause the most trouble?
  • Who wastes the most time chasing bugs?

If most of your time and energy is spent fighting with an outside piece of software, it might be time to bring it in-house. Conversely, if you have something in-house, that’s nothing but trouble… It might be time to outsource.

In any case, remember that the person (or people) who are most plagued by defects are almost always the most motivated to work hard at solving the problem.

All Things Considered

There’s no limit to the number of things that can be measured, so the most important task you can do up front is to define measurable goals: “We should halve the time between a bug report being filed, and a fix being released.” “We should scale to 10x our current transactions to keep up with sales forecasts.” “We should be able to replace a file server with zero downtime for our customers.”

Goals that can’t be quantified will invariably end in misery and confusion.