Stop measuring product development progress in dollars

In my advisory work with enterprise clients navigating major new-product introductions, I keep running into the same frustration: how best to measure progress. This is frustrating because clients readily agree with me that the only useful measures of progress are how much longer and what is left to do.

Yet a moment’s reflection reveals an uncomfortable truth: There really is no way to know; there are no viable metrics. No executive wants to hear this, of course, so managers tasked with developing competitive new products are under constant pressure. And sooner or later they all turn to the same measure of progress: costs incurred.

Why is this so frustrating? Because money invested, or sunk cost, is a poor measure of engineering progress. How so?

Dollars aren’t design inputs

At CIMdata, I continually struggle to keep clients focused on the “Big Three” inputs of new-product development: Materials, Information, and Energy.

Money is not mentioned anywhere, not even in Information, yet it is needed to acquire material, information, and energy. The only way to minimize the use of money—the cost of a product—is to first optimize the use of material, information, and energy for the entire lifecycle. Tracking project spend only measures financial effort, it reveals nothing about the quality of the results or whether the product will succeed in the market.

I am often frustrated by clients’ unwillingness to see this, to let go temporarily of their focus on transactions and to keep confusing the costs of product input with the substance of what goes into new products. In short, measurement is not substance, which means that these clients are misleading themselves. They should focus on what their engineers are achieving, the optimization of the resulting product design, and its use of materials and energy. This is more important than tracking costs during development. This is because a product’s lifecycle cost (i.e., the cost to produce and service it) is usually significantly larger than the cost of designing it.

When I run into this, I ask clients if they think they are watching the right metrics in their product development efforts; the immediate response is almost always: “Yes, of course … transactions. What else?” I take a step back and challenge the assumption: “Maybe you’re not focusing on the right things. The metric that truly matters is the progress your engineering team is making toward the development of viable, competitive intellectual assets that define an optimized product and its associated manufacturing and lifecycle support. That is way more important than the money in product development transactions.”

While they digest that insight, I point out that the “numbers” clients show me are invariably inaccurate, leading to bad decisions by top management and, even worse, to new products that perform poorly. Both can be very expensive, and unfortunately no one seems to have the complete picture.

What transactions actually measure

Users are adding up data about money spent in product-development efforts, but measuring these “costs” only captures effort, revealing little if anything, about results to date or helping to get a fix on expectations and outcomes. The only value in measuring transactions during product development is as a pointer to the growth of the ROI that must be recovered. Other than this, measuring costs reveals very little that is worth knowing.  Cost measurements can never really tell you if you have optimized the product in a way that will maximize your lifecycle ROI.

Relying on transaction costs can lead to bad decisions by leading executives into mismeasuring new-product development progress. (And ditto for even some line and department managers who should know better.) These financial measurements are fundamentally misleading because of the myriad arbitrary ways that “costs” reflect engineering time and effort.

This means the ways in which measurements are applied have no verifiable significance in determining the success of a new product or even in the progress of its development. When struggling for predictability, costs provide only hindsight. Because these costs are assigned arbitrarily, they offer neither useful comparisons nor consistency. Hence, overreliance on transactions causes problems in budgeting.

Why does this happen? Because every product development unit has dozens of different tools, each different from those used by similar units. And because the inputs and outputs of these tools are measured in different, if not inscrutable, ways, you must be aware of measurement masquerading as “progress.”

Two metrics that actually predict success

The most viable measurement of new-product development is the engineering time and resources consumed in bringing a new product to market. (Time wasted on dead-end efforts should not be ignored, if only to identify their causes.)

Understanding how close project managers believe they are to completion is far more useful than “knowing” how much money they have spent.

Why? Because money spent is too easily conflated with day-to-day efforts and “results.” In short, information generates money, but the reverse is not true, except at lending institutions, the only place where money is a physical resource.  

In short, there are no valid bases for cost comparisons of a new product until it launches, and its sales profile is known. Before that, there is no objective way to determine costs, benefits, and ROI (i.e., the product’s true lifecycle profitability). However, tracking time “spent” and anticipated time to product launch is extremely valuable for competitive comparisons.

This can only be done by staying in touch with the managers of the product development team and, above all, by making sure that they get all the materials, information, and energy they need.

I believe that these two metrics are far more important metrics than costs:

• How close a product-development team is to completion, and
• How thoroughly a product has been optimized for lifecycle profitability.

If top management insists on evaluating product developers’ progress, they should focus on these two metrics: They determine a new product’s ultimate revenue and ROI.

Never confuse measurement with physical inputs

In business, virtually all useful measurements are reported in terms of money. Measuring money spent, no matter how carefully done, is not needed until the end of the development process.

The cost of a new product is the sum of its components, materials, specifications, energy, and the engineering resources needed to bring them all together. Products are made with money, but they are not made of money.

As I keep advising clients, a cost measurement can never be a product constituent. A measurement is neither a physical new-product input nora substitute for one.  Thus I tell them, measuring “effort” is like measuring thoughts. Is there an incremental value? No. Could incremental value be allocated/assigned/attached? No.

Why executive focus is fixated on spend

Clients keep doing this because money transactions are supposedly easy to track and seem to offer viable means of comparison. Moreover, top management is always ROI-focused, so they continually demand that progress toward ROI goals be shown even though everyone knows those goals are shaky. Our usual caution: don’t confuse digital indications with physical inputs (or outcomes).

Why is money referred to with a dollarsign, or for that matter, any other currency symbol?  Because, in a very real sense, money is an indicator (and a subjective one at that, as I have shown); money spent has little relevance until a project is complete, all its costs known, and its lifecycle ROI is understood.

Evaluating early-stage project value is notoriously difficult due to two key realities:

• Incompatible Measurement Systems: The many inconsistent/incompatible ways in which effort and resources are measured
• Fluctuating Valuations: The constant changes in costs and in valuations of efforts and inputs (change is not only continuous, but it is also universal).

Focus on intellectual assets over cost and transactions

I continually remind clients that new products are built not with money but with things acquired with, and measured by, money and those measurements are shaky. In this context, money transactions are more than just distractions, they can become disruptions.

Executives monitoring new-product development should not zero in on costs and transactions until the end of the process. Only then can the new product’s pricing and its competitive standing in the marketplace be reliably known. In summary, in-process money tallies are at best incomplete, and thus inaccurate and invariably misleading.  

Product-related intellectual assets are a far better metric than money. Without effective product data management, there can be no competitive new products—no products optimized in a manner that will bring consistent, measurable, and maximized return.  And without a steady stream of viable new products, no enterprise can assure its long-term survival.

The post Stop measuring product development progress in dollars appeared first on Engineering.com.