78
9 ways to fail with KPIs
9 WAYS TO FAIL WITH KPIS
Supporting Agile Adoption (Bjarte Bogsnes, Claudia Melo, Esther Derby, Kati Vilkki, Michael Sahota)
!
Organizations often use Key Performance Indicators (KPI’s) and metrics in dysfunctional ways, which result into gaming, reduced performance,
undesired behaviors and focusing on unimportant things. The most common traps we have seen organization and managers falling into are listed
below:
TRAP COUNTER MEASURE / WHAT TO DO INSTEAD!
1) Losing sight of purpose: The real purpose of the KPI
is forgotten. Improving the KPI becomes the
purpose and the perspective becomes too
mechanical and narrow.
Purpose: Always connect and communicate the purpose together with the KPI,
because it encourages people not only to understand, but also question the goals
behind the KPI.
Actions: Identify what actions that are being taken to reach the goal / meet the
purpose.
2) KPI becomes a target: KPI’s and metrics are seen as
only thing that matters.
Challenge KPI: Systematically challenge the measurement, both the KPI and the value.
When targets are associated with KPI, the more absolute the targets are, the more
you need to challenge the measurement. For example:
! What is measurement not picking up?
! Is this KPI really taking us forward the goal/purpose?
3) Measuring the wrong thing: organizations measure
only what is easy for them to measure. If something
can’t be measured (e.g. productivity in SW
development), other metrics are used to replace it
(e.g. lines of code)
Do not measure things that do not matter: Einstein said: “Not everything that can be
counted counts; not everything that counts can be counted.” Measuring the wrong
thing can be more harmful than not measuring anything, especially if there is a target
attached to the KPI. If you can’t measure what you really want to measure, drop the
KPI.
Focus on the purpose, actions, and observations
! What does success look like?
! How do we know if we have succeeded?
4) Lagging Indicators: organizations use only/ mostly
lagging indicators; we are only measuring the past,
not present and we can’t take corrective action
based on them.
Leading Indicators: Use leading indicators that are early indicators of what will move
us towards our goal. (e.g. measure queue length for goal of reducing cycle time). Ask
“what actions can we take based on this KPI? Who will act?”?
5) Targets for all KPI’s: When there is always a target
connected to a KPI it is likely we have returned to
management by numbers and are allowing KPI’s to
divert us from actual goals.
Tracking Trends: Use KPIs as data to improve the system, and look at trends rather
than just one point or an absolute number. “If you have a stable system, then there is
no use to specify a goal. You will get whatever the system will deliver. A goal beyond
the capability of the system will not be reached. If you have not a stable system, then
there is again no point in setting a goal. There is no way to know what the system will
produce: it has no capability.” - W. Edwards Deming
6) Same KPI for all levels. Cascading targets and KPI’s Translate KPI: Translate the targets and KPI’s so people define their own KPI’s for
improvement.
! What do we need to focus on to contribute the bigger goal?
! How do we know we have succeeded / contributed?”
7) Same KPIs for all phases of product life cycle. Using
the same KPIs to manage products in early market
and mainstream
Life-cycle sensitive KPI’s: Understand what KPIs are most relevant at each phase of
product life-cycle.
8) Aggregating metrics to bigger organizational unit.
E.g. calculating averages over teams.
Look at Details: Ask “who needs to take action to fix this” and measure at that level;
do not aggregate upwards. Use KPI’s as indicators to GO & SEE and look deeper.
9) Relying on KPIs to drive people’s motivation. E.g.
Establishing KPIs with aggressive targets hoping to
create a challenge for people.
KPIs to support intrinsic motivation. Engaged people are driven by intrinsic
motivation: doing something purposeful, having autonomy and striving for mastery.
KPIs can provide direction and feedback, but not motivation . However, allow people
to define success criteria for themselves.
!
Further reading:
Daniel H. Pink (2009). Drive: The Surprising Truth About What Motivates Us. Riverhead Hardcover.
Pat Kua (2013): An Appropriate Use of Metrics.
Robert D. Austin (1996). Measuring and Managing Performance in Organizations. Dorset House Publishing.
1. ⽬的を⾒失う：KPIの真の⽬的が忘れ去られている。KPIを改善することが⽬
的になり、視点が機械的で狭くなりすぎてしまう。
2. KPIが⽬標になる：KPIや指標だけが重要と考えてしまう。
3. 間違ったものを測定する：組織は、⾃分たちにとって測定しやすいものだけを
測定する。測定できないもの（例：ソフトウェア開発の⽣産性）は、他の測定
基準で代⽤する（例：コード⾏数など）。
4. 遅⾏指標：組織は遅⾏指標のみ／ほとんどを使⽤している。現在ではなく過去
を測定しているだけであり、それに基づいて是正措置を取ることはできない。
5. すべてのKPIに⽬標を設定する：KPIに常に⽬標が設定されている場合、私た
ちは数字による管理に戻り、KPIが実際の⽬標から⽬をそらすことを許してい
る可能性があります。
6. すべてのレベルで同じKPI：カスケードした⽬標とKPIを使っている
7. 製品ライフサイクルの全フェーズで同じKPIを使⽤する：初期市場と主⼒製品
で同じKPIを使⽤して製品を管理すること。
8. より⼤きな組織単位に指標を集計する：（例：チームごとの平均を計算する）
9. KPIに依存することで、⼈々のモチベーションを⾼める：例）積極的な⽬標値
を設定し、チャレンジ精神旺盛な⼈材を育成する。
こういった罠にハマらないよう注意したい。
（参考）Bjarte Bogsnes, Claudia Melo, Esther Derby, Kati Vilkki, Michael Sahota. “9 WAYS TO FAIL WITH KPIS - Agile Alliance”.
Agile Alliance, https://www.agilealliance.org/wp-content/uploads/2018/07/How-to-use-KPI_June2014_pretty.pdf, (参照 2023-1-10)
visional_engineering_and_design
1000ch
nagiss
lemon
makotu
kochizufan
konifar
miyashino
nishiharatsubasa
ytaisei
line_developers
unclejoe
nasa_desu
zenorocha
mclloyd
colly
destraynor
kastner
paulrobertlloyd
mraible
chriscoyier
jonyablonski
lara
ufuk