GSAS Talk: Pragmatic Approach to Architecture Metrics – Part 1

Share This Post

In their thought-provoking presentation titled “Pragmatic Approach to Architecture Metrics” at GSAS’22 organized by Apiumhub, Sonya Natanzon, and Vlad Khononov delivered valuable insights. They offered the audience ample food for thought, emphasizing the importance of considering relevant metrics and adopting a pragmatic approach shed light on understanding the objectives of metrics and the need to make informed decisions about what to measure.

Pragmatic Approach to Architecture Metrics

The importance of architecture metrics

With metrics, we start thinking in terms of …

What gets measured, gets managed -Peter Drucker

But it may be more appropriate to rephrase it as …

What gets measured, gets managed — even when it’s pointless to measure and manage it, and even if it harms the purpose of the organization to do so

-Simon Caulkin on V.F. Ridgeway’s paper “Dysfunctional Consequences of Performance Measurements”, 1956

Not everything that we can measure necessarily holds significant importance, and conversely, there are crucial elements that defy easy measurement. The key lies in striking a balance between measurable indicators and a comprehensive understanding of the broader purpose and objectives of the organization

What do we want to measure?

We want to know if software architecture is successful

Everyone wants quantitative metrics to prove success

Software architecture is nebulous and difficult to measure

Ralph Johnson aptly states, “Architecture is about the important stuff. Whatever that is.” 

The purpose of architecture lies in its alignment with the business goals, emphasizing that what truly matters for architecture is what effectively supports those goals

  Apiumhub becomes a Future Travel Summit collaborator

Identify what architectural qualities are important to support the business goals

See if you can measure them. These are your architecture metrics

(Successful) Product Evolution

The journey of software architecture involves Inception, Evolution, Market Adoption Expansion, and Scaling. It is crucial to recognize that Evolution and Scale represent distinct dimensions of change. 

The success of software architecture hinges upon its adaptability to meet evolving business requirements. Consequently, we assess the capacity of architecture to embrace change through various metrics. It is important to acknowledge that the strategies that brought us to our current state may not suffice for future growth. Metrics should align with business progression rather than being bound by architectural considerations, necessitating their continual evolution.

But be very careful about metrics, because…

When a measure becomes a target, it ceases to be a good measure. – Goodhart’s Law

For example:

1. Code Coverage

2. Bugs found

3. Deployment frequency

4. Crash rate

Some good metrics to take into consideration: 

Change Frequency

How often can we make meaningful changes?

Change Effectiveness

Did the change bring value to the users?

Change Cost

What does it take to make a change?

From Accelerate book

Complexity

As complexity escalates, so does the cost associated with change. The cost of change is intrinsically linked to the complexity of the modifications required. Consequently, our goal is to comprehend and evaluate the complexity of architecture. Complexity, as Geoffrey West aptly puts it, “You know it when you see it”.

Complexity in the Cynefin Model

• It is characterized by the relationship between cause and effect

  IoT solutions world congress in Barcelona

• How predictable is the outcome of an action?

• In complex situations, the outcome can only be determined through empirical experimentation.

For example:

• What’s going to happen if I change this line of code?

• A change breaking unrelated parts of the system

• Not knowing what modules are affected by new business requirements

What Causes Complexity?

  • Complexity arises from the interplay of system design and our cognitive capacities. It is influenced by the intricacies embedded within the system’s structure and behavior, as well as our ability to comprehend and navigate its complexities using our cognitive faculties.

• Estimated capacity for processing information in 1950s: 7 +/- 2

Miller, G. A. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343–352

• Estimated capacity for processing information in 2000s: 4 +/- 1

Cowan, Nelson. “The magical number 4 in short-term memory: A reconsideration of mental storage capacity.” Behavioral and brain sciences 24.1 (2001): 87-114.

Evaluating Complexity

Metrics play a crucial role in addressing complexity within systems, enabling us to quantify and assess the intricate aspects of system design and cognitive abilities.

  • Abstractness Metric

The ratio of the number of abstract classes and interfaces in the analyzed package to the total number of classes in the analyzed package. (Wikipedia)

Abstractness (A) = number of abstract types/number of concrete types

  • Instability Metric

The ratio of efferent coupling (Ce) to total coupling (Ce + Ca).

This metric is an indicator of the package’s resilience to change.

I=0 indicates a completely stable package

I=1 indicates a completely unstable package.

  Interview With Michael Feathers, GSAS22 speaker

Instability (I) = no. of types you depend on / (no. of types you depend on + no. of types depend  on you)

  • Distance From the Main Sequence (D)

The perpendicular distance of a package from the idealized line A + I = 1. This metric is an indicator of the package’s balance between abstractness and stability. A package squarely on the main sequence is optimally balanced concerning its abstractness and stability. Ideal packages are either completely abstract and stable (I=0, A=1) or completely concrete and unstable (I=1, A=0).

D=|A+I-1|

COST OF CHANGE IS PROPORTIONAL TO THE DISTANCE BETWEEN COMPONENTS

The purpose

The goal of evaluating complexity is to identify design decisions that contribute to an increased cognitive load, ultimately leading to higher levels of complexity.

In this first part, we have explored the importance of metrics and evaluated complexity within a software architecture. However, there is much more to uncover. Stay tuned for the second part, where we will delve into metrics for success and the concept of integration strength. Join us as we continue this insightful journey towards a pragmatic approach to architecture metrics.

Author

  • Oriol Saludes

    Experienced Full Stack Engineer with a demonstrated history of working in the information technology and services industry. Skilled in PHP, Spring Boot, Java, Kotlin, Domain-Driven Design (DDD), TDD and Front-end Development. Strong engineering professional with a Engineer's degree focused in Computer Engineering from Universitat Oberta de Catalunya (UOC).

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Subscribe To Our Newsletter

Get updates from our latest tech findings

Have a challenging project?

We Can Work On It Together

apiumhub software development projects barcelona
Secured By miniOrange