I’ve just been reading “The blind men and the elephant: towards an empirical evaluation framework for software sustainability” - doi: 10.5334/jors.ao.

Comments on the reason I read this paper - kind of by accident?

I’m on the plane and the price for wifi is extortionate, so I’m not able to follow links and thoughts too much - this is okay for the purposes of dedicated study downtime, to be fair! Anyway - I had been looking at the CHAOSS guidelines, and come to the conclusion that the Elephant Factor was confusing and didn’t necessarily sound like a meaningful measurement. Looking at the citations in the end of this section of metric, I saw this paper and figured I should read it, especially since Caroline was one of the co-authors on the paper.

I printed it out, read it, and part way through realised it probably had nothing to do with the CHOASS Elephant Factor metric.

Thankfully I do have a copy of the CHAOSS metrics downloaded offline - reading through, I noticed this explanatory snippet:

Elephant Factor provides an easy-to-consume indication of the minimum number of companies performing a parameterized filter (i.e. 50%) of the work. The origin of the term “elephant factor” is not clearly delineated in the literature, though it may arise out of the general identification of software sustainability as a critical non-functional software requirements by Venters et al (2014).

I guess I skimmed that when I decided to read this paper, or possibly I didn’t quite fathom just how unrelated it was with the exception of worlds “elephant” and “software”?

Actual notes on the paper

It was an interesting read despite the fact it wasn’t what I thought it was, which was nice. It mostly looked at different ways people define sustainability (spoiler: many different ways), pointed out that metrics for sustainability are often wholly subjective, and seemed to focus on technical architecture as an important part of sustainability.

Reasons it may be useful for me:

  • it has lots more reading about sustainability measures
  • repeatedly states that software sustainability is a non-functional requirement, and points out that academic software often has haphazard construction and may be especially proof-of-concept-ey, and/or focused on nonfunctional requirements.
  • points out that software metrics can not always be measured in a quantifiable way - e.g. reliability.
  • Uses the parable of the bling men and the elephant (one feels the trunk (or maybe tail?) and asset it’s like a rope, another feels a different part of the elephant and makes different claims about the elephant) to remind us that we need to find metrics that genuinely assess what we want to assess clearly. Using this theme could be useful to point out vanity metrics and gamifiable metrics.
  • lots of nice bits I can quote. Some samples:

“The concept of sustainability goes beyond the software artifact itself”

“What measures and metrics are suitable to demonstrate software sustainability is an open research question”

“Selection of the appropriate methods is highly dependent on the context…”