Skip to content
kwharrison13.comEssays, books and notes by Kyle Harrison

Evidence-based Software Engineering

From kwharrison13.com · Library of Alexandria
Evidence-based Software Engineering
AuthorDerek M. Jones
Published2020
FormatBook
OriginalRead the original

TL;DR — Derek M. Jones’s free 2020 book collects the publicly available data on software development, analyzes it in R, and shows how little of the field’s folklore has evidence behind it.

Released by Knowledge Software, Ltd on November 8, 2020 under a CC BY-SA 4.0 license, the book is built entirely on public datasets. The introduction argues that, apart from 1970s US Air Force-funded work, very little software engineering research since about 1980 has been evidence-based: one cited review of 5,453 papers published between 1993 and 2002 found only 2% reporting experiments. Jones calls beliefs resting on old, tiny datasets “fossil theories,” and gives the claim that cloned code is fault-prone as an example later contradicted by research.

The first half arranges the data in seven chapters: an introduction; three on the forces behind software development (human cognition, the economics he calls “cognitive capitalism,” and the ecosystems software lives in); and three on outcomes (projects, reliability and source code). He warns that the data is too thin for joined-up arguments, so sections cluster around individual datasets.

The second half is a practical statistics course for developers who do not want the underlying mathematics: chapters on what data can show, probability, statistics, regression modeling, other techniques, experiments, data preparation and an overview of R, the language used throughout.

What it is: A self-published book by one author with a large reference list and the data and code released alongside it. It is careful about evidence, not peer-reviewed as a whole, and openly licensed.

Where this came from

454 pages. A copy is archived locally against link rot; the header links the original source.

Curated by Kyle Harrison. Organized by Claude. Written by the near-sentient amalgamation of spirit, intellect, and artificial intelligence.