In Quest for Requirements Engineering Oracles: Dependent variables and measurements for a (good) RE

For many years, researchers and practitioners have been proposing various methods and approaches to Requirements Engineering (RE). Those contributions remain, however, too often on the level of apodictic discussions without having proper knowledge about the practical problems they propagate to address, or how to measure the success of the contributions when applying them in practical contexts. While the scientific impact of research might not be threatened, the practical impact of the contributions is. In this research, we aimed at better understanding practically relevant variables in RE, how those variables relate to each other, and to what extent we can measure those variables. This should allow for the establishment of generalisable improvement goals, and the measurement of success of solution proposals.

We established a first empirical basis of dependent variables in RE and means for their measurement. We classified the variables according to their dimension (e.g. RE, company, SW project), their measurability, and their actionability. So far, we reveal 93 variables with 167 dependencies of which a large subset is measurable directly in RE while further variables remain unmeasurable or have too complex dependencies for reliable measurements. Far more important than the variables themselves, the results allow us to critically reflect on the direct implications for evidence-based research in the field of RE. In our paper, we discuss, inter alia, a variety of conclusions we can draw from our results. For example, we show a set of first improvement goals directly usable for evidence-based RE research such as “increase flexibility in the RE process”, to what extent such goals are measurable, and finally what implications we can draw, e.g., on the type of study we should aim at.

The paper is published at the 18th Intl. Conference on Evaluation and Assessment in Software Engineering (EASE 2014), the preprint version can be taken from here.