The value of assessing weights in multi-criteria portfolio decision analysis

Analytic efforts in support of portfolio decisions can be applied with varying levels of intensity. To gain insight about how to match the effort to the situation, we simulate a portfolio of potential projects and compare portfolio performance under a range of analytic strategies. Each project is scored with respect to several attributes in a linear additive value model. Projects are ranked in order of value per unit cost and funded until the budget is exhausted. Assuming these weights and scores are correct, and the funding decisions made this way are optimal, this process is a gold standard against which to compare other decision processes. In particular, a baseline process would fund projects essentially at random, and we may estimate the value added by various decision processes above this worst case as a percentage of the increase arising from the optimal process. We consider several stylized decision rules and combinations of them: using equal weights, picking one attribute at random, assessing weights from a single randomly selected stakeholder. Simulation results are then used to identify which conditions tend to make which types of analytic strategies valuable, and to identify useful hybrid strategies.