I have been a fan of trying to show ROI for
automation in a way that is simple enough to understand easily, and provides a
way of showing the benefit of automation compared to its cost.
I have been developing a spreadsheet with sample
calculations (sparked off initially by Molly Mahai, including an example from
Mohacsi & Beer's chapter in the new book, and some other people have also
been influential - thanks). I have sent this spreadsheet out to around 300
people, including most who have attended my automation tutorials.
The problem with showing ROI is that it's hard to
quantify some of the important factors, so I have focused on showing ROI using
what is the most straight-forward to quantify - people's effort and/or time.
This can be converted into money, using some kind of salary cost, if desired,
and either effort or money can then be plugged into the ROI calculation =
(benefit - cost) / cost.
So basically, this is showing how a set of tests
requires less human effort when those tests are automated than would be
required if those same tests were run manually.
This is great, right? We have a clear business
case showing savings from automation than are greater than the cost of
developing the automation, so our managers should be happy.
Recently, however, I have been wondering whether
this approach can be dangerous.
If we justify automation ONLY in terms of reduced
human effort, we run the risk of implying that the tools can replace the
people, and this is definitely not true! Automation supports testing, it does
not replace testers. Automation should free the testers to be able to do better
testing, designing better tests, having time to do exploratory testing, etc.
So should we abandon ROI for automation? I don’t
think that’s a good idea – we should be gaining business benefit from automation, and we should be able to show this.
Scott Barber told me about Social ROI – a way of
quantifying some of the intangible benefits – I like this but haven’t yet seen
how to incorporate it into my spreadsheet.
In our book, there are many success stories of
automation where ROI was not specifically calculated, so maybe ROI isn’t as
critical as it may have seemed.
I don’t know the answer here – these are just my
current thoughts!
