According to http://www.investopedia.com/terms/r/returnoninvestment.asp, return on investment is (Gain - Cost) / Cost.
So if we have a set of automated tests that save 5 hours of manual testing, then if those tests took one hour to build and run, our ROI would be (5 - 1) / 1 = 4 (if my arithmetic is correct). So that would be a four-fold ROI for those automated tests.
Of course in the initial stages of automation, before our automation regime is mature, it might take us 10 hours to build and run those same tests. This would give an ROI of (5 - 10) / 10, i.e. a negative return on investment of -0.5.
Negative ROI is not generally a good idea, as it means you are losing money. But you may have negative ROI in the early stages of automation, but a very good positive ROI long-term. It might also make more sense to look at ROI over a larger sample, say all of the automation build and run in a month, and monitor that month by month.
How do you measure ROI for automation?