Evaluating IT Investments
Government organizations seeking to evaluate the worth of an information technology project have typically used metrics such as return on investment or cost/benefit ratios. But it’s wise to factor in elements that can’t necessarily be expressed in financial terms, such as the value the IT project will bring to the public.
Hence, a concept similar to ROI — the public return on investment (PROI). The Center for Technology in Government defines PROI as a measure of the delivery of specific value to the people and the improvement of the value of government itself as a public asset. This is not easily expressed in financial terms, but rather in scalable terms of current or potential end-user, statutory, policy or regulatory cost avoidance.
For example, a notification system that allows local school administrators to e-mail school closings to parents clearly benefits the health and safety of students. But there are also time savings for parents who don’t have to scan a scrolling list of statewide school closings and for schools that are able to disseminate rapid updates should the situation change. What is the financial benefit of such a service to the public? PROI gets at the societal benefits derived from an IT investment. These might include:
- Increased efficiency: Obtaining better results with the same resources or obtaining the same results with less. For example, ERP systems achieve substantial efficiencies in financial management. Our School Alerts program allows a single school administrator to alert a targeted group of a school closing, early dismissal or delayed start without inconveniencing those not concerned.
- Increased effectiveness: Increasing the quality or quantity of the desired outcome. One example would be an online registry for land data that helps improve property tax administration. Iowa School Alerts speeds up the notification process by sending alerts directly to subscribers who do not have to search long lists of statewide notifications.
- Enablement: Providing the means for an otherwise unfeasible activity, or preventing or reducing an undesirable event. For example, putting birth and marriage records online enables research by local historians and genealogists. School Alerts enables administrators to provide near-real-time updates on closings, modifying notifications as conditions change throughout the day.
- Intrinsic enhancements: Changing the environment or circumstances of a stakeholder in a way that is valued for its own sake, such as opening government financial decision making to greater transparency through enhanced accounting systems. School Alerts promotes direct communication with particular school administrators and parents.
Notice that there is little financial information contained in these benefits, but indeed there are benefits and we intuitively know that there is a return on our IT investment. PROI analysis centers around the larger social and economic benefits and costs of a project. These measures are based on either the specific program results desired by an agency or on general social benefits and improved quality of life. Though not impossible, the breadth and complexity of this kind of ROI analysis is rarely found in IT investment planning.
Identifying PROI leads to two questions: (1) How does a public-sector IT project generate increased public value, and (2) for whom? The answers lie in considering the following:
- Finances. Impacts on current or anticipated income, asset values, liabilities, entitlements or other aspects of wealth or risks to any of the above;
- Politics. Impacts on personal or corporate influence on government actions or policy, role in political affairs, or influence in political parties or prospects for current or future public office;
- Society. Impacts on family or community relationships, social mobility, status and identity;
- Strategy. Impacts on economic or political advantage or opportunities, goals, and resources for innovation or planning;
- Ideology. Impacts on beliefs, moral or ethical commitments, alignment of government actions, policies or social outcomes with beliefs, or moral or ethical positions;
- Stewardship. Impacts on the public's view of government officials as faithful stewards or guardians of the value of the government itself in terms of public trust, integrity and legitimacy.
Iowa considers public time to be worth $10 an hour, based on the average income of a typical 2.5-person household as cited by the U.S. Census Bureau. We estimate the time to perform a task in person as a baseline cost for performing a task. That cost is contrasted with the online version of the same task, giving us a financial ROI, the final criterion in our ROI calculation. Compiling scores for the other elements in an unweighted scoring allows for a somewhat objective means of evaluating a potential project. Iowa annually allocates its largest funding to the top 10 projects providing the greatest financial return to the state as measured in cost savings or gains in efficiency. If government is concerned primarily with the financial investment it is considering, the ROI analysis can stop here.
However, when considering projects that have benefit to citizens that are more difficult to measure, discussion among government officials coupled with traditional ROI measurements may result in more informed and accurate assignment of the public return on the IT investment. The Iowa statewide school-closings service is hard to measure solely in financial terms, but consideration of derivative benefits makes an argument for support. In addition, public ROI governance allows for consideration of benefits that are secondary, expanding the scope of candidates.
Which Projects Get the Cash?
- Make sure the primary beneficiary is the citizen or the user. Using service fees to support more government services makes sense to the average citizen. Point out the value of online services and the fees that support them.
- Consider derivative benefit. While it may not be readily apparent that a project directly benefits citizens or users of government services, further analysis may reveal that benefits flow beyond the primary beneficiary.
- Try for objectivity. Government bodies seek ways to make their decision making easier with the least amount of subjectivity. They need to maintain their impartiality and provide quantifiable, objective measures for approval. Doing so allows for scoring that will withstand scrutiny.
- Don’t overdo it. Whatever submission documents are employed should not be burdensome. Working through a lengthy document makes the task of governing even more difficult and may actually work against approval of an otherwise beneficial project.
- Novel ideas aren’t the easiest to quantify. The most interesting ideas may be the most difficult to quantify in terms of the benefits and to whom they flow.
- Don’t overlook the financial advantages of project development. A project may open new avenues for revenue not readily apparent. Making government services more accessible may increase usage of a service. Those new revenues can augment the budget.
Source: Malcolm Huston and John Gillispie, Iowa CIO
Iowa’s Consideration Criteria
Iowa reaped recognition for its pioneering efforts in determining the return on investment of the IT dollar. In 2000, Iowa founded a return on investment program within the state IT department to “quantify, measure, evaluate, and verify technology ‘investment’ benefits to both state government and to Iowa citizens.” The governor wanted the program to evaluate proposed IT projects and expenditures to ensure that they deliver quantifiable returns.
The Iowa program requires ROI analysis for all nonroutine IT expenditures (expenditures over $100,000 or any request for money from the pooled technology fund, which supports special cost savings, innovation or re-engineering).
Applications to the Iowa ROI program consist of five sections: a proposal description, a project administration plan, a description of the technology, a financial analysis and evaluation criteria. The criteria include:
- Statutory requirement (15 points): Project is required by law or regulation, or it is needed to comply with state IT standards. No points unless the project fulfills a mandate.
- Improves customer service (15 points): The bigger the improvement, the more points.
- Impacts citizens (10 points): More points for greater impact.
- Reengineers government processes (10 points): The most points for "significant" use of IT to revamp business processes.
- Project participants (10 points): The most points for projects needed by every state agency.
- Risk (10 points): More points for projects with low technical and business risk and high chance of success.
- Experience and past performance (5 points): More points to agencies that are historically better at achieving project objectives.
- Funding requirements (10 points): More points to projects at advanced stages of implementation.
- Additional funding sources (5 points): More points to projects for which agencies will share costs with the IT department.
- Financial ROI (10 points): The higher the ROI, the more points.
Of all these criteria, only the last scoring point, Financial ROI, could be considered a hard dollar benefit. The others are designed to promote citizen access and interagency cooperation and make existing agency processes and procedures more efficient.
