Evaluator Roundtable Highlights Include Data Source Methods and ‘Second-Time’ Strategies

In this summary, originally published by The Pew Charitable Trusts in December 2023, Jim Landers, associate professor of clinical public affairs and Enarson fellow, John Glenn College of Public Affairs, The Ohio State University, shares the highlights from the ninth annual Roundtable on Evaluating Economic Development Tax Incentives event, held in 2023. In addition to data source and evaluation techniques, Landers features presentations on second-time evaluation strategies.

###

Evaluation perspectives:

The 2023 Roundtable Wrap-Up. What Are the Important Take-Aways?

Jim Landers
Associate Professor of Practice in Public Affairs, Enarson Fellow
John Glenn College of Public Affairs
The Ohio State University

Introduction

The National Conference of State Legislators (NCSL) and The Pew Charitable Trusts once again welcomed evaluators from throughout the country to the ninth annual Roundtable on Evaluating Economic Development Tax Incentives, this time at NCSL’s headquarters in Denver.

The 2023 roundtable provided no shortage of informative sources of data and interesting methodological approaches to evaluate incentive programs. It also featured an extensive and robust discussion of how evaluators can approach their work once they begin conducting second-round evaluations.

Below is a recap of some of this year’s highlights. You can find the agenda, materials, and recording of the full event on NCSL’s website.

Data, data, data and methods, methods, methods

Several roundtable presentations explored various data sources and evaluation approaches not presented at past roundtables. 

Anthony Circo from the Nebraska Legislative Audit Office (LAO) presented a 2022 evaluation that used private research and development (R&D) spending measures from the National Science Foundation to evaluate the potential impact of the state’s R&D incentive. LAO used this data in a shift-share analysis to compare growth in the state’s private R&D spending to that experienced nationally and regionally. Evaluators also made comparisons of R&D spending intensity, measured as R&D spending as a percentage of state GDP.

Yichen Su, of the Federal Reserve Bank of Dallas, employed data from Lightcast to estimate the impact of work from home on the urban wage premium and urban agglomeration. Lightcast is a rich source of U.S. employment data, containing job posting information from about 40,000 online job boards and company websites across the U.S. The data represents 70% of U.S. job vacancies and includes salary and wage information.

Rebecca Johnson’s research with the Government Services Administration’s Office of Evaluation Sciences, by contrast, used simulations to assess the distributional impacts of different funding disbursement schemes to small businesses during the COVID-19 response. This approach and lessons from the research could potentially be applied to economic development incentive programs to assess the impacts of administrative and selection processes on who benefits from these programs.

Kyle Easton from the Oregon Legislative Revenue Office presented a 2023 evaluation of the state’s agricultural worker housing tax credit. The study employs a combination of administrative tax data and data from the U.S. Department of Agriculture and U.S. Department of Labor to describe the fiscal impact of the tax credit and explain the shift in the types of housing projects accessing the tax credit. One element of the report that stood out is that taxpayers claiming the tax credit must report the credits they are awarded in a tax year, the amount of unused credits they were previously awarded, and the amount being carried forward. This eliminates a challenge that evaluators face in many states where a substantial number of credits are carried forward with a potentially large and unknown future fiscal impact.

The Second Time Around

The last two sessions of the roundtable highlighted the challenges and opportunities faced by evaluators in states that have completed, or are nearing completion of, a multiyear evaluation schedule for the first time. Evaluators from Colorado, Indiana, and Oklahoma provided informative presentations about how they have tackled second-round evaluations or are considering how to approach them in the future.

James Taurman from the Colorado Office of the State Auditor distilled the core considerations of second-round evaluations into four questions for evaluators to consider:

1. Can we offer something new when covering the same ground?
2. In what other areas can we provide information?
3. What do we do if the Legislature hasn’t taken action on our last report?
4. What do we do when our views and approach have changed?

Consistent with questions 1 and 2, Lauren Tanselle of the Indiana Legislative Services Agency (LSA) explained that they typically try to employ different analytical perspectives rather than updating previous analyses or models. For instance, they used very different analytical approaches in second-round evaluations of the state’s Tax Increment Financing (TIF) program, and its primary job creation incentive, the EDGE tax credit.

In 2015, the LSA conducted an econometric analysis to estimate the impact of TIF on property values, employment, and wages. The second evaluation in 2019 took a much different tack, providing extensive data analysis describing the sheer scale of TIF in Indiana, the types of investment in TIF districts, and debt issues used to support these investments, and again analyzed how TIF affects property values. Similarly, the LSA varied its approaches when studying the state’s EDGE jobs tax credit in 2017 and 2022. The 2017 evaluation of the EDGE tax credit compared employment changes for businesses receiving the credit to those that did not to measure whether the credit led to additional employment. However, evaluators were unable to isolate the impact of the EDGE tax credit from the impacts of market conditions and other incentives on employment. In 2022, evaluators conducted a break-even analysis to estimate the economic activity attributable to the EDGE tax credit that would be required to offset the state revenue loss from the tax credit.

Oklahoma, meanwhile, augmented its first-round analytical approach when it re-evaluated the state’s coal production tax credit. In 2017, the state’s evaluator recommended repealing the incentive because the industry was relatively small scale and limited geographically in the state. The Incentive Evaluation Commission, however, did not recommend repeal of the credit. The Legislature, while not repealing the credit, did place an annual cap on credit awards based on estimates of the credit’s cost. The second evaluation in 2021 reiterated some of the same findings from the first round and augmented it by examining the effect of the credit on business behavior and concluded that it did not induce additional coal production. This time the commission recommended the repeal of the credit, and the Legislature allowed it to sunset in 2021.  

While not a part of the session on second-round evaluations, a presentation from an earlier session answered all four of the questions referenced earlier. Washington's evaluators in the Joint Legislative Audit and Review Committee first assessed a public utility tax deduction for income from interstate transportation activities in 2010. At that time evaluators identified that the purpose of the deduction when it was enacted in 1935 was to comply with the Commerce Clause of the U.S. Constitution prohibiting state actions that impede interstate commerce.  As a result, they concluded that the deduction was no longer relevant because a 1977 U.S. Supreme Court decision upheld the imposition of a gross income tax (like the public utility tax) on interstate transportation provided the tax is apportioned to the in-state portion of the transportation activity.¹

However, the Legislature did not repeal the deduction and thus it was subject to another evaluation. This time evaluators conducted an extensive study analyzing the impact of the preference on the transportation industry’s tax burden and competitiveness, and the extent to which the preference affects transportation costs in Washington and, as a result, employment in the state’s transportation industry and other sectors that are customers of the transportation industry.

Final Thoughts

Another highly informative roundtable is in the books. It was great to connect with incentive evaluation experts from around the country and learn about important data sources, evaluation methods, and reporting practices. Thanks to NCSL for organizing the roundtable and being wonderful hosts in Denver. Thanks to everyone who participated in the roundtable, especially those who presented, and to the Incentive Evaluators Advisory Group members and staff from The Pew Charitable Trusts who led the efforts in developing an excellent agenda. We will see you in 2024!

1. Complete Auto Transit Inc., v. Brady, 430 U.S. 274 (1977).