Behavioral Analytics Help Save Unemployment Insurance Funds

This is the first in a series of briefs on how states are using data for decision-making.

© iStock

Overview

Each year, billions of dollars in improper payments are disbursed throughout states as a result of unemployment insurance (UI) fraud and overpayments. To address this problem, New Mexico’s Department of Workforce Solutions (DWS) modernized and integrated two previously disconnected data systems and then analyzed its existing UI data in an effort to understand where in the claims process people were submitting misinformation. By updating its technology and subsequently applying behavioral economic theory, which uses psychology to explain financial decision-making, the state has been able to enhance its claims process. This effort resulted in a dramatic decrease in unemployment overpayments (payments to wrong parties or ineligible individuals), including fraudulent claims, and saved the state millions of dollars.

Background, Phase 1

In 2009, as the impact of the Great Recession filtered throughout the country, the national unemployment rate rose to 10 percent, the highest in more than 25 years.¹ As the number of people receiving unemployment benefits increased, so did improper payments. According to the U.S. Department of Labor (DOL), approximately $7.1 billion in overpayments and $1.5 billion in fraud were improperly obtained from unemployment insurance funds across the nation in 2009.² Owing to this increase, the DOL identified “unemployment integrity” as a significant problem affecting the UI system and offered grants to states to help reduce improper payments. New Mexico received grant funding in 2011 and 2012 to address the high cost of overpayments and assist its long-term unemployed residents.

The path to improving New Mexico’s unemployment claims system began when the DWS used the DOL funding to modernize and integrate two outdated and disconnected systems: one that dealt with businesses’ unemployment tax information and another that handled unemployment claims submitted by former workers and any corresponding job separation information provided by their former employers.

The new system allowed employers to enter job separation information electronically, replacing a two-step process in which employers submitted paper documentation and DWS staff typed it in. Overpayments decreased significantly because the upgraded and combined systems enabled the department to more quickly identify when a claimant’s application was contradicted by his or her employer’s filings. The new system also allowed for data-matching between the UI system and other state information, such as birth and death records. This cross-matching helped the DWS uncover payments that were being made to ineligible individuals, such as through the use of a deceased individual’s Social Security number. These efforts contributed to New Mexico reducing UI fraud by 60 percent, or $10 million, between 2012 and 2013.³ On Jan. 8, 2014, Governor Susana Martinez released a statement saying, “We’ve been diligent in fighting this problem, saving taxpayer dollars, and preserving our UI system for those who are truly out of work due to no fault of their own.”⁴

Later that year, the savings that came from cross-matching leveled off. DWS Deputy Cabinet Secretary Joy Forehand expressed a desire for the department to continue to improve the UI system, saying “We were not satisfied.”⁵ She wanted to get ahead of the “pay and chase” model—meaning attempts to recover overpayments after they have been sent to claimants—asking the question, “How can we predict overpayments earlier?”⁶

Building capacity, Phase 2

Harnessing the data. Deputy Secretary Forehand explained that the system had a “lot of great information” on employers and claimants that was not being utilized, including their job industries, prior claims, work history, and other variables that factor into improper payments. In 2015, the DWS launched an effort to find new ideas that would generate additional claim-related cost savings, and it partnered with Deloitte Consulting LLP because the company had upgraded and integrated the UI tax and claim systems and thoroughly understood this new system’s potential. The DWS dedicated a diverse team composed of a general counsel, labor economists, and workforce, communications, and information technology staff to the project.

Using the existing UI data, Deloitte showed the department how to uncover trends and patterns in UI applicant behavior. More specifically, the DWS used data analytics to discover the points in the application process when a claimant was more likely to make a decision that would lead to improper payments. That knowledge led Deputy Secretary Forehand to ask, “How do you use the data to cause the customer to change their behavior?”

Together, the DWS and Deloitte teams crafted solutions that combined years of institutional knowledge with innovative techniques such as data analytics and behavioral nudges—targeted messaging that influences a person to make a predictable decision.

Implementing behavioral nudge theories. The department realized that nudges implemented in the application process at the right times could encourage claimants to provide more accurate responses when applying for benefits, which could reduce overpayments. New Mexico was one of the first to use this approach to address unemployment integrity.⁷

Developing and testing the analysis before full implementation. Deloitte used the trends and patterns found in the historical data to design an analytics model that could predict future applicant actions. Selected department staff members were then trained in how to use the model. This ensured sustainability and established a sense of ownership within the DWS by equipping employees with knowledge to manage the system and the data analysis that informed the behavioral nudges.

In April 2015, before a departmentwide implementation of these analytics, Deloitte ran the model for a 30-day test period to ensure that its predictions were accurate. New Mexico was conscientious about guaranteeing that eligible claimants’ benefits were never at risk and stressed that the state would not accept false positives, which produce results showing claimants to be at risk for improper payments when they are rightly entitled to benefits. In response, Deloitte was careful to avoid wrongfully accusing individuals of the types of fraud that had resulted in lawsuits in other states and lost benefits for claimants. After running live for a month, New Mexico was confident that the system was making predictions correctly based on its analysis and that claimants were not being negatively affected.

Fully implementing behavioral nudges. The DWS launched the nudge messages in May 2015, along with a continuous randomized controlled trial (RCT) of the self-service portal in multiple places to test the effectiveness of messaging and communications. For the nudges, the department deployed individualized pop-up messages in online application forms that were triggered by the identification of fraud-related risks associated with a claimant’s answers during the application process. Instead of generic messaging on each screen, the system sent out personalized messages based on an individual’s responses during the application process.

The messages included 12 different wording options and phrasing combinations, some of which the department found to be more effective than others, demonstrating that simpler is better.⁸ For example, claimants did not change their behavior upon seeing a screen displaying the law and penalties for breaking it. However, if an individual received a pop-up stating that “9 out of 10 people in [your county] report their earnings accurately,” early analyses show that a quarter of claimants are more likely to report their actual income (see Figure 1).⁹

The pop-ups proved to be a particularly effective change, prompting more individuals to accurately report key elements of the UI eligibility process: earnings and the cause of their unemployment, whether they were laid off, terminated, or had quit.¹⁰ Although the national average dropped less than a percentage point from 2015 to 2016, New Mexico reduced its Benefit Year Earnings (BYE) overpayments, which are made when a claimant returns to work but continues to claim and receive unemployment benefits, from more than 5 percent to 2.9 percent during the same time period.¹¹ The department expects to see a 35 percent reduction in BYE fraud, or $1.9 million annually.¹²

Additionally, the department stated that 15 percent more individuals applying for benefits self-reported issues, such as the reason they left a job, which led to holding payment.¹³ The reasons for individuals leaving their jobs are important in determining whether claimants are eligible for UI benefits. The department was able to identify individuals who were not eligible for unemployment, based on their reasons for job separation, and saved on average about eight weeks of disbursed overpayments.

The DWS provided the behavioral nudges to claimants who filled out their application online and those who called the UI Operations Center to file a claim. When customers reached out by phone to ask questions, representatives read pre-written responses verbatim, simulating the process experienced by individuals applying online. This ensured that the same message was delivered, regardless of how a claimant applied for a benefit, and that the new system was deployed uniformly. The DWS invested significant time and effort in customer service training to guarantee that staff could meet the needs of individuals applying by phone.¹⁴

Sustaining the process. The department maintains the RCT in order to sustain the success of these efforts. Over the past year, more than 29,000 claimants filed nearly 200,000 weekly unemployment certifications. All were included in the RCT, resulting in a large sample that the DWS was able to use to measure various responses. The DWS continually monitors and updates the analysis behind the system, recalibrating the model periodically. The department holds quarterly meetings to examine the data and ensure that this project is not a one-time solution, but instead leads to a culture change that employs data to drive decisions. New Mexico is currently researching other policy areas, such as health and human services, that could benefit from a similar analytic approach.

Key strategies

Removing silos. At first, the department lacked the technological capacity to develop complex predictive analyses, so establishing partnerships with internal and external experts was critical.

Cutting down on improper payments necessitated the knowledge of individuals across divisions within the DWS: workforce staff providing institutional knowledge; a general counsel managing legal issues; labor economists modeling the analysis and interpreting results; communications staff messaging those results; and Workforce Technology Division staff furnishing technical assistance.¹⁵ Preventing improper payments required experts from throughout the agency in partnership with Deloitte in order to achieve a full-circle solution and support system.

Continuing UI integrity work. The 2013 launch of New Mexico’s Unemployment Insurance Tax and Claims System accelerated access to data and provided more accurate information to the state and employers. Because the new system allowed employers to add unemployment insurance information online, the state was able to cross-check employer and claimant entries to ensure their validity. DWS Deputy Secretary Joy Forehand said this system integration “greatly improved data quality.”¹⁶

The department was also able to use data analytics to begin helping the unemployed return to work. Analyses showed that two-thirds of people with current unemployment claims had previously made claims.¹⁷ With its new systems, the state sensed an opportunity to decrease the amount of time people are unemployed by encouraging potential UI recipients to complete a work search plan to create work-related goals and stay organized when seeking employment (see Figure 2). The department determined that individuals who checked the “plan to interview” response option were more than 20 percent more likely to return to work, while those who checked “plan only to inquire” were 20 percent less likely to do so. Those who stated that they planned to search for work online were also more likely to return to work.¹⁸ Moreover, these work search plans provide the department with information that can directly identify who is in danger of remaining unemployed, and they allow the state to offer more services to assist the long-term unemployed.

Conclusion

In 2012, New Mexico was wasting millions of dollars in improper UI payments. By integrating data from different systems and harnessing data to identify potential fraud and overpayments before they happened, the state was able to make demonstrable improvements to its UI system, remove waste, and save millions in UI funds. The state also utilized strong analytical tools to help ensure that unemployment insurance benefits were not unduly withheld from eligible individuals.

New Mexico’s approach stands out from the more common and administratively burdensome method of trying to identify instances of fraud and other overpayments after they occur. Behavioral analytics proved to be a pivotal tool for New Mexico to help predict improper payments and allowed the state to implement successful preventive actions.

Endnotes

1. Bureau of Labor Statistics, U.S. Department of Labor, “Labor Force Statistics From the Current Population Survey,” accessed May 11, 2016, http://data.bls.gov/timeseries/LNS14000000.
2. U.S. Department of Labor, “Calendar Year 2009 Benefit Accuracy Measurement Data Summary,” accessed May 26, 2016, http://www.ows.doleta.gov/unemploy/bam/2009/bam-cy2009.pdf.
3. New Mexico Department of Workforce Solutions, “Governor Susana Martinez Announces New Mexico Cuts Unemployment Insurance Fraud by 60%, Significantly Reduces Overpayment Rate for Second Year in a Row,” news release, Jan. 8, 2014, https://www.dws.state.nm.us/News/Latest-News/governor-susana-martinez-announces-new-mexico-cuts-unemployment-insurance-fraud-by16060.
4. Ibid.
5. The Pew Charitable Trusts, interview with Joy Forehand (deputy cabinet secretary, New Mexico Department of Workforce Solutions), March 24, 2016.
6. Ibid.
7. The Pew Charitable Trusts, interview with Joy Forehand (deputy cabinet secretary, New Mexico Department of Workforce Solutions), June 17, 2016.
8. The Pew Charitable Trusts, interview with Michael Greene, Jim Guszcza, Lynnette Stern, Dave Duden, and Anil Gosu (Deloitte Consulting LLP), April 6, 2016.
9. The Pew Charitable Trusts, email message with Joy Forehand, April 12, 2016.
10. The Pew Charitable Trusts, interview with Joy Forehand (deputy cabinet secretary, New Mexico Department of Workforce Solutions), March 24, 2016.
11. U.S. Department of Labor, “Benefit Year Earnings Overpayment Report,” accessed Aug. 2, 2016, http://oui.doleta.gov/unemploy/bye/bye_report.asp.
12. The Pew Charitable Trusts, email message with Joy Forehand, April 12, 2016.
13. Ibid.
14. The Pew Charitable Trusts, interview with Joy Forehand (deputy cabinet secretary, New Mexico Department of Workforce Solutions), March 24, 2016.
15. Ibid.
16. Ibid.
17. Ibid.
18. The Pew Charitable Trusts, email message with Joy Forehand, April 12, 2016.