Wednesday, 13 September 2017 07:55

Updated: How to collect better data on government programs—and improve privacy, too


Update, 8 September, 11:30 a.m.: The two key congressional sponsors of a new report on making better use of government records (see story, below) say they are thrilled with the panel’s recommendations and have already begun to implement them.

“This is impressive and important work you’ve done,” Representative Paul Ryan (R–WI), speaker of the House of Representatives, told members of the Commission on Evidence-Based Policymaking yesterday during brief remarks at the report’s unveiling on Capitol Hill in Washington, D.C. But there’s more to do, emphasized Senator Patty Murray (D–WA). “A report is only as good as the work that comes from it,” she said, adding that she and Ryan are crafting a bill “to turn several of the nearly two dozen recommendations into law, and to lay down a foundation for even more work to come.”

Murray said the pending legislation, dubbed the Foundations for Evidence-Based Policymaking Act, is focused on the report’s three core ideas: expanding access to the data, ensuring privacy, and strengthening the government’s capacity to evaluate how spending trillions of dollars every year on programs affects the health, education, and economic wellbeing of millions of Americans. Ryan said the bottom line for him is “changing our approach [to government] … to get the results we want and to improve people’s lives.”

By providing your email address, you agree to send your email address to the publication. Information provided here is subject to Science's Privacy Policy.

Lawmakers have laid out a multistep approach toward achieving that goal, Murray explained. “The down-payment legislation will ensure we make immediate progress,” she said, “while we hear from our constituents and stakeholders–and while we work with the commissioners and [congressional] committees of jurisdiction on the remaining recommendations.” And her colleagues aren’t wasting any time. On 12 September, the House Committee on Oversight and Government Reform will hold a hearing to discuss the report’s 22 recommendations.

Although the commission officially shuts down at the end of the month, some of its staff will set up shop at the Bipartisan Policy Center in an attempt to keep the topic on the public agenda. “Most reports by presidential commissions have a 72-hour life span,” notes Jason Grumet, founder and president of the 10-year-old Washington, D.C.–based think tank. “But we think that the ideas behind this report are worth a sustained effort.” Two foundations are providing support for the effort, Grumet says, and several commissioners have pledged to use their Rolodexes and familiarity with how the federal government operates to help bring about change.

Here is our original story, posted on 7 September:

A blue-ribbon panel has recommended creating a secure, digital portal for researchers to study the impact of U.S. government spending on health care, education, housing, labor markets, and other sectors of the economy. A report out today by the congressionally mandated Commission on Evidence-Based Policymaking says the new mechanism, called the National Secure Data Service (NSDS), would provide one-stop shopping for scientists while improving privacy protection for everyone whose personal information appears in the databases.

“It’s a clever, elegant approach” that is also politically savvy, says a congressional aide who tracks the issue. “They’ve gone the extra mile in emphasizing privacy and transparency.” Indeed, the word “privacy” appears 408 times in the 114-page report.

Each year the U.S. government spends trillions of dollars on programs aimed at improving the lives of Americans—and a minuscule amount evaluating how well they work. One big reason for the imbalance is that rules designed to safeguard privacy also make it hard—if not impossible—for social scientists to examine the data that agencies collect while administering those programs.

“There’s a perception that if you increase data access, you will necessarily reduce privacy,” says Katharine Abraham, an economist at the University of Maryland, College Park, and chair of the commission. “We are saying that we think we can do better on both fronts.”

Accessing data on myriad government programs, ranging from Head Start to housing vouchers to food stamps, has long been a politically sensitive subject. Conservatives love the concept of greater accountability but worry about creating a Big Brother government agency that would trample on privacy. Many liberals share those concerns about privacy and also fear that research aimed at boosting efficiency could be used to justify reducing government spending on many social programs.

Even politically popular programs, such as those aimed at helping returning military veterans acquire new job skills, are hamstrung by policies that ignore the need for better evidence on how well they work. The Department of Education, for example, tracks what happens to students who have received the money but can’t identify which ones are veterans. In contrast, the Veterans Administration tracks those supported on the GI Bill but mostly ignores how they did in school.

Last year Congress created a 15-member commission to study the problem and suggest how the government could make better use of so-called administrative records to set policy. The commission was the brainchild of Ryan and Murray, both of whom have a long-standing interest in evidence-based policy. Leaders from both parties in Congress and former President Barack Obama’s administration chose the panel’s members to ensure a bipartisan lineup. And they ordered the commission to deliver its recommendations by 30 September.

A new portal

The commission’s report makes 22 recommendations. Some are relatively innocuous, like the creation of a chief evaluation officer at every agency to promote good practices. Others, like a call for a publicly available, searchable inventory of approved projects, are designed to allow skeptics of greater access to monitor what the government is doing with the data. At the same time, some proposals will certainly face strong pushback, including the commission’s call for states to cough up all the data they collect from federally funded programs they administer. (Now, federal agencies must negotiate data-sharing agreements with each state.)

The centerpiece of the report is the proposed data service. Congress had asked the commission to think about creating a physical repository that would permanently store data, but members unanimously rejected that approach in favor of a secure portal that would assemble specific data for approved studies on an as-needed basis.

“We were very much influenced by concerns about privacy and the experience about a much earlier proposal in the 1960s for a data bank,” says Abraham, a former head of the Bureau of Labor Statistics and a member of the Council of Economic Advisers under President Obama. “And I think that many members of the public, rightly, responded negatively to the idea. Instead, we all favored a proposal to bring together data needed for a specific project that had been reviewed and approved and that would only be retained in that form for the time needed to carry out that project. That way you’d never build up a huge repository.”

NSDS is also meant to deal with concerns about the size of the federal government. “None of us are politicians, but we are all aware of the fact that there would be a lot of opposition from Republicans to building up a big bureaucracy,” says Ron Haskins, a senior fellow at the Brookings Institution in Washington, D.C., and co-chair of the commission. To blunt that criticism, the report recommends housing NSDS within an existing agency, the Department of Commerce, home of the Census Bureau, the nation’s largest statistical agency. It also suggests launching the effort with help from the Census Bureau and the 12 other statistical agencies scattered across the government.

“There’s no way that all our recommendations, over the next many years, can be implemented without additional funding,” Haskins says. But he believes that shifting funds from “current spending that is not critical” could be enough to get NSDS up and running.

The report also recommends easing existing restrictions on data access. The Internal Revenue Service, for example, now can only participate in studies aimed at improving the tax code. That limitation frustrates researchers interested in using wage and other data to study social inequality. Student financial aid data managed by the Department of Education are entirely off-limits, despite their obvious value in tracking academic and career outcomes.

The commission urges Congress to ease such blanket restrictions and avoid imposing any new ones. “We should not have bans,” Haskins says. But he concedes that crafting more nuanced rules will be tricky “because there are big political issues.”

Incremental approach

Aware of those potential land mines, advocates for more evidence-based policymaking plan to move ahead gradually. The first step is likely to be a push for legislation creating the infrastructure needed for NSDS. A central element would be an oversight body, called a steering committee, that would set out performance indicators, review new privacy tools, devise a process for handling complaints and enforcement, and carry out strategic planning. Its members would be drawn from federal and state agencies, academia, and the general public.

“There’s no reason to wait” in taking that step, the congressional aide says. “The goal would be to get NSDS up and running as soon as possible, with pilot projects that can demonstrate it is a viable approach to addressing the problem.”

Once the service shows its value, the aide notes, lawmakers could tackle the more difficult challenge of easing current laws and rules that hamper wider use of administrative records. The ban on student records will be an especially tough nut to crack, Haskins and Abraham predict.

“Our recommendation is that Congress evaluate whether these bans are appropriate, and be cautious about implementing any future bans,” she says. “The people who argue for maintaining these bans on student unit records were raising issues related to privacy, and we’re very sensitive to those issues. We think we’ve proposed an approach that would ensure privacy is protected.”

The new report cites three “emerging approaches” to protecting privacy that it says are especially promising. One, called differential privacy, allows researchers to quantify the additional risks to privacy for an individual whose data would be used in a study and then to set boundaries on acceptable levels of privacy loss. The Census Bureau and internet companies like Google and Uber are already using such algorithms to evaluate risks, it notes. A second approach would use a synthetic data set that substitutes values for the original data set and then tests to see whether the synthetic data yield results that “are good enough” to satisfy the researcher.

“There are enough examples of these approaches that the commission felt comfortable making a promise to the public” that they would improve privacy protection, says commission member Robert Groves, provost of Georgetown University in Washington, D.C., and former Census Bureau director. “Scaling it up would be something on the to-do list for the NSDS,” he adds.

The commission formally goes out of business on 30 September, but some of its members hope to keep the pressure on policymakers by setting up shop in a Washington, D.C., think tank. And Haskins is under no illusion that the task will be easy.

“Congress is very interested in using evidence to make policy, so I think they will absolutely look at this report,” he says. “But that doesn’t mean it will override their political assumptions or party philosophy. There will always be a fight. But what people interested in data want is to have a seat at the table.”

Source: This article was published By Jeffrey Mervis


World's leading professional association of Internet Research Specialists - We deliver Knowledge, Education, Training, and Certification in the field of Professional Online Research. The AOFIRS is considered a major contributor in improving Web Search Skills and recognizes Online Research work as a full-time occupation for those that use the Internet as their primary source of information.

Get Exclusive Research Tips in Your Inbox

Receive Great tips via email, enter your email to Subscribe.