ECONOMIC OPPORTUNITY STUDIES

 

The Community Service Block Grant’s

R.O.M.A and G.P.R.A Reports:

A Matchmaking Challenge

 

September 30, 2002

 

Prepared for

 The National Association for State Community Services Programs

http://nascsp.org/

 

Meg Power, Ph.D.   &   Kay Shedenhelm M.A.

 

Executive Summary:

This analysis examines the way the Department of Health and Human Services Administration for Children and Families (ACF) applies the Community Services Block Grant (CSBG) Government Performance and Results Act (GPRA) measures and results as compared to the way it measures other Administration for Children and Families programs.  Next it looks at the compativilities and contradictions between HHH’s performance Measurement and the Results-Oriented  Management Assessment (ROMA) system under development for the CSBG agency network. Four questions are explored:

1. How do the CSBG GPRA measures compare to the measures used by other ACF programs reporting on their achievement of the same Strategic Goal?
2. How do the CSBG outcomes compare to those of other ACF programs reporting on similar measures?
3. Could the CSBG ROMA or CSBG Information System (IS) data provide additional useful outcome measures that reflect grantees’ contributions to other ACF strategic goals?
4. Can CSBG network local agencies that manage and report on multiple ACF programs consolidate their GPRA reports about multiple ACF goals?

It finds that CSBG met its GPRA performance targets and provided valid, consistent annual reports; only a few other ACF programs achieved this during GPRA’s early years.  It finds ACF’s CSBG performance measures fit the goal to which they are assigned, but are among the most costly to collect; it also discovered most ACF performance targets are set without regard to projected funding. Some other ACF programs, including several managed by many CAAs, were poorly matched to a GPRA strategic goal, did not capture final results of expenditures, and required multiple, overlapping reports for GPRA and other required purposes.

 

1.     The GPRA Strategic Goal for CSBG:

ACF’s Goal #3:  “Increase the Health and Prosperity of Communities and Tribes”

This ACF Strategic Goal is similar to ROMA community goals (4 and 5) and to proposed new national CSBG community measures.  The measure and targets are taken from the CSBG/IS Reports on leveraging non-federal funds and on volunteers in CAAs.  The network met its 2000 and 2001 targets.

There are minor problems with CSBG’s measures; for FY 2003, CSBG faces the prospect of a 12 percent funding reduction, but the ACF FY 2003 plan, produced four months after that Administration budget proposal, does not reduce its growth target for performance.  As in all of the Goal 3 measures for all programs, no funding variances are considered, so analysts cannot tell whether ACF programs became more or less productive and efficient by comparing results to the targets.  A tool adopted by the National Association for State Community Services Programs for its GPRA reports to Office of Community Services (O.C.S.) is a leveraged rate-per-dollar of CSBG.  Such normalized measures could be used by any ACF program.

CSBG has done as well or better than the other three ACF programs (LIHEAP, Tribal Grants, and Domestic Violence Prevention Services) also working towards Goal 3, and it has not had to change its measures appreciably, as did all the others. All four programs shared some early problems with establishing a reasonable baseline from which to measure change.  All but LIHEAP reported performance near or above target levels in 2000 or 2001. All have been assigned goals that require continuing increases in ‘outputs’ or results, regardless of the funding available to achieve them.  However, most of the other programs’ measures are not measuring the kind of community-level results as CSBG, in spite of being assigned to the ACF Community Goal.

Further only LIHEAP’s and CSBG’s reports require time-consuming local record keeping. All the CSBG data needed were being voluntarily collected by CAAs prior to the initiation of GPRA report, so no additional cost was imposed on participating agencies.

In the future, as the other Goal 3 programs experiment with measurement, the CSBG network will also add two national community-level measures to its ROMA reports.  One will mirror GPRA volunteer counts. The other will count the number of new community resources created or under development, such as service programs, facilities, infrastructure, housing, and public services. ACF could consider accommodating this as a third and ‘developmental measure’ in its performance plan. When fully implemented, it promises to be the most robust measure of Strategic Goal 3 results offered by any program in the Department.

 

2.     Can CAA Program Outcomes Contribute to Achieving the Other two ACF Program Goals?

ACF Goal 1 is “to increase economic productivity and independence for families.”

CAAs have measured increasing individual self-sufficiency as a result under ROMA’s program goals 1 and 6: “low-income people increase self-sufficiency” (1) and “strengthen family” (6).

The ACF programs reporting on this goal offer a variety of income supports and family services.  They are Temporary Assistance for Needy Families (TANF), Social Services Block Grant (SSBG), Child Care, Developmental Disabilities, and Refugee Assistance. 

SSBG uses client participation figures, indicators that ROMA terms “outputs,” as its Goal 1 results. It has limited the scope of its new reports on the basis that its funds are used by states to support and to co-mingle with other categorical programs.  This paradigm, ACF argues; make measurements of client populations the appropriate indicator.  CSBG, of course, plays a similar role in most CAAs.  Child Care measures were similar to BSBG’s.  Developmental Disabilities and Refuge Resettlement establish measures very similar to ROMA’s self-sufficiency measures.

TANF has employment and job retention measures much like ROMA goals.  The program achieved many of its demanding targets in 1999 and 2000.  CAAs support half a million TANF households with services, and their reports may be included.  Many Goal 1 programs target an increase in results at levels established independent of funding and economic conditions; TANF modified its 2003 goals for wages downward in the face of 2002 economic conditions. All programs reporting under Goal 1, except TANF, had fallen short on most of these measures as of FY 2000. [1]

 

ACF Strategic Goal 2 is “to improve healthy development, safety and well-being of children and youth”

ACF measures results of Head Start, Child Care, Runaway Youth, and a few of its other programs serving children.

The Head Start Program, which dominates the budget of the CSBG network, [2] is aimed at achieving this goal. The CAAs also used more than $200 million in federal and state child care programs to support the families of Head Start clients and older children; the results of that support help achieve Head Start and Child Care goals and ACF strategic Goals 1 and 2.

While CAAs’ reporting on Goal 2 Hs and CC results is not known, an ACF memo on CSBG/Head Start performance described best measurement practices and the possible incorporation into ROMA of CAA achievement of all 14 Head Start GPRA targets.  The cost to HSB for the 2400 children, currently tracked as eight of those, is substantial, so delays in including another 650,000 children should be expected.  As an alternative in the immediate future, the 600 or more CAAs that are not hosting the FACES research can consider providing GPRA reports to match Strategic Goal 2; alternatively ACF could add CSBG as a reporting sub-category, separate from HS, under the measure: “Percent of CSBG subgrantees that used CSBG in coordination with HS and also achieved or exceeded their HS performance targets on measures (or at least 5 of 6 measures.)”

 

How many ACF and other HHS GPRA reports are CAAs providing?

Local agencies that integrate programs from many sources are burdened by multiple reporting requirements in direct proportion to their success at becoming the multi-service one-stop for families. The reporting burden depends on the number of programs they operate, not their size or funding.

Four of the ACF programs that are common in CAAs, LIHEAP, CCDBG, Head Start, and TANF, demand client-level data collected locally, as opposed to program data or service counts. On the other hand, many outputs in very specific categories are required for Head Start, Child Care, and SSBG.  Health programs take a growing share of CSBG resources, and HHS requires clinical reports from several categorical programs.

 

Conclusion:

The CSBG network was fortunate ACF chose measurements that were taken from the statutory requirements and  that were already in place in the CSBG/IS as adequate GPRA result indicators. Examination of other programs contributing to ACF Community Goal #3 indicates only a few of their measures reflect either their legislative objectives or, indeed any community-building objective, as clearly. Most are indicators of the type ROMA veterans deem to be “outputs, not outcomes”.

The CSBG network leadership’s pride in its accomplishments leads it to consider presenting other CAA accomplishments that contribute to achieving other ACF goals for GPRA reports. Indeed, since the credit or discredit for success now is attributed to block grants and programs, not their managers, successful agencies wish their institutions’ contributions to a funding category’s success could be known.  Yet, ACF has limited most programs to a single goal.

A basic question must be asked of other HHS Programs and of other Departments: What are your reports measuring that is related to the community-building and family-building mission of CAAs?  Of other programs?   A federal multi-agency report on the outcomes of managing integrated programs and establishing good linkages might prove a better measure of CAA performance. Indeed, CSBG is almost never the sole resource for meeting ROMA individual and family goals. One alternative reporting scheme could derive all ROMA individual or family outcomes from categorical programs’ GPRA reports  - like those submitted by Head Start, W.E.I.A, W.I.C., H.U.D. and other homeless services, while evaluating CSBG outcomes using only  ROMA’s new national community and agency goals.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

400 NORTH CAPITOL STREET, SUITE G-80, WASHINGTON, D.C. 20001

Tel. (202) 628 4900        Fax (202) 393 1831        E-mail info@opportunitystudies.org