Child Welfare: State Performance on Child and Family Services Reviews

Contents

• Introduction
• Origins of the CFSR
• How Is a CFSR Conducted, and What Is Assessed?
• Statewide Assessment
• Onsite Review
• What Does "Substantial Conformity" Mean?
• Outcomes
• Systems
• Aggregate State Performance on Outcomes in the Initial CFSR
• Case review only
• National standards only
• State Performance by Outcome
• Outcomes Assessed by Case Review and National Standards
• Protecting Children from Maltreatment
• Permanency and Stability in Living Arrangement
• Outcomes Assessed by Case Review Alone
• Families have enhanced capacity to provide for their children's needs
• Children receive adequate services to meet their physical and mental health needs
• Children are safely maintained in their own homes whenever possible and appropriate
• The continuity of family relationships and connections is maintained for children in foster care
• Children receive appropriate services to meet their educational needs
• State Performance on Individual Case Review Indicators
• State System Performance in the Initial CFSR
• A Closer Look at the Weaknesses Identified by the CFSR
• Permanent and Stable Living Arrangements
• Common challenges to achieving permanent and stable living arrangements
• Enhancing Families' Capacity to Meet the Needs of Their Children
• Common challenges to enhancing the capacity of families to meet the needs of their children
• Do States in System Compliance Achieve Better Outcomes?
• Assessing Penalties
• Did the Initial CFSR Accurately Measure State Performance?
• National standards
• State variation in policy and caseload
• Accuracy of data
• What is measured versus what is meant to be measured
• Measuring Outcomes
• Who Is Accountable?
• The Next Round of CFSRs

Summary

While child welfare programs are a primary responsibility of state and local governments, the federal government appropriates close to $7 billion annually to support these programs (primarily for foster care and adoption assistance) and states are required to meet certain federal policies in order to receive this funding. Child and Family Services Reviews (CFSRs) gauge state efforts and ability to achieve the primary goals of safety and permanence for children, and well-being for children and their families. The review is intended both to measure state compliance with federal child welfare policy and to strengthen and improve state child welfare programs.

The Department of Health and Human Services (HHS) conducted the initial round of onsite reviews between March 2001 and March 2004. No state was found to be in substantial conformity with all of the outcomes and systems assessed. Some critics of the CFSR argue that while the outcomes reviewed are on target, the criteria established to determine state achievement of those outcomes may give misleading information about a state's performance.

Although much attention has focused on states' uniform inability to meet all of the federal criteria, the reviews also showed certain relative strengths. States showed the greatest ability to ensure that children were not exposed to child abuse and neglect and remained safely in their homes whenever appropriate and possible, and in preserving their family relationships and connections. They had the most difficulty in achieving permanent and stable living arrangements for children, enhancing the capacity of families to meet the needs of their children and in seeing that appropriate mental and physical health services were available to children served. Information regarding ensuring provision of educational services to children was more mixed.

In addition to reviewing outcomes, the CFSR assesses state compliance with federal child welfare policy by examining certain federally required systems. States were most likely to be found successful at operating a statewide information system; maintaining foster and adoptive parent licensing, training, recruitment and retention; and responding to community concerns. They were least likely to have a strong service array or case review system in place. Ratings of state quality assurance and training systems were more mixed.

To avoid immediate assessment of penalties for failure to comply with federal policy, each state was required to develop a Program Improvement Plan (PIP). A PIP must address each one of the outcomes or systems with which a state was found to be out of substantial conformity and must describe the state's specific plan for moving toward full conformity with federal policy. A few states have successfully completed their PIPS but most are still in the process of implementing them.

The Children's Bureau is preparing for a second round of CFSRs, and onsite reviews are scheduled to begin in early 2007. This report will describe the origins and design of CFSRs before turning to its primary discussion: state performance in the initial round of CFSRs. This report will not be updated.

---

Child Welfare: State Performance on Child and Family Services Reviews

Introduction

The Child and Family Services Review (CFSR) is the central and most comprehensive component of federal efforts to determine state compliance with federal child welfare policies and, equally, to help ensure that positive outcomes are achieved for the children and families served by state child welfare programs.¹ The review intends to gauge state efforts and ability to achieve the primary goals of safety and permanence for children, along with well-being for children and their families. The U.S. Department of Health and Human Services (HHS) began the first onsite reviews in March 2001 and, as of March 2004, had completed the initial round of the CFSR in all states, the District of Columbia and Puerto Rico. In this report, both the District of Columbia and Puerto Rico will be referred to as "states."

Many states were found to have substantially achieved the goal of safety, permanence, or well-being for a majority of the cases reviewed and to have in place systems adequate to achieve positive outcomes for children. At the same time, the initial round of CFSRs found that no state's child welfare programs met the criteria that HHS established as demonstrating "substantial conformity" with all of federal child welfare policy requirements. As a result, all states have or are implementing Program Improvement Plans (PIPs). To avoid financial penalties associated with noncompliance, states must meet the improvement goals established in their PIP.

Although the final CFSR regulation states that a subsequent full CFSR is to occur in each state two years after the approval of a state PIP, in practice this has been judged impossible (e.g., such a time frame would mean that a state's second-round CFSR performance would be based on the same data that are used to assess the success of the initial PIP). The second round of CFSRs is now being planned with 15 states (DE, NC, VT, IN, NM, GA, KS, DC, TX, MA, AZ, AL, OK, OR, and MN) slated for an onsite review from March through September 2007.

This report begins with a short history of the legislation and other factors that led to the creation of the current CFSR and then briefly describes how a CFSR is conducted and what "substantial conformity" with federal child welfare policy means in the context of this review. Much has been made of the fact that no state was found to be in substantial conformity with all aspects of federal policy reviewed during the initial (FY2001-FY2004) round of the CFSRs. This report seeks to better understand that fact by looking closely at state performance on each of the performance indicators that determined compliance. Taking apart this general "not in substantial conformity" finding permits a more complex understanding of state performance, and the report uses this analysis to identify and discuss those areas in which states showed the greatest inability to achieve compliance with federal policy. (Readers should note that in addition to the text describing state performance, tables included in Appendices B and C of the report provide detailed ratings information for each state and Appendix D presents information on the case characteristics (e.g., age or race/ethnicity of child in case being reviewed vis-à-vis outcome achievement)). Finally, the report concludes with a brief discussion of 1) how penalties for non-compliance are assessed; 2) the requirement that states not in compliance with federal policy develop Program Improvement Plans (PIPs) (Appendix A shows the status of PIP implementation); 3) some of the criticisms of how the initial CFSR assessed state performance (especially with regard to the national standards); and 4) planning for the second round of CFSRs.

Origins of the CFSR

With the 1980 Adoption Assistance and Child Welfare Act (P.L. 96-272), Congress established the basic framework for the current federal-state child welfare programs. That legislation created a set of federal protections applicable to all children in foster care (e.g., a written case plan and regular case review). States were required to provide these protections, without regard to a child's eligibility for federal foster care funding, if they wished to ensure receipt of their full funding for Child Welfare Services (authorized under Title IV-B of the Social Security Act). To determine if states were indeed providing these protections to all children and were thus eligible for their full Child Welfare Services allotment HHS created "427 reviews," (which were named for the Section of the Social Security Act that established certain voluntary protections for all foster care children). The 427 review process was established without formal regulations and it came to be viewed as arbitrary, designed only to check policies on paper not in practice, and interested in identifying weaknesses for the purpose of punishment only (not for designing improvements).²

As a part of the Social Security Amendments of 1994 (P.L. 103-432) Congress required HHS to develop a new system of review for state child welfare programs that focused on outcomes achieved for children and families and that would replace the older, and discredited, process-driven reviews. The same 1994 legislation also repealed the former Section 427 but made each of its formerly voluntary protections a part of the Title IV-B state plan requirements. This made the protections mandatory for all foster care children in all states.

P.L. 103-432 (1994) further provided that the newly designed review system must cover the full range of statutory and regulatory child welfare policies and that the criteria used to measure states' compliance with specific factors were to be spelled out in formal regulations. It added that these regulations were also to detail what the penalties were, tie the amount of any financial penalty assessed to the degree of noncompliance found, suspend withholding of any penalties while states take corrective action, and further, rescind the penalties if a state successfully implemented corrective action. Finally, the legislation required HHS to offer technical assistance to any state needing to take corrective action.

During the last half of the 1990s HHS consulted with child welfare administrators and conducted 13 pilot reviews. In January 2000, the Department released its final regulation on the Child and Family Services Review (CFSR). Guided in part by the 1997 passage of the Adoption and Safe Families Act (P.L. 105-89), HHS established the overall goals of safety and permanency for children, and well-being for children and families, as the overarching aims of each state child welfare program. To achieve substantial conformity with federal child welfare policy, the final review regulation lists seven specific outcomes that a state must achieve and seven specific systems that a state must demonstrate are in operation.³ A CFSR Procedures Manual was subsequently produced and listed 45 items—or performance indicators—which were associated with the seven outcomes and seven systems and used to guide the review team through an evaluation of the state's performance.⁴ HHS also issued separate policy guidance on six national standards, the concept of which was outlined in the formal regulations, and which were developed as complimentary performance indicators using statewide data.⁵ Table 1 provides definitions for certain key terms used in this report.

How Is a CFSR Conducted, and What Is Assessed?

The CFSR begins with a state's own assessment of its child welfare programs. This self-assessment is followed by an onsite review conducted by a team of federal and state investigators. The final determination of substantial conformity with specific outcomes and systems is made following the onsite review and is based on information gathered during the onsite review as well as the analysis of statewide data for the period for which the state is under review.

Statewide Assessment

During the statewide assessment, a state must review and prepare a report on all aspects of its program performance, including its provision of services to children in foster care and those who have been reported to the child welfare system but who have not been removed from their homes. As an important part of this self-assessment, the state must analyze certain statewide program data and measure its own performance for the period under review against established national standards. The statewide assessment must include consultation with non-agency and community stakeholders in the system. The report prepared from this assessment serves as a basis for the state's onsite review but does not constitute a final determination of substantial conformity.

Onsite Review

The onsite review, conducted by a team of federal and state members, follows the statewide assessment and occurs simultaneously in three locations in the state (including the largest city or metropolitan area). It includes intensive review of a sample of cases (usually a total of 50, roughly half in foster care and half in-home cases) and interviews with a variety of stakeholders (both statewide and local), who have particular experience with or knowledge of the state child welfare programs. Stakeholders that must be interviewed include children, parents, foster parents, case workers and other service providers in the individual cases being reviewed and, with regard to the local or state program more generally, the child welfare director, case workers, foster parents, dependency or juvenile court judges, guardians ad litem, and other representatives of groups that the state consults with to design its child welfare program and services.

What Does "Substantial Conformity" Mean?

Both qualitative and quantitative information is solicited and analyzed to determine whether or not a state is in substantial conformity with federal child welfare policy. States were assessed on both the outcomes they achieved and the systems they had in place to achieve those outcomes.

Outcomes

State conformity with the seven specific outcomes is measured via case reviews and the national standards. These outcomes are:

• Children are first and foremost protected from abuse and neglect;
• Children are maintained in their own homes whenever possible and appropriate;
• Children have permanence and stability in their living situations;
• Family relationships and connections are preserved for children;
• Families have enhanced capacity to provide for their children's needs;
• Children receive appropriate services to meet their educational needs; and
• Children receive adequate services to meet their physical and mental health needs.

In the initial round of CFSRs a state needed to achieve the desired outcome in 90% of the applicable cases reviewed.⁶ Whether a state achieved one or more of the seven outcomes in a given case was based on an onsite review of the case records combined with interviews of case-specific individuals (e.g., the foster parents, case worker, and/or child). Using a standardized survey instrument reviewers then determined whether or not each specific outcome was achieved in the case by rating each of the applicable case review indicators. (See also Table 3.)

In addition to achieving an outcome in 90% of the cases, for a state to be found in substantial conformity with two of the outcomes (one associated with safety, the other with permanency) it also needed to meet each of the six national standards (data indicators). The national standards used in the initial round of the CFSR were based on state administrative data regarding recurrence of maltreatment, incidence of maltreatment while in foster care, the rate of re-entries to foster care, the stability of foster care placements, and the state's achievement of adoptions and reunifications on a "timely" basis. For a state to meet the required national standards, an analysis of statewide administrative data for the 12 months (generally) that were a part of the formal review period must indicate that the state matched or exceeded each of the established national standards. The national standards and a description of the data used to determine a state's performance with regard to these standards is provided in Table 2. (New data measures have been developed for use in the second round of the CFSR.)⁷

Systems

State compliance with certain "system" requirements was also rated and a state was found in substantial conformity with a given system requirement if that system was in place and functioning. The systems assessed were—

• Statewide information system;
• Case review system;
• Quality assurance system;
• Staff training;
• Service array;
• Agency responsiveness to community;
• Foster and adoptive parent licensing, recruitment, and retention.

Using a standardized survey instrument, reviewers rated 22 system indicators—one or more of which was linked to each of these systems—as either a "strength" or an "area needing improvement." In general, for a state to have a system found in substantial conformity all of the indicators associated with the system must be in place and no more than one of those same indicators can be functioning below the level described by the requirements. (Only one indicator was associated with the statewide information system so that states needed to achieve a strength rating on that single indicator to be found in substantial compliance with that system requirement.)

Table 3 shows the overarching goals of the review, with the seven outcomes and seven systems, and each of the associated performance indicators, including the national standards (as they were configured for the initial review).

Aggregate State Performance on Outcomes in the Initial CFSR

In general both the onsite case review and the analysis of statewide data showed that states performed best with regard to safety outcomes, had greatest difficulty with most of the permanency outcomes and showed more mixed results in relation to the well-being outcomes.

Case review only

The case reviews indicate that, as a whole, states performed most successfully in protecting children from abuse and neglect (outcome substantially achieved in 85% of cases nationally) and in ensuring that children receive appropriate services to meet their educational needs (outcome substantially achieved in 84% of cases nationally). They were least successful in ensuring that families have enhanced capacity to provide for their children's needs (outcome achieved in 55% of cases nationally) and in providing that children have permanence and stability in their living situations (outcome substantially achieved in 56% of cases nationally).

Figure 1 illustrates the percentage of cases in which states, cumulatively, were found to have substantially achieved the desired outcome, partially achieved the outcome, or not achieved or addressed the outcome. (For a list of states that achieved substantial conformity with each outcome, see Appendix B, Table B-1.)

Figure 1. Performance Rating For All Applicable Cases Assessed in the Initial Child and Family Services Review

Source: Figure prepared by the Congressional Research Service (CRS) based on data provided in the final reports of the Child and Family Services Review for 50 states, the District of Columbia and Puerto Rico.

National standards only

Table 4 shows how states performed vis-à-vis the national standards. No state met all six of the national standards. The percentage of states achieving any one of the standards ranged from about half to a little better than one out of four. As measured against the national standards used in the initial round of the CFSR, states were most successful in protecting children from maltreatment while in foster care and in ensuring a low rate of children re-entering foster care. Conversely, states were least successful at ensuring stability of placements and timely adoptions for foster care children. (For a list of states that met or did not meet each of the national standards see Appendix B, Table B-3).

State Performance by Outcome

Outcomes Assessed by Case Review and National Standards

To be found in substantial compliance with two outcomes: 1) children are first and foremost protected from abuse and neglect, and 2) children have permanence and stability in their living arrangement, states must have achieved the outcome in 90% of the applicable cases and also must have met the applicable national standards.

Protecting Children from Maltreatment

Although the states scored relatively well on both the case reviews and national standards for this outcome, when the two tests were combined, just six states (AL, AZ, AR, DC, PA, SC) were determined to have substantially achieved the goal of protecting children from abuse and neglect.

Nationally there were close to 2,350 applicable cases rated on this safety outcome. The share of cases in which individual states achieved this outcome ranged from a low of 62% (AK) to a high of 100% (NY); the median state performance was 86%. Beyond the case reviews, to determine conformity with this outcome, statewide data were examined to measure 1) the incidence of children who were the found to be the victims of child abuse or neglect more than once in a six month period; and 2) the incidence of child maltreatment occurring in foster care. Twelve states met both of these national standards. (See Appendix C, Table C-2 for information on this outcome by each state.)

Permanency and Stability in Living Arrangement

In general, states scored relatively poorly on both the case review and data measures used to determine conformity with this outcome and no state was found to be in conformity with this permanency outcome.

This outcome was applicable only to those cases reviewed in which the children were in foster care. Nationally there were close to 1,500 applicable cases rated on this outcome. The share of cases in which individual states achieved this outcome ranged from a low of 7% (KY) to a high of 92% (ND); the median state performance was 51%.⁸ Beyond the case reviews used to determine conformity with this outcome, statewide data were examined to measure 1)for children entering foster care, the percentage of those who were re-entering care within 12 months of a prior entry to foster care; 2) for foster care children who were reunified with their parents or a caretaker, the percentage of those reunifications happening within 12 months; 3) for foster care children who were adopted, the percentage of those adoptions happening within 24 months; and 4) for children in foster care less than 12 months, the percentage who were placed in no more than two settings during that time. No state met all four of those standards. However more than half of the states (28) met the standard related to foster care re-entries and no standard was achieved by fewer than one in four of the states. (See Appendix C, Table C-4 for information on this outcome by each state.)

Outcomes Assessed by Case Review Alone

Five outcomes were not associated with any national standards and states were determined to be in conformity, or not, solely on the basis of whether 90% or more of the applicable cases reviewed had substantially achieved the outcome. Although the number of states that received a substantial conformity rating for any given outcome was quite low, the range of performance on each outcome was considerable.

Families have enhanced capacity to provide for their children's needs

No state substantially achieved this outcome in 90% or more of its case review sample and thus no state achieved substantial conformity with this outcome. Nationally there were more than 2,500 cases given a performance rating for this well-being outcome. The share of cases in which individual states achieved this outcome ranged from a low of 18% (NJ) to a high of 86% (NY); the median state performance was 66%.

Children receive adequate services to meet their physical and mental health needs

Reviewers determined that only one state (DE) substantially achieved this outcome for 90% or more of its case review sample; thus it was the only state found in substantial conformity with this outcome. Nationally there were more than 2,400 cases given a performance rating for this well-being outcome. The share of cases in which individual states substantially achieved this outcome ranged from a low of 51% (PR) to a high of 92% (DE); the median state performance was 70%.

Children are safely maintained in their own homes whenever possible and appropriate

Reviewers determined that 6 states (AZ, IA, KS, NM, NY, UT) substantially achieved this outcome in 90% or more of their case review sample and thus these states were found in substantial conformity with the outcome. Nationally there were close to 2,400 cases given a performance rating on this safety outcome. The share of cases in which individual states achieved this outcome ranged from a low of 48% (NJ) to a high of 93% (IA); the median state performance was 81%.

The continuity of family relationships and connections is maintained for children in foster care

Reviewers determined that 7 states (FL, ID, LA, MA, ND, OR, TX) substantially achieved this outcome in 90% or more of their case review samples and thus these states were found in substantial conformity with the outcome. This permanency outcome applies to children who are in foster care only. Nationally there were close to 1,500 cases rated on this outcome. The share of cases in which individual states achieved this outcome ranged from a low of 38% (TN) to a high of 94% (TX and OR); the median state performance was 77%.

Children receive appropriate services to meet their educational needs

Reviewers rated 16 states (CO, CT, HI, ID, IA, KS, KY, ME, MT, NH, NY, ND, UT, VT, VA, WI) as having substantially achieved this outcome in 90% or more of their case review sample and thus these states were found in substantial conformity with the outcome. Nationally there were more than 2,000 cases rated on this well-being outcome. The share of cases in which individual states achieved this outcome ranged from a low of 65% (NJ) to a high of 100% (UT); the median state performance was 83%.

State Performance on Individual Case Review Indicators

An additional view on the areas that showed the greatest strength or need for improvement can also be gained by studying whether each of the items, or performance indicators, associated with the case review process was rated as an overall "strength" or an "area needing improvement" for the state. The "strength" or "area needing improvement" ratings for a given state represent aggregate performance across all applicable cases for a single one of these items (performance indicators). Because these ratings are not case-specific findings, they do not directly affect the determination of a state's conformity or nonconformity with a particular outcome. Nonetheless, these aggregate item ratings are discussed in the state final report and may be used to understand what contributed to a state's overall rating on an outcome (and consequently what areas should be addressed in the state PIP.)⁹

Ranking state outcome performance by case review indicators, produces a pattern of strengths and weakness similar to what the case-by-case analysis suggests. As discussed earlier no state was found to be in substantial conformity with two outcomes: 1) families have enhanced capacity to provide for the needs of their children and 2) (foster care) children have permanent and stable living arrangements. Seven of the 10 separate case review indicators associated with achieving these two outcomes were rated as a "strength" in 14% or fewer of the states. At the same time, state performance on two of the case review indicators associated with the outcome ensuring permanent and stable living arrangements for foster care children far outstripped the number of strength ratings determined for any other performance indicators. Sixty-nine percent of the states received a "strength" rating for their efforts at "keeping brothers and sisters together in foster care" and 94% received a strength rating for "placing children close to their birth parents or their own communities."

Table 5 lists each of the case review indicators assessed from those least likely to receive a strength rating to those most likely to receive a strength rating.

State System Performance in the Initial CFSR

Reviewers also rated state performance based on the state's policy and practice with regard to seven federally required "systems." This part of the CFSR is intended to measure a state's capacity to achieve positive outcomes related to safety, permanency and well-being for the children and families its serves. Ratings for this part of the review are largely based on interviews with state and local stakeholders in the child welfare system. Overall states were more likely to be assessed as having the capacity to produce positive outcomes for children than they were to have been rated as achieving these outcomes. At the same time, because these systems are intended to work together, a poor rating on any one of the systems may affect a state's ability to achieve one or all of the outcomes assessed. Further, state capacity was judged weakest with regard to case review system and service array and these systems are arguably keystones of a successful child welfare program.

Of the 52 states, 49 were found to have child welfare agencies that were "responsive to the community," 45 were judged in substantial compliance with federal requirements for a statewide information system, and 43 were found to have adequate recruitment, retention, and licensing programs for foster and adoptive parents. A less substantial majority of states were found to have a functioning quality assurance system in place (35) and to adequately meet the federal staff training requirements (34). States had more difficulty meeting the system requirements related to service array and case review. Less than half of states (23) were judged to have a service array system in compliance with federal policy and just 13 states were found in compliance with the case review system requirements.

Compliance with the system requirements of federal child welfare policy was determined based on stakeholder interviews and the number of "strength" or "area needing improvement ratings" given to each of the items associated with the implementation and proper functioning of a system. Figure 2 shows a composite (national) rating for each system—with 100% representing a strength rating for each of the items associated with an outcome for all states. The figure illustrates again that service array and case review system were determined to be the most significant areas of weaknesses in state efforts to achieve positive outcomes for children. The findings also suggests that performance in each of these areas may be more nearly equal than would appear simply by looking at the number of states in compliance with each system.

Figure 2. State Performance on Systems by Composite Strength or Area Needing Improvement Rating

Source: Figure prepared by the Congressional Research Service (CRS) based on the strength or area needing improvement ratings given for each item associated with a given system factor.

Compliance with a particular system is directly determined by the number of associated indicators that are given a strength rating. Table 6 ranks each of the indicators associated with this compliance determination, from those least likely to receive a strength rating to those most likely to receive this rating. As might be expected, it shows that items associated with the service array and case review system are more likely to be rated as areas needing improvement than are most of those items associated with agency responsiveness to the community and foster and adoptive parent recruitment, retention and licensing. However, a few items run counter to this trend. For instance, despite being an indicator related to the case review system, the large majority of states (42 states – 81%) received a strength rating for the case review item that sought to assess how good a state was at conducting a periodic review of the status of each child in foster care. By contrast just 21 states (40%) received a strength rating for their efforts to ensure the diligent recruitment of potential foster and adoptive families that reflect the ethnic and racial diversity of children needing those homes, although the vast majority of states were found in substantial conformity with the overall system related to licensing, recruitment and retention of foster care and adoptive parents.

A Closer Look at the Weaknesses Identified by the CFSR

The following discussion looks more closely at each of the indicators explicitly associated with the two outcomes for which no state was found to be in substantial conformity; it also examines indicators associated with systems that might be expected to affect state performance on those two outcomes. Many of those indicators are related to the case review and service array systems on which states were the least likely to have been found in substantial conformity.

Permanent and Stable Living Arrangements

Nationally, of the 1,479 foster care cases reviewed, permanent and stable living arrangements were determined to have been substantially achieved in 56% (822) of the cases and only partially achieved in 37% (548) of the cases. Reviewers rated 7% of the cases (109) as not having achieved or addressed this outcome at all. In assessing how well a state assists children in achieving permanent and stable living arrangements, the CFSR looks at six performance indicators for each applicable case and also examines statewide data to judge its performance with regard to four national standards. Two states substantially achieved the outcome of permanent and stable living arrangement in 90% or more of the cases (DE and ND) but no state achieved compliance with all four of the national standards associated with this outcome.¹⁰ The indicators associated with this outcome were—

• number of re-entries into foster care (case review and national standard);
• stability of placements (case review and national standard);
• timeliness and appropriateness of permanency goals (case review);
• timeliness of reunification (case review and national standard);
• timeliness of adoption (case review and national standard); and
• appropriate use of the permanency goal "another planned permanent living arrangement" (case review).

Of these indicators, states were most successful at limiting re-entries to foster care. Half of the states (26) met the national standard—meaning that statewide data showed that 8.6% or less of the children who were entering foster care in the year under review had entered foster care previously within the past 12 months—and close to half (24) received a "strength" rating for this indicator based on the applicable case reviews. Based on the case reviews, states were least successful in meeting the indicators regarding minimizing the number of placements for children and developing appropriate permanency goals for foster care children on a timely basis. Only five states received a strength rating for either one of these indicators and just 14 states met the national standard of 86.7% (or more) of the children who were in care for 12 months or less experiencing two or fewer placements.

Among the indicators related to specific permanency goals, achieving timely adoption (within 24 months of foster care entry) was the most difficult for states to achieve—only six states achieved a strength rating in this area based on the case reviews while 14 met the associated national standard. With regard to reunification with a parent or caretaker within 12 months of entering foster care, 19 met the associated national standard.¹¹ Finally, 16 states received a strength rating for their use of the permanency goal, another planned permanent living arrangement.

The case review system, which is spelled out in detail in Section 475 of the Social Security Act, is a key part of federal child welfare policy designed both to ensure the protection of children while in care and to enable them to achieve stable and permanent living arrangements. States had the most difficulty achieving strength ratings for indicators associated with implementing parts of the case review system at the case level.

• Only six states received a strength rating for development of written case plans, which are intended to guide the work done with children and families, and which federal law states must be created jointly with parents.
• Less than half of the states (22) received a strength rating for complying with the termination of parental rights (TPR) procedures outlined in the Adoption and Safe Families Act (ASFA), including identifying children who have been in foster care for 15 of the past 22 months, pursuing TPR for these children and reviewing and documenting exceptions to seeking TPR.
• While 41 states (81%) received a strength rating for holding (administrative or court) review of the status of each foster child no less frequently than every six months, only one-half of the states (26) achieved a strength rating for assuring that children in foster care received the required court permanency hearing within 12 months of entering foster care and holding subsequent court permanency hearings no less frequently than every 12 months thereafter.
• Half of the states (26) received a strength rating for ensuring that foster parents, pre-adoptive parents and relative care givers received notice of hearings or reviews held with respect to the child in their care (and had an opportunity to be heard).

Common challenges to achieving permanent and stable living arrangements

In a report providing general findings on the initial CFSRs, HHS identified "common challenges" that were related to 5 of the 6 indicators used to assess state compliance with the outcome: children have permanent and stable living arrangements.¹² To ensure comparability, this content analysis was based only on states that were reviewed in FY2002-FY2004 (35 states); a "common challenge" is defined as one noted in the final report of at least one-third of those 35 states.¹³ Close to three-fourths of the states noted that efforts to identify adoptive or other permanent placement settings at the same time as reunification efforts continued (concurrent planning) did not consistently occur and more than two-thirds of them reported that reconsideration of the goal of reunification is too often delayed. More than half of the states suggested that placement instability is related to insufficient provision of services to foster parents, not enough placement options for certain special needs children, placements made based on availability rather than suitability and frequent use of emergency shelters for temporary placements. More than half of these states also cited inconsistent access to or provision of services to parents to enable timely family reunification. Table 7 lists each of the reported challenges associated with achieving a permanent and stable living arrangement for children in foster care.

Enhancing Families' Capacity to Meet the Needs of Their Children

The performance indicators associated with the outcome discussed above, "achieving a permanent and stable living arrangement," applied only to children in foster care and outcome compliance was determined via case reviews and comparison of statewide data with the national standards. By contrast, while no statewide data indicators were used to assess compliance with the outcome, "families have enhanced capacity to meet the needs of their child," virtually all of the case review sample—in-home cases and foster care cases—was assessed with regard to this well-being outcome. Nationally of the 2,571 cases reviewed 55% (1,426) were rated as having substantially achieved this goal, 28% (727) were found to have partially achieved the goal, while 16% (418) were found to have not addressed or achieved this goal. In assessing how well a state meets this outcome in each of these cases, the CFSR looks at four specific indicators:

• assessment of the needs of children, parents and foster parents and matching the appropriate services with those individuals;
• involving parents and children in the case planning process;
• conducting regular and quality visits with children; and
• conducting regular and quality visits with parents, pre-adoptive parents, and permanent relatives or guardians of children in care.

Only one state received a strength rating for the outcome indicator concerning assessment of the child and parents needs and provision of needed services. States did relatively better on the remaining three indicators associated with the outcome "enhancing a families' capacity to provide for their children's needs"—but no more than 12 states received a strength rating for any one of these indicators.(See Table 3 above.)

Common challenges to enhancing the capacity of families to meet the needs of their children

In its content analysis of the final reports of all CFSRs conducted in FY2002-FY2004 (35 states), HHS identified 11 "common challenges" states had that were related to enhancing a family's capacity to provide for the needs of its children. All 35 states reported insufficient involvement of mothers, fathers, and children (if age-appropriate) in case planning and all but one of them cited insufficient frequency of face-to-face contacts between the case worker and the parents as a concern. A large majority of the 35 states also reported inconsistent assessments of the needs of children, parents and/or foster parents and that even when the needs are identified appropriate services are not always provided. Table 8 lists each of the reported challenges.

Do States in System Compliance Achieve Better Outcomes?

Federal child welfare policy requires states to have certain systems in place that are intended to improve how the child welfare agency functions. If these systems do increase the ability of a state to achieve positive outcomes for a state, and assuming that the CFSR accurately measures how well a state implements a given system, then states that achieved higher system compliance might be expected to have achieved the CFSR outcomes for a greater share of the cases reviewed.

Statistical analysis of the relationship between system compliance and achievement of the desired outcomes for children shows that states whose array of available services was determined in substantial compliance with federal policy had a significantly higher percentage of cases in which families were found to have enhanced capacity to meet the needs of their children and in which foster children experienced permanent and stable living arrangements (when compared to states found out of compliance with the service array requirement). Additionally, states determined to have adequately implemented a quality assurance system had a significantly higher percentage of cases in which families were found to have enhanced capacity to meet the needs of their children.¹⁴

There were too few states in compliance with all of the case review system requirements to allow for a statistical comparison based on compliance with this system. However, achievement of a "strength" rating for several of the performance indicators used to assess compliance with the case review system was associated with higher performance on certain outcomes and for other performance indicators associated with the case level review. States that received a strength rating for their implementation of termination of parental rights (TPR) proceedings in compliance with the provisions of the Adoption and Safe Families Act (ASFA) had a significantly higher percentage of cases that substantially achieved the outcome of permanent and stable living arrangements for children in foster care than did those states that were not in compliance with the ASFA TPR provisions. States that received a strength rating for ensuring periodic review of all cases no less often than every 6 months had a significantly higher percentage of cases rated as having substantially achieved the outcome of enhanced family capacity to provide for children's needs. Finally, states that received a strength rating for the 6-month case review and those that received a strength rating for implementing the required 12-month permanency hearings had a significantly higher percentage of cases that received a strength rating for achieving timely adoptions.¹⁵

Assessing Penalties

In mandating the creation of the new review system, Congress required HHS to specify in regulations how financial penalties would be determined for states found to be out of conformity with federal child welfare policy and to make those penalties commensurate with the degree of nonconformity. At the same time, it required HHS to allow states found out of conformity with federal policy to develop and implement a corrective action plan and to rescind any penalties if that plan was successfully implemented.

Financial penalties for non-conformity with federal child welfare policy are to be withheld from a specific pool of child welfare funds that, as defined in the regulation, includes all Title IV-B funds to the state (funding for both Child Welfare Services and Promoting Safe and Stable Families) and 10% of the foster care administrative costs claimed by the state under Title IV-E for the specified penalty period. For the state's initial review, penalties may range from 1% of the specified penalty pool (for failure to achieve a specified level of conformity with one of the outcomes or one of the systems studied) up to 14% (for failure to achieve a specified level of conformity with each of the 14 outcomes or systems studied).

The regulations also specify that if a state reaches the end of its required Program Improvement Plan (PIP) and fails to have successfully completed the plan, HHS must withhold funds from a state (based on the number of outcomes and systems that were found not in substantial conformity during the CFSR and for which the state did not successfully complete the level of improvement outlined in its PIP). This withholding is to begin with the last specified completion date in the PIP and continue until the state successfully achieves the relevant PIP goal or is found in substantial conformity by a subsequent full review. HHS has at times stressed that the CFSR process is about program improvement—not recoupment of federal funds. Through May 2005 it had completed evaluation of PIP implementation by eight states and determined that each of them had met their PIP goals and that therefore no penalties would be assessed. Evaluation of additional states (10 through May 2005) that had completed their PIPs continues. (For state-by-state information on PIP implementation and status see Appendix A.)

Did the Initial CFSR Accurately Measure State Performance?

State officials, advocates and researchers have raised a number of concerns about how "substantial conformity" was measured in the initial CFSR. Several of the measurement concerns are related to the national standards and might call into question the accuracy of the overall assessment that no state was found in substantial conformity with federal child welfare policy. For instance, some researchers and state officials argue that in certain cases what is actually measured does not accurately reflect what HHS intended to measure. They further argue that accurate measures are necessary not only to ensure correct penalty assessment but, as important, to ensure that the required Program Improvement Plans (PIPs) are properly focused. Other concerns raised about the CFSR raise questions about what is not measured and ensuring that accountability for outcomes is properly assigned.

National standards

The national standards are an early effort to establish a quantitative benchmark by which to judge state child welfare performance. State conformity with the national standards directly effects a state's performance rating on just two of the seven outcomes; none of the seven systems that are assessed in the CFSR are directly affected by a state's rating on the national standards. Thus the ability to meet or not meet the national standards might be understood as a relatively small factor in achieving CFSR compliance. At the same time, no state can be found in complete substantial conformity with federal policy until it meets each of the six national standards. Further, if the national standards are not designed in a way that truly measures state performance for a given issue, they might lead HHS to incorrectly require program improvements (or not require improvements) of states who risk financial harm if they do not comply. A number of critiques of these standards have been offered. HHS, which provided in the final regulations that it could "add, amend, and suspend any such statewide data indicator(s)" has announced revised national standards for the second round of the CFSR and these seek to respond to at least some of the criticisms discussed here.¹⁶

State variation in policy and caseload

Child welfare systems are administered differently by each state, and, sometimes, by each county within a state, and this can be a problem if each state is judged by a single federal standard. For instance, the federal government largely defers to the states with regard to a definition of child abuse and neglect and state definitions of what constitutes child abuse and neglect; thus, how reports of child abuse and neglect are classified varies significantly.¹⁷ This is a concern for some, who argue that requiring all states to meet the same national standard for preventing recurrence of maltreatment in foster care might unduly punish states that have enacted broader definitions of child maltreatment and/or that have a well-developed system of reporting and investigation while allowing states with more narrow definitions or less efficient reporting and investigating systems to more easily achieve federal compliance.

During the course of the initial review and analysis HHS uncovered a separate example of how varied state policy might affect a state's ability to meet the national standards established with regard to recurrence of maltreatment. Close to half of the states reviewed in FY2002-FY2004 (35 states) indicated that child abuse and neglect allegations for families with open child welfare cases (e.g., in-home cases) are not reported as new allegations of abuse or neglect and therefore there is no formal assessment of the validity of the allegation. Instead state policy or practice usually provides that the caseworker assigned to the family would informally assess the information and act to protect the children if necessary. In short, if child maltreatment is identified, this information—while it is likely acted on—would not be reported to the state's National Child Abuse and Neglect Database and thus would not be a part of the statewide data used to determine whether or not a state meets the standard on recurrence of maltreatment.¹⁸

The composition of a state's caseload may also vary for reasons that are not in control of the child welfare agency or, again, because of state policy decisions. Some researchers have called for "risk-adjusted" standards, although they acknowledge that not enough is known about, for instance demographic characteristics and achievement of certain outcomes to allow for such adjustments.¹⁹ In reviewing findings from the initial CFSR, the report General Findings from the Federal Child and Family Services Review, analyzed certain characteristics of the case review sample and the degree of success achieved on certain outcomes.²⁰ For the most part this analysis found that a state's overall performance on the CFSR outcomes—when compared to all other states—was largely unaffected by the significant state variation in the case sample demographics (including age, race/ethnicity, primary reason for case opening, and geographic location). The single exception to this overall finding was in regard to the age of children in a state's case review sample, and state performance on the safety outcome: children are safely maintained in their own homes whenever appropriate and possible. States with a higher percentage of children in their case review sample that were younger than age six at the time of their entry into foster care had a lower percentage of cases rated as substantially achieving that outcome.

At the same time, many more significant relationships were established when the lens of the analysis was shifted from a comparison between states' performance to the relationship between the particular characteristics of a child in a given case and the performance ratings for cases with children of similar or different characteristics. (For more information on these specific findings see Appendix D.)

Accuracy of data

Closely related to the issue of state variation are certain concerns about data used to determine compliance with the national standards. The most basic concerns the accuracy and comparability of the data. The majority of the data used to set the national standards as well as to assess state performance on those standards are collected via the Adoption and Foster Care Analysis and Reporting System (AFCARS). An analysis by the HHS Office of the Inspector General found that the federal "guidance on reporting AFCARS data supports states beliefs that the lack of clear definitions leads to inconsistent reporting. States believed AFCARS data elements were not clearly and consistently defined and expressed concerns about foster care placement definitions, which potentially affect child welfare performance measures. In addition, differences in states' methods of reporting dates of discharge and juvenile justice populations may further inhibit uniform performance measures." The March 2003 Inspector General report recommended that HHS provide more precise definitions for data elements, increase accessibility of technical assistance on data collection, and other provisions.²¹ An April 2004 report from the U.S. General Accounting Office (GAO) reported some of the same concerns about data inaccuracies and also made similar recommendations.²²

HHS has and continues to work on providing more guidance, assessing what states are doing and improving state data collection systems. The agency has also solicited comments on revisions to the AFCARS data collection system. Nonetheless, the national standards used for the initial CFSR were derived from data collected via AFCARS in the early years of the system's operation in most states. Although the department worked closely with each state to ensure that the data used to measure the state against the standards were the best possible data available, some find this troubling. Supporters of the policy to use statewide data measured against a national standard, however, argue that no data system is perfect, that improved data collection is a by-product of using the data (or, in essence that you have to start somewhere, sometime), and that the CFSR assessed state performance in multiple ways (using both qualitative and quantitative data) so that no state could be judged on all of the measures purely by quantitative data.

What is measured versus what is meant to be measured

Researchers have taken particular issue with three of the data indicators that are used in the national standards. These indicators seek to ensure that states 1) are reuniting children with their families in a timely manner or, 2) are finding timely adoptive placements for children who cannot be reunited with their parents; and 3) do not allow children to exit foster care to placements that are not permanent and simply result in the return of these children to foster care. While researchers generally applaud the intent of these measures—that is the effort to use quantitative data to determine how quickly and effectively states re-establish a permanent home for children who come into care—they argue that some of the measures used in the initial round of reviews might penalize a state even if its performance was improving.

Timely adoption and timely reunification

The national standards for both of these measures look at only children who exit foster care in a given year. They determine of those who were reunited with their parents, what share were reunited within 12 months of entering care and, of those who were adopted, what share were adopted within 24 months of entering care. Some researchers have argued that the focus exclusively on children who exit foster care means that the overall likelihood of a child being adopted or reunited is not being measured—i.e. the measure does not look at all children who have been in care during the year but only those who exit that care. Further, in the case of adoption, they note that if a state did the hard work of moving many of its longer-staying and likely more difficult to place children into adoptive homes, even if it held steady or improved the rate at which newer entrants moved to adoption, it might perform badly on the official measure. That is the longer staying children moved to adoption—and who presumably were a legacy of older agency practice—would necessarily reduce the share of children adopted within two years.²³

Rather than looking at children who exit foster care, one alternative measure of time to reunification or adoption might follow the group of children (or "cohort" in researcher lingo) who entered foster care in a given year to determine how many were adopted within two years or reunited within one year. This kind of measure however would require the administrative data to follow a single child's records across more than one fiscal year (i.e., it requires longitudinal data) and AFCARS is currently not set up to enable this kind of analysis.²⁴ Another alternative might group certain measures together. For instance, with regard to timely adoption, in addition to recording the share of children who exited foster care to adoption within 24 months of entering care, the measure would also look at overall likelihood of adoptions (e.g., total children served in given fiscal year divided by number of adoptions in that year).

Foster care re-entry rate

Criticism of the national standard that looks at the foster care re-entry rate focuses on the fact that changes in a state's caseload size in the given fiscal year affect state performance in this area—even though those caseload changes may be unrelated to how well a state did at achieving permanency for children in the previous year. This is because the current measure looks at all the children who enter foster care in the given year and asks what percentage of them are re-entering care within 12 months of their prior entry to foster care. This means that a state that experiences a decline in new entries to care may have fewer—or at least no greater number of children re-entering than in previous years but its performance might nonetheless be rated as worse than the previous year. By contrast a state with an increasing number of entries to foster care might increase the number of re-entries but show no decline in its performance. One alternative measure might compare the number of children who re-entered care in a given year as a share of the number of children who left foster care (via reunification, adoption, or guardianship) in the previous year.²⁵

Measuring Outcomes

Some critics of the CFSR also believe it does not go far enough in its effort to measure outcomes versus processes. In particular they cite the measurement of child and family well-being outcomes as weak. The Pew Commission on Children in Foster Care recommended that Congress require the National Academy of Sciences to study and make recommendations for appropriate measures and outcomes—especially those related to child well-being and further, that HHS convene an ongoing advisory panel of experts to periodically review the measures to ensure that they remain timely and appropriate.²⁶ Such a panel would however have to reconcile its ultimate recommendations with the federal policies currently in law. The criteria used to judge state performance is based on the federal statute, including its interpretation in regulation. Current law includes relatively limited child welfare provisions specifically related to, for instance, health treatment or education services. It requires that a child's updated health and education record be accessible to foster parents. While a logical interpretation of these requirements might be that these records are supposed to be current and accessible so that children can receive the education services and health services needed, this requires an interpretation. And an easier case might be made for measuring states compliance based on process. (For example: are the records up-to-date and available, as opposed to did the availability of the records lead to adequate health and education services for the child.)

Who Is Accountable?

This same concern might also be viewed as a question of accountability. A child welfare system must be able to count on other agencies and, especially the courts, if it is to successfully achieve positive outcomes for children. Is the state's failure to achieve "substantial conformity" with the performance measure related to the system "service array" and the well-being outcome related to providing adequate mental health services a true problem of the child welfare agency not making the appropriate services available and accessible? Or is it simply a lack of widely available child mental health services in the state? If a state is not in compliance with the statute's termination of parental rights (TPR) provisions, is this because the courts in the state have a tremendous backlog and cannot hear the cases on a timely basis or is it because the child welfare agency is not making an effort to begin these proceedings in a timely manner?

The Next Round of CFSRs

The regulation provides that for states not found in substantial conformity, a second full CFSR is to begin two years after the date HHS approves the state's PIP. However, HHS has since determined that state performance may not be reasonably re-reviewed until at least one year following the completion of its PIP. This time is necessary to ensure data used to measure state performance are based on a period after the state has completed its improvement plan.²⁷ The second round of CFSRs is now scheduled t o begin in 2007 when 15 states (DE, NC, VT, IN, NM, GA, KS, DC, TX, MA, AZ, AL, OK, OR, and MN) will undergo an onsite review. In this second round of reviews, the regulations provide that the onsite case review must find that the desired outcome was substantially achieved in 95% of the cases reviewed (as opposed to 90% in the initial round). As mentioned above, HHS has revised the national standards that will be used for this round of the CFSR and may also revise its procedures manual and survey instruments for this round of reviews.²⁸

Appendix A. Initial CFSR Review and PIP Implementation Schedule

States generally have 90 days after the Final Report date to submit a Program Improvement Plan (PIP). There is no time frame for approval of the plan by HHS, and the Department may require changes before granting approval. The approval date is also the formal implementation date for the plan. States generally have two years in which to implement their PIP. In rare instances, the regulations provide that a state may receive approval from HHS for an additional year to complete their PIP. Florida, Louisiana and Mississippi were each granted an additional year to complete their PIPs.

As of the end of September 2006, 42 states had reached the end of the PIP implementation period and for at least 20 of those states HHS had evaluated their success in meeting the agreed-upon level of improvements in those plans. In 19 of the states, the Department determined the goals of the PIP had been met and therefore all penalties that would have resulted from the state's initial CFSR were rescinded. In one jurisdiction (District of Columbia), HHS determined that not all of the agreed-upon improvements had been achieved and a fine of $135,285 has been assessed.

Appendix B. Overview of State Performance on Outcomes, Systems, and National Standards

This appendix includes three tables that summarize state performance in the initial round of Child and Family Services Reviews. Tables included are—

• Table B-1. State Performance on Outcomes in the Initial Child and Family Services Review
• Table B-2. State Performance on the National Standards in the Initial Child and Family Services Review
• Table B-3. State Performance on Systems in the Initial Child and Family Services Review

Appendix C. State Performance on Outcomes Assessed in the Initial Child and Family Services Review

This appendix contains a summary table showing aggregate state performance on the case reviews along with detailed tables showing, for each state and each outcome, how many cases were found to have substantially achieved, partially achieved and not addressed or achieved a given outcome. Individual state performance on the national standards is also shown for the two outcomes where these standards were a part of determining the state's overall compliance. Tables included are—

• Table C-1. Performance Ratings for Applicable Cases Reviewed in the Initial Child and Family Services Review
• Table C-2. Safety Outcome 1: Children are First and Foremost Protected from Abuse and Neglect
• Table C-3. Safety Outcome 2: Children are Safely Maintained in Their Own Homes Whenever Possible and Appropriate
• Table C-4. Permanency Outcome 1: Children have Permanence and Stability in Their Living Situation
• Table C-5. Permanence Outcome 2: The Continuity of Family Relationships and Connections is Preserved for Children
• Table C-6. Well-Being Outcome 1: Families Have Enhanced Capacity to Meet Their Needs of Their Children
• Table C-7. Well-Being Outcome 2: Children Receive Appropriate Services to Meet Their Education Needs
• Table C-8. Well-Being Outcome 3: Children Receive Adequate Services to Meet Their Physical and Mental Health Needs

Appendix D. Analysis of Case Characteristics and Ratings of Outcome Achievement

As part of rating each case, reviewers collected certain data about the characteristics of the child whose case was under review (e.g. age, race/ethnicity). These case specific data are not available in the final reports but have been studied by HHS (via a contract with James Bell Associates) for statistically significant relationships between outcomes achieved and the characteristics of the case. The full report discussing these findings is available on the Children's Bureau web site.²⁹ A synthesis of some of the key findings is provided below.

Age of child

Cases involving children under the age of 6 at the start of the CFSR review period were more likely to be rated as having substantially achieved permanency and stability in their living situations than cases involving children of all other ages. Age was independently established as significantly related to two of the performance indicators—placement stability and establishment of permanency goal—used to determine whether permanency and stability have been achieved for a child. These analyses generally supported the anecdotal information that placement stability is harder to achieve for adolescents and that establishing appropriate permanency goals for children in a timely manner is easier for younger children than for adolescents. The percentage of cases receiving a strength rating for placement stability decreased with increasing age until age of 16; for youth age 16-18 a strength rating for placement stability was as likely as it was for children younger than 9 at the start of the CFSR review period. The percentage of strength ratings related to establishment of the permanency goal was highest for children under the age of 6 at the start of the CFSR review period and lowest for children who were 10-12 years of age at that time.³⁰

Race/ethnicity of child

Cases involving white (non-Hispanic) children were more likely to be rated as having substantially achieved permanency and stability in their living situations than were cases involving children who are Alaska Native/Native American, Asian/Pacific Islander, or black (non-Hispanic). Separate analysis shows that cases involving children who are white (non-Hispanic) were significantly more likely than cases involving children of any other race/ethnicity to have been rated as substantially achieving the well-being outcome: children receive adequate services to meet their physical and mental health needs. Finally, cases involving children who were white (non-Hispanic), black (non-Hispanic), or of "two or more races" were significantly more likely to have substantially achieved the well-being outcome families have enhanced capacity to meet children's need than were cases involving children who are Alaska Native/Indian or Asian/Pacific Islander.³¹

Primary reason for case opening

Cases opened primarily for issues related to a child's behavior (e.g. child's substance abuse or juvenile delinquency) were more likely to be rated as having substantially achieved permanency and stability than were cases opened for any other reason. Cases opened primarily for all other reasons (e.g. parent's substance abuse, abuse or neglect of the child, mental/physical health of family) were more likely to be rated as not having substantially achieved permanency and stability in living situation than to have been rated as substantially achieving this outcome.³²

Location of case review

As noted earlier, the onsite CFSR takes place at three locations in a state, including the most populous city or county. Other locations in each state varied from very rural to metropolitan/suburban. No significant relationship between case ratings for most outcomes and review location (largest population compared to smaller population sites) was found. However, for two of the well-being outcomes, families have enhanced capacity to meet the needs of their children, and children receive appropriate services for their physical and mental health needs, cases reviewed in smaller population sites were significantly more likely to have been found to have achieved these outcomes.³³