21st Century Community Learning Centers: Evaluation and Implementation Issues

The 21st CCLC program was originally authorized as Part I of Title X, of the Elementary and Secondary Education Act (ESEA), as amended. This program was reauthorized as part of the reauthorization of the ESEA by the No Child Left Behind Act of 2001, P.L. 107-110, and was signed into law on January 8, 2002. This report discusses implementation of the reauthorized 21st CCLC program, and the recent evaluation of the program and its implications.

Order Code RL32174 CRS Report for Congress Received through the CRS Web 21st Century Community Learning Centers: Evaluation and Implementation Issues December 16, 2003 Gail McCallion Specialist in Labor Economics Domestic Social Policy Division Congressional Research Service ˜ The Library of Congress 21st Century Community Learning Centers: Evaluation and Implementation Issues Summary The 21st Century Community Learning Centers (21st CCLC) program, as reauthorized in the No Child Left Behind Act of 2001 (P.L. 107-110), and signed into law on January 8, 2002, is structured as formula grants to states, with local grants awarded competitively by states to eligible local entities for a period of 3 to 5 years. Recipients of 21st CCLC grants are to use them for out of school time activities that advance student academic achievement. The program’s focus is now exclusively on these activities for children and youth and on educational activities for their families. Increasing public and congressional attention has been focused on the program, in part because the 21st CCLC program has grown rapidly — from $750,000 in funding in its first year, FY1995, to $1 billion in FY2004. Public support for after school programs is very strong; however, a recent evaluation of the 21st CCLC program based on data from the 2000-2001 school year, conducted for the U.S. Department of Education, did not find significant improvements in most academic outcomes. The reauthorized 21st CCLC program requires state and local recipients to use funds to help students acquire the skills they need in order to meet state academic achievement standards, employing scientifically based research where relevant; and to demonstrate the success of their programs in achieving academic improvement. Several issues have arisen regarding implementation of the reauthorized program. Some of the hurdles that states and program implementers must confront in achieving academic goals include: the difficulty in securing participation by students (particularly middle schoolers) in a program focused on academics; the relatively limited amount of program time per student on average, in which to make a difference in academic achievement; and the vast divergence in after school program content and goals. Although there is limited research to date on the 21st CCLC program itself, there is an extensive and growing body of research on after school programs more generally. This research has provided many examples of promising practices in after school programs that focus on academic improvement. Some of the elements found promising for program success are well qualified and extensively trained staff, a connection to the school day curriculum, opportunities for one on one tutoring, and a structure designed to achieve clear goals. This report will not be updated. Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Accountability Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Evaluation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 The Mathematica Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Academic Improvement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Program Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Directions for the Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 List of Tables Table 1. 21st Century Learning Centers: Funding History . . . . . . . . . . . . . . . . . . 3 21st Century Community Learning Centers: Evaluation and Implementation Issues Introduction The 21st CCLC program was originally authorized as Part I of Title X, of the Elementary and Secondary Education Act (ESEA), as amended. The amendment authorizing the 21st CCLC program was included as part of the Improving America’s Schools Act of 1994, P.L. 103-382. The 21st CCLC program was initially authorized for 5 years, FY1995-FY1999. The 21st CCLC program was not reauthorized in the 106th Congress, and consequently its authorization (but not its funding) expired at the end of FY2000. This program was reauthorized as part of the reauthorization of the ESEA by the No Child Left Behind Act of 2001, P.L. 107-110, and was signed into law on January 8, 2002. Increasing public and congressional attention has been focused on the program, in part because the 21st CCLC program has grown rapidly — from $750,000 in funding in its first year, FY1995, to $1 billion in FY2004 (See Table 1 for the program’s funding history). Public support for after school programs is very strong, but first year results from a recent evaluation of the 21st CCLC program did not find significant impacts on most academic outcomes or on numbers of latchkey kids (children in self-care). This report discusses implementation of the reauthorized 21st CCLC program, and the recent evaluation of the program and its implications. Background Originally, the 21st CCLC program was a competitive grant program with grantees selected by the U.S. Department of Education (ED). Awards were for up to 3 years and were required to include at least 4 out of 13 potential activities intended to serve the local community.1 Funds were required to be equitably distributed among urban and rural areas across the nation, among the states, and among rural and urban areas within states. The program shifted in emphasis as the amount appropriated for the program increased. The original authorizing language required the Secretary of ED to give priority to those 21st CCLC projects that “offer a broad selection of services which address the needs of the community.”2 Beginning with the program’s significant expansion in FY1998, an additional priority was added 1 For a list of these 13 activities see 20 U.S.C. 8245. 2 20 U.S.C. 8243. CRS-2 for: “activities that offer expanded learning opportunities for children and youth in the community and that contribute to reduced drug use and violence.”3 In contrast, the 21st CCLC program as reauthorized in P.L. 107-110, is structured as formula grants to states, with local grants awarded competitively by states to local entities for a period of 3 to 5 years. States are allocated funds in proportion to the awards they received under Subpart 2 of Title I-A of the ESEA for the preceding year. State educational agencies (SEAs) must award at least 95% of their state allotment to eligible local entities (defined as local educational agencies (LEAs), community based organizations (CBOs), other public or private entities, or consortia of one or more of the above). To the extent possible, SEAs are to distribute funds equitably among geographic areas within the state, including urban and rural communities. SEAs may only award grants to eligible entities that will be serving students who attend schools eligible for schoolwide programs under Section 1114 of the ESEA, or schools that serve a high percentage of students from low income families, and the families of these students.4 States are to give priority to applications that propose to target services to students who attend schools that have been identified as in need of improvement under Section 1116 of the ESEA (schools that fail to make adequate yearly progress for 2 consecutive years by state measures) and are submitted jointly by an LEA and a CBO or other public or private entity.5 Recipients of 21st CCLC grants are to use these funds for before and after school activities that advance student academic achievement.6 The program’s focus is now exclusively on these before and after school activities for children and youth, and educational activities for their families. The stated purposes of the program as reauthorized, are threefold: ! ! Provide opportunities for academic enrichment to help students (particularly those attending low-performing schools) to meet state and local student academic achievement standards; Offer students a wide variety of additional services, programs and activities intended to reinforce and complement their regular academic program; and 3 For ongoing updated information on the status of the program see CRS Report RL31240, 21st Century Community Learning Centers in P.L. 107-110: Background and Funding, by Gail McCallion. 4 In general, schools eligible for schoolwide programs under Section 1114 of the ESEA, are those public schools where 40% or more of the students are from low income families. 5 An exception is made if an LEA demonstrates that it is unable to partner with a CBO of sufficient quality and reasonable geographic proximity. 6 Throughout this report, before and after school activities and out of school time activities are used synonymously. 21st CCLC funds can be used for activities before and after school, during school breaks and during the summer. CRS-3 ! Offer families of students served an opportunity for literacy and related educational development.7 Table 1. 21st Century Learning Centers: Funding History FY a. President’s budget request (in $) Appropriation (in $) 1995 0 750,000 1996 0 750,000 1997 0 1,000,000 1998 50,000,000 40,000,000 1999 200,000,000 200,000,000 2000 600,000,000 453,377,000a 2001 1,000,000,000 845,614,000b 2002 845,614,000 1,000,000,000 2003 1,000,000,000 993,500,000c 2004 600,000,000 1,005,000,000d This amount includes a rescission of FY2000 discretionary budget authority required by the FY2000 appropriations act (P.L. 106-113). b. This amount includes an across the board rescission of FY2001 appropriations adopted in the Miscellaneous Appropriations Act (H.R. 5666) enacted into law by The Consolidated Appropriations Act for FY2001 (P.L. 106-554). c. This amount includes an across the board reduction per P.L. 108-7. d. This amount has been approved by conferees, but has not yet been finalized. It may be subject to an across the board reduction of 0.59%. Accountability Requirements The reauthorized 21st CCLC program requires state and local recipients to use funds to help students acquire the skills they need in order to meet state standards, employing scientifically based research;8 and to demonstrate the success of their programs in achieving academic improvement. States are required to conduct comprehensive evaluations of local programs’ effectiveness using performance indicators and measures they have developed for that purpose. To meet the principles of effectiveness, a program must: 7 For a list of these activities see 20 U.S.C. 7175. 8 Scientifically based research is defined in 20 U.S.C. 7801. CRS-4 (A) be based upon an assessment of objective data regarding the need for before and after school programs (including during summer recess periods) and activities in the schools and communities; (B) be based upon an established set of performance measures aimed at ensuring the availability of high-quality academic enrichment opportunities; and (C) if appropriate, be based upon scientifically based research that provides evidence that the program or activity will help students meet the State and local student academic achievement standards.9 Each local program is required to participate in periodic evaluations based on these principles of effectiveness “to assess its progress toward achieving its goal of providing high quality opportunities for academic enrichment”; and the evaluation’s results must be: (i) used to refine, improve, and strengthen the program or activity, and to refine the performance measures; and (ii) made available to the public upon request, with public notice of such availability provided.10 Evaluation Issues Most of the research into after school programs has examined after school programs in general, rather than 21st CCLC programs in particular, in part because of the relative newness of the 21st CCLC program. The research often focuses on individual programs that provide high quality out of school time services to a group of children or youth over an extended period of time. In general, research that employs random assignment methods is considered the most rigorous. However, because of the rarity of such research designs in the field, many surveys of after school programs discuss at least some programs that did not employ random assignment in order to be able to examine evidence from the field on what elements in programs are most promising. There have been many surveys of this quasiexperimental and case study research that have indicated positive effects from after school programs on a variety of outcomes.11 9 20 U.S.C. 7175. Also, U.S. Department of Education, 21st Century Community Learning Centers Non-Regulatory Guidance, Feb. 2003. 10 20 U.S.C. 7175. Local grantees are also required to submit Annual Performance Reports that provide detailed information about the program. Information, including APR forms is available on the Department of Education’s website [http://www.ed.gov]. See also Evaluation of 21st Century Community Learning Center Programs A Guide for State Education Agencies, Harvard Family Research Project, Harvard Graduate School of Education, 2003. 11 A Review of Out-of-School Time Program Quasi-Experimental and Experimental Evaluation Results, Harvard Family Research Project, Harvard Graduate School of Education, 2003. This discussion is only illustrative of the extensive literature evaluating after school and youth development programs. For reviews of programs looking at the impact on youth development see Richard Catalano, et al., Positive Youth Development in the United States: Research Findings on Evaluations of Positive Youth Development (continued...) CRS-5 One example of this research is published by the Harvard Family Research Project (HFRP). The HFRP maintains a database of evaluations on the impact of out of school time programs on outcomes related to academic achievement, prevention, and youth development. Drawing on its database in a recent report, the HFRP selected 27 evaluations that employed either experimental or quasi-experimental designs for part or all of their research. Twenty-five of these evaluations assessed academic outcomes. The data from these evaluations indicated that out-of-school time programs resulted in: ! ! ! ! Better attitudes toward school and higher educational aspirations. Better performance in school, as measured by achievement test scores and grades. Higher school attendance (as measured by attendance and tardiness). Less disciplinary action (e.g., suspension).12 The Mathematica Study. The U.S. Department of Education contracted with Mathematica for both an implementation and an impact study of 21st CCLC after-school programs. The Mathematica study is the most rigorous study to date specifically focused on the 21st CCLC program. The Mathematica study is a multiyear study based on baseline and follow up data collected on 4,400 middle school students and 1,000 (in the first year) elementary school students. Data are collected from students, teachers, principals, program staff members and school records. The first results from the evaluation were published in February 2003. Based on 1 year of data for the 2000-2001 school year, the evaluation did not find significant improvements from 21st CCLC programs in most academic outcomes or in the numbers of latchkey kids (children in self-care). The study was designed to focus on outcomes of typical 21st CCLC programs, rather than of programs implementing best practices.13 The middle school sample was based on data for 4,400 students. Because these middle school programs were undersubscribed, random sampling was not used. 11 (...continued) Programs, U.S. Department of Health and Human Services and the National Institute for Child Health and Human Development, 1998; and Jacquelynne Eccles and Jennifer Appleton Gootman, Eds., Community Programs to Promote Youth Development (Washington, D.C.: National Academy Press, 2002), 411 p. 12 The HFRP report cautions that it is not possible to link specific out of school time activities with specific outcomes because the outcomes are reported as attributable to the combined effects of program components; and HFRP also indicated that their information is based on data provided in public reports authored by the program evaluators and leaders. Harvard Family Research Project, A Review, p. 2. 13 The study authors indicated that the comparison design employed: “offers a rigorous assessment of the impacts of after-school programs on middle school students. The design used for the assessment, however, was dictated by the lack of oversubscription for most middle school programs. The findings lack the same high degree of internal validity of random-assignment designs.” Mathematica and Decision Information Resources, When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program, U.S. Department of Education, 2003, p. 10. CRS-6 Instead, the study used a comparison-group model, where similar non-participating students are compared with those who participated. After school programs in middle schools typically had 45 minutes to an hour of academic programming (such as homework assistance), and then middle-schoolers would usually be able to choose an activity in which to participate (typically recreational and/or enrichment and interpersonal activities). Positive findings from the middle school centers were: small increases in school attendance, math grades, and parent involvement for students who participated. There was no significant difference between the treatment and control groups in English, science, or history grades, or in classroom performance or social development. Small negative findings were found regarding victimization outside of the classroom and engaging in negative behaviors. On average, students attended middle school programs for 32 days during the year.14 Minority students benefitted the most: African American students who participated had positive results in classroom effort, math grades and reduced lateness. Hispanic students who participated also did better in math grades and reduced lateness. The elementary school sample was based on the random assignment of 1,000 elementary school children to either a treatment group (participating in the 21st CCLC) or a control group (not participating). An additional 1,600 children were added in the 2001-2002 school year. Mathematica considers these first year results for the elementary school sample preliminary; it anticipates that first year results from the complete elementary sample will be available in the winter of 2003-2004. Random sampling was facilitated by the fact that many elementary school 21st CCLC programs were unable to serve all the students who would have liked to participate. The elementary schools in the initial sample had a higher percentage of minority students, especially African Americans, and had higher poverty levels, than other elementary centers overall, that are in the same cohort. Nevertheless, the study’s authors believe: “the elementary school findings have strong internal validity for attributing student outcome differences to the 21st Century program.”15 A typical elementary center had 45 minutes to an hour for snack and homework, an academic activity for an hour, and recreational or other activity for the remaining 1 to 2 hours. However, some elementary centers had programs that were structured differently. Some focused on skill building for state assessment tests, for example.16 Overall, positive effects for children participating in the elementary programs, compared to control group children, were found for social studies grades and some measures of parent involvement. There was no significant difference between the treatment and control groups in reading scores, homework completion, reading 14 Mathematica and Decision Information Resources, When Schools Stay Open Late, pp. 53 and 58. 15 Ibid., p. 13. 16 Ibid., p. 87. CRS-7 confidence, interpersonal skills, perceived safety during after-school hours, or behavior problems. On average, students attended the elementary programs for 58 days during the year. Mathematica’s study results for the 2000-2001 school year also indicated that participation in programs, by both middle and elementary school students, declined over time. The issuance of the first year results from the Mathematica study has prompted considerable discussion and controversy. Part of the controversy regarding the Mathematica evaluation is due to the fact that the study results were linked to a significant reduction in FY2004 spending for the 21st CCLC program proposed by the Administration in its budget request. Critiques of the Mathematica study have noted that the results are only first year results, and are for the 2000-2001 school year, before the requirements of the newly reauthorized program were in place. For example, in a 2003 Policy Commentary, the non-governmental Forum for Youth Investment states: At least a dozen other studies that employed experimental and quasiexperimental designs offer different, more positive findings about after-school programs. Drastic cuts are recommended based on the findings of one study, using one year of data collected on programs that are only a few years old, when positive findings from strong studies abound. Such decisions should be informed by an accumulation of documented knowledge.17 Additionally, some have argued that programs that are relatively new and undergoing changes should not be evaluated for funding decisions at that stage of their development. Others have criticized some aspects of the methodology employed in the study.18 Implementation Issues How will after school programs and states meet the 21st CCLC program accountability requirements? What are the hurdles program implementers might encounter in achieving academic improvement goals? Data from the first year Mathematica study have provided insight into some of the implementation issues that have been encountered by 21st CCLC programs. Academic Improvement. The reauthorized 21st CCLC program requires that programs focus on academic assistance and enrichment; and that states and program implementers evaluate the success of funded programs. Some of the hurdles 17 The Forum for Youth Investment, Out of School Time Policy Commentary #3: Reflections On System Building: Lessons from the After-School Movement (Washington, D.C.: The Forum for Youth Investment, Impact Strategies, Inc., 2003), 7 p. 18 Joseph Mahoney and Edward Zigler, The National Evaluation of the 21st Century Community Learning Centers: A Critical Analysis of First-Year Findings (New Haven, Yale University: 2003), 43 p. See also Joan Bissell, Christopher Cross, Karen Mapp, Elizabeth Reisner, Deborah Lowe Vandell, Constancia Warren, Richard Weissbourd, Statement Released by the Members of the Scientific Advisory Board for the 21st Century Community Learning Center Evaluation, May 10, 2003. Available on the Web. CRS-8 that states and program implementers must confront in achieving academic goals include: firstly, the difficulty in securing participation by students (particularly middle schoolers) in a program focused on academic improvement; secondly, the relatively limited amount of program time per student on average, in which to make a difference in academic achievement; and finally, the vast divergence in after school program content and goals. Firstly, because programs are usually voluntary, students must find the after school programs appealing. The Mathematica study found that on average, middle school students attended a center only 32 days a year, and elementary school students, 58 days a year. And, attendance was found to drop off as the year progressed. Thus, low attendance rates and the decline in participation over time may be a significant impediment to academic improvement. Because participation is voluntary, it may occur on a regular or drop in basis. For middle school students, the decision of whether or not to participate is generally made by the student. Programs must be enticing to potential participants by not making them too much like school, and by including recreational and other activities that appeal to middle schoolers. Mathematica interviewed students who chose not to participate to explore this phenomenon: Students who had chosen not to participate (surveyed in six selected programs) said that they would rather ‘hang out’ after school, were involved in other organized activities after school, or were not interested in the activities. Almost half of the students thought the centers were ‘mostly a place kids go when their parents are at work,’ and a quarter considered them ‘just for kids who need help in school.’ Participants who had stopped attending echoed these sentiments.19 Secondly, student success in programs focusing on academic achievement has been found to be strongly correlated with more time when learning actually occurs (time on task or academic learning time).20 However, most after school programs have limited “time on task” for academic activities. The after school programs evaluated by Mathematica most often provided homework assistance as the main type of academic assistance, and in many cases the assistance was minimal. In middle schools visited, for example: Site visitors observed that most homework sessions resembled study halls in which students were expected to know their assignments, bring their materials, and work independently. These sessions typically consisted of about 20 students monitored by 2 staff members (usually certified teachers or a certified teacher and a paraprofessional). Although having teachers from the host school oversee homework sessions offered a potentially fruitful path for helping students after school, the caliber of homework assistance was low ....21 19 Mathematica and Decision Information Resources, When Schools Stay Open Late, p. xvii. 20 Testimony of Dr. Gary Estes, in U.S. Congress, Senate Committee on Health, Education, Labor and Pensions, Hearing on Illiterate Graduates: Social Promotion and Retention, 106th Congress, 1st sess., Apr. 29, 1999 (Washington: 1999). 21 Mathematica and Decision Information Resources, When Schools Stay Open Late, p. 21. CRS-9 These data indicate that some existing programs may have difficulty providing academic assistance of sufficient duration, and with sufficient time on task, to significantly improve academic outcomes. There is also evidence, however, that more attendance alone may not be sufficient.22 These issues have caused some to ask whether an expectation of significantly improved academic outcomes is reasonable for these after school programs: Holding after-school programs accountable for improving standardized test scores at a time when schools are struggling under the weight of this demand makes little sense. At best, policy makers could push for proportionate joint accountability and encourage schools and out of school programs to work together to achieve this long-term goal.23 Finally, programs receiving 21st CCLC funds have traditionally been a diverse group with different goals, content, and structures. As a consequence, monitoring, setting goals, and evaluating these diverse programs can be difficult: 21st Century is a funding stream that provides resources to a range of programs, many of which are not connected by an overarching system at the local or state level. There is no guarantee that grantees are part of networks that set common standards or create a shared sense of accountability.24 Because of this diversity and lack of coordination, widespread dissemination of promising practices, and their implementation across programs, is made more difficult. Among the goals of the reauthorized 21st CCLC are providing a clearer focus for programs and facilitating the dissemination of promising practices.25 Program Sustainability. The 21st CCLC program requires prospective grantees to submit an application to states that includes (among other things): “a description of a preliminary plan for how the community learning center will continue after funding under this part ends.”26 22 First year results from Mathematica did not find more positive outcomes for middle school students who participated for more days during the year, however, it did find that these students were more disadvantaged, and more motivated, on average, than those who participated less. Mathematica and Decision Information Resources, When Schools Stay Open Late, p. 79. 23 The Forum for Youth Investment, Out-Of-School-Time Policy Commentary #1: Out-ofSchool Research Meets After-School Policy (Washington, D.C.: The Forum for Youth Investment, Impact Strategies, Inc., 2003), p. 5. 24 The Forum for Youth Investment, Out-Of-School-Time Policy Commentary #3: Reflections on System Building: Lesson from the After-School Movement (Washington, D.C.: The Forum for Youth Investment, Impact Strategies, Inc., 2003), p. 4. 25 ED is conducting a 3-year evaluation of effective academic materials that facilitate successful academic interventions that could be used by after-school programs. Department of Education, Fiscal Year 2004 Justifications of Appropriation Estimates to the Congress, p. C-34. ED has also set up a clearinghouse on what works at [http://www.w-w-c.org]. 26 20. U.S.C. 7174. CRS-10 However, the first year results from Mathematica indicate that sustainability planning is limited in many programs. Program staff indicated to Mathematica their desire to continue programs after 21st CCLC funding expired, but few had successfully lined up sufficient outside funding: ... site visitors observed few concrete actions leading toward sustainability. At the time of the visits, about one-third of grantees had made no plans and taken no actions to sustain their program; half had developed some plans but had not yet taken any action. In surveys of center coordinators and host school principals, only 10% of principals and 12% of coordinators reported that funding sources had been identified or secured.27 Mathematica cited three factors that potentially impede program sustainability: (1) project directors were often responsible for a host of tasks, many times including non-21st CCLC tasks, and as a consequence had little time for planning for the future and seeking potential funding sources; (2) project directors in remote or poor areas expressed concerns to interviewers about the lack of availability in their area of potential sources of future funding; and (3) the lack of a matching requirement for local grantees may have delayed planning for future funding. Additionally, foundation, community, and local funding for these programs is limited and the 3year grant period provided a relatively short time frame to conduct future planning.28 The reauthorized 21st CCLC program has increased the time frame for grants (they may now be up to 5 years); and permits states to require matching funds. Directions for the Future What have researchers found that may be helpful in creating effective after school programs? As noted above, research on after school programs is by and large not based on 21st CCLC programs because of its relative newness, but on other individual after school programs and largely focuses on promising practices gleaned from high quality after school programs.29 For broadly focused after school programs, one researcher has stated that effective programs must have supportive staff, positive peer relations, and opportunities for sustained participation in meaningful activities (which may be sports, arts, music, or academics).30 For after school programs with an academic focus, some research indicates that successful programs need well qualified and extensively trained staff, a connection to the school day curriculum, opportunities for one on one tutoring, and clear goals with a well laid out structure to achieve their goals.31 27 Mathematica and Decision Information Resources, When Schools Stay Open Late, p. 46. 28 Ibid., p. 48. 29 For an extensive list of resources see U.S. Department of Education, 21st Century Community Learning Centers Non-Regulatory Guidance, Feb. 2003. 30 Deborah Vandell, Forum for Youth Investment, Out-of- School Research Meets AfterSchool Policy, p. 3. 31 Fashola Olatokunbo, Review of Extended Day and After-School Programs and Their Effectiveness, Center for Research on the Education of Students Placed at Risk, Oct. 1998, (continued...) CRS-11 The North Central Regional Educational Laboratory (NCREL) has published several resources on after school programs, including the Beyond the Bell Toolkit, which is a comprehensive guide intended to provide advice on setting up an effective after school program. NCREL cites five elements that it believes are essential for effective programs: ! ! ! ! ! First, programs need to be designed based on individual students’ academic needs revealed by the school’s student assessments and teacher reports .... the regular classroom teacher should regularly share the specific needs of individual students — skills that should be learned more completely — with after-school staff. Second, staff need specific content knowledge and instructional strategies to facilitate learning .... It is not enough to have staff that simply supervise homework completion. Third, class sizes need to be small .... Generally, a ratio of 1:15 or lower for younger students seems to be ideal. Fourth, there needs to be consistent, formal, and specific communication between extended and regular day staff .... Daily planners and academic communication logs can serve as vehicles for student-led conferencing among students, staff and parents. Finally, programs need to be evaluated for their effect on raising student achievement. This means collecting pre- and postassessment data and conducting longitudinal studies on the effects of extended academic support.32 Additionally, ED, Arnold Schwarzenegger, and the C.S. Mott Foundation, hosted an after school summit in the summer of 2003 to explore ways to improve and measure the performance of 21st CCLC after school programs. The attendees developed an extensive list of student performance indicators in several major areas — academic, social, skill building, health, and community — that were viewed as important in indicating success in after school programs. In addition, a beginning list of evaluation measures and program elements essential for attaining improvement in these student performance indicators was developed by the attendees.33 As states and the local grantees work to adapt to the changes required by the 21st CCLC program as reauthorized by P.L. 107-110, they confront a variety of difficulties in implementing successful programs, some of which are discussed here. However, they will also have increasing access to a growing body of research and evaluation, including the best practice literature, to help guide the evolution of these programs. 31 (...continued) p. 55. 32 NCREL, Viewpoints No. 10, After-School Learning and Beyond, Learning Point Associates, 2003. 33 U.S. Department of Education and the C.S. Mott Foundation, After School Summit Summary Report, 2003.