Click here to share. Breadcrumb Youth Topics Afterschool Programs. Afterschool Programs. Resources Preparing K School Administrators for a Safe Return to School in Fall This webpage from the Centers for Disease Control and Prevention provides practical guidance intended to aid school administrators as they consider how to protect the health, safety, and wellbeing of students, teachers, other school staff, their families, and communities in fall Health Resources and Services Administration.
National Institutes of Health. Office of Safe and Drug-Free Schools. Office of Special Education and Rehabilitative Services. Office of Special Education Programs. Resource: Broadband Access Landscape. Departments 4-H Military Partnerships. Department of Health and Human Services.
Feature Articles Adventure Central. Just Launched! Redesigned YE4C. Programs 21st Century Community Learning Centers. Caring for Every Child's Mental Health. Children's Health Insurance Program. Family and Youth Services Bureau. Gang Resistence and Education Program. Internet Safety. Media-Smart Youth. Neighborhood Networks.
Race to the Top. Seniorcorps - Foster Grandparent Program. Publications Addressing the Problem of Juvenile Bullying. How Access to Technology Benefits Children.
Playground Injuries Fact Sheet. Playground Safety Publications. Safety Hazards in Child Care Settings. Cooperative Extension System. Mean effects were small and non-significant for attendance and externalizing behaviors. A moderate to large amount of heterogeneity was present; however, no moderator variable tested explained the variance between studies.
Significant methodological shortcomings were identified across the corpus of studies included in this review.
Interventions that did not use more than one activity also failed to meet a number of other inclusion criteria. Roughly half of the interventions followed a manual to implement either the entire program The majority of the interventions were conducted locally The interventions had considerable variety in the dosage and frequency of treatment. The mean number of treatment sessions was , with interventions most frequently meeting 3—4 The majority of treatment sessions lasted 3—3.
Overall, there was a high risk of bias across the included studies see Fig. In terms of performance and selection bias, only one study reported the use of blinding of participants or personnel and outcome assessment. These studies reported either high overall or differential attrition and did not use a missing data strategy; thus, the results may be biased and reflect differences between groups based on participant characteristics associated with dropping out of the study rather than the effects of the intervention.
Three studies were assessed as low risk of attrition bias as they reported either low or no overall or differential attrition or used missing data strategies to perform the analysis with data from all participants assigned to condition. The remaining 10 studies were assessed as unclear risk as the studies did not clearly indicate the procedures used for the management of attrition or the attrition rates could not be reliably calculated.
Risk of bias by study is found in Table 4. H high risk of bias, L low risk of bias, U unclear risk of bias. Figure 3 depicts a forest plot of the effect sizes for attendance outcomes. Half of the studies used a measure of total attendance in school, while the other half assessed the number of absences from school.
We transformed the effect sizes so that a positive effect size indicated greater attendance. Given sufficient heterogeneity, we conducted a series of moderator analyses. Only five analyses were conducted because the focus variable did not include sufficient variability i. Figure 4 depicts a forest plot of the effect sizes for externalizing behavior outcomes.
The reporter for one externalizing behavior outcome was unknown. All effect sizes were transformed so that a positive effect size indicated a positive treatment effect i. As such, we conducted moderator analyses using all seven variables. Despite the popularity of after-school programs and the substantial resources being funneled into after-school programs across the United States, surprisingly few rigorous evaluations have been conducted to examine effects of after-school programs on behavior and school attendance outcomes.
A systematic review and meta-analysis was conducted to quantify and synthesize the effects of after-school programs on externalizing behavior and school attendance and to provide an up-to-date review of a growing research base. A comprehensive search for published and unpublished literature resulted in the inclusion of 24 after-school program intervention studies, 14 of which have not been included in prior reviews. Sixteen of the included studies measured school attendance, and 19 studies measured externalizing behaviors.
Overall, the after-school programs included in this review were found to have small and non-significant effects on externalizing behavior and school attendance outcomes. On average, students participating in after-school programs did not demonstrate improved behavior or school attendance compared to their comparison group peers.
These results contradict Durlak et al. Prior narrative reviews have reported promising but tentative conclusions about the effects of after-school programs on behavior Redd et al.
For school attendance, the evidence from this review converges with prior quantitative and narrative reviews. Simply, after-school programs have not demonstrated significant effects on school attendance Zief et al. Although 16 studies in the current review measured school attendance, few specified increasing school attendance as a primary goal of the after-school program or explicated a theory of change connecting the mechanisms of the after-school program to school attendance.
If school attendance truly is a goal of after-school programs, then it is important for after-school programs to state that explicitly as a goal and develop their programs to affect school attendance using a theory of change to drive program elements that would likely impact school attendance outcomes.
Simply implementing an after-school program with hopes that it will have positive impacts on a number of outcomes without building in specific mechanisms to impact those outcomes is likely to fail. Although the present results support findings of Zief et al. The contrast between our findings and the more positive findings from prior reviews likely stems from several factors. We included substance use measures in the construct of externalizing behaviors whereas Durlak et al.
Also, Durlak and colleagues estimated an effect size of zero for all outcomes when the primary study authors reported the result as non-significant and did not report enough data to calculate a true effect size.
Imputing a single value for an effect size may lead to biased results and is only adequate for rejecting the null hypothesis Lipsey and Wilson Imputing zero for studies that report non-significant results is not recommended as it artificially decreases the variability of the variables. Underestimating the variance is particularly problematic for answering questions related to magnitude of the mean effect, moderators of effects, and heterogeneity of studies Pigott Durlak and colleagues, furthermore, did not adjust for pre-test differences in all cases and did not require that studies control for pre-test differences or demonstrate pre-test equivalence.
This is problematic because, although all included studies in Durlak et al. Without adjusting for baseline differences, the effects could be over- or under-estimated.
Any of these procedures could have resulted in a bias of the effect size estimates and could explain the difference in the meta-analytic results between the two reviews. Finally, the current review included different studies than the Durlak review based on slightly different inclusion criteria and more recent search for studies. Unlike attendance outcomes, more attention has been paid to empirical evidence of youth development and delinquency and theories of change connecting after-school programs to externalizing behavior outcomes.
Gottfredson, Cross, and colleagues discussed routine activity theory and social control theory in their after-school program intervention studies see Cross et al. By providing adult supervision and structured activities, after-school programs have the potential to reduce delinquency Cross et al.
Cross et al. Evidence from our included studies suggested that attendance at after-school programs is poor or sporadic, and those most at-risk may be less likely to attend. Furthermore, James-Burdumy et al.
Instead, their findings suggested that participants would have been supervised had they not attended the after-school program James-Burdumy et al. Prior reviews and primary research suggest that program and participant characteristics may moderate effects of after-school programs, such as program quality and characteristics Simpkins et al.
Although theory and research suggest several possible moderators of effects of after-school programs on youth outcomes, the heterogeneity of programs included in this review and the relatively poor reporting related to specific elements and staffing of the included after-school programs made it difficult to parse out differential effects across programs related to program or participant characteristics.
Overall, evidence related to moderators and mediators of after-school program impacts is sparse and poorly developed. Future research of after-school programs could include testing of moderators and mediators in the evaluation design to improve our understanding of whether after-school programs are more or less effective based on program or participant characteristics. In addition to findings of effects, another important finding of this review is related to the quality of evidence and the extent to which the findings are valid.
All of the included studies had a number of methodological flaws that threaten the internal validity of the studies. The vast majority of the studies were rated high risk for selection bias, performance bias, and detection bias. Relatively few studies employed randomization, and those that did randomize participants rarely reported the methods by which they performed randomization.
In all, results from this review and the conclusions that can be drawn about the effects of after-school programs are limited by the quality and rigor of the included studies. However, it is clear from this and prior reviews that the rigor of after-school program research must be improved. Although past after-school program reviews have consistently suggested improvements in the rigor of after-school program research, limited progress has been made.
Our included studies displayed a high risk of bias within and across studies, impairing the extent to which we can draw conclusions about the effects of after-school programs on attendance and externalizing behaviors. However, since , when prior reviews concluded their search Durlak et al. This trend is encouraging, but must be maintained, and researchers must attend to other potential risks of bias, including minimizing performance and attrition bias, which were problematic in the studies included in this review.
Randomization is the best approach to mitigating threats to internal validity; however, randomly assigning participants to condition is not always possible.
When randomization is not possible, researchers could use more rigorous quasi-experimental designs, such as using propensity score matching, regression continuity design, or other design elements that could mitigate specific threats to internal validity that are often present in non-randomized studies Shadish et al.
In addition to using more rigorous study designs and minimizing and mitigating potential bias, it is important for studies to measure and report variables that may moderate effects of after-school programs. Some reviews and individual studies have identified potential moderators of after-school programs and other types of organized youth activities, such as study quality and characteristics of the program i. To further examine these variables in a meta-analysis, the data on moderator variables need to be consistently measured and reported.
We recommend for future studies that researchers measure and report key participant, intervention, and implementation characteristics that may moderate program outcomes.
Although Durlak et al. In particular, socio-economic status, as measured by eligibility for free or reduced lunch, was reported in less than half of our included studies.
Both demographic and participation information is important, however, to understand whether after-school programs may be more effective for some youth than others. Furthermore, greater attention must be given to the program characteristics and mechanisms by which after-school programs may impact youth or with which effects may be associated. Research has found after-school program studies lack well-defined theories of change and intervention procedures, have poor utilization of treatment manuals, provide limited training and supervision for implementers, and infrequently measure fidelity Maynard et al.
While Durlak et al. This lack of attention to intervention processes and implementation impedes our ability to examine program characteristics that may impact the effectiveness of after-school programs. Future studies could be improved by explicating a theory of change and reporting and measuring treatment procedures and fidelity. Statistical power, particularly for the attendance outcome, could be low thereby inhibiting the ability to detect effects.
Power analyses for robust variance estimation analyses are still being developed; thus, it is difficult to know for certain. We also suspect that outcome reporting bias may be an issue in after-school program intervention research as it has been found to be problematic in education research Pigott et al.
We only included studies that reported attendance or externalizing outcomes with sufficient data to calculate an effect size; however, researchers could have measured these outcomes but chose not to report them if they were not significant, thus potentially inflating the effects of after-school programs reported in this meta-analysis.
Additionally, the review did not examine all outcomes for which after-school programs have been suggested to impact; thus, the results cannot be generalized to draw conclusions about the effect of after-school programs beyond the outcomes examined in this study.
We were also limited in the number and types of moderators that we could examine in this study due to the lack of statistical power. Moreover, due to insufficient reporting of moderator variables in many of the studies included in this review, it was not possible to extract the data for potential moderator variables that may have been of interest.
Also, because selection bias is problematic in after-school program intervention research, we limited studies included in this review to those that provided pre-test data of the outcomes of interest or adjusted for pre-test on the outcome, so we could control for selection bias to some extent.
Although unlikely, we could have introduced review level selection bias by excluding studies that did not provide pre-test data or adjust for baseline differences. Despite our broad search and attempt to find studies in other countries, only one included study was conducted outside the United States; thus, the results of this review cannot be generalized to programs outside the United States.
This review was also limited by the studies included in this review. Most of the studies lacked rigor and internal validity of the studies was compromised, thus limiting the causal inferences that could be drawn from the studies and the conclusions that can be made from this review. After-school programs in the United States receive overwhelming positive support and significant resources; however, this review found a lack of evidence of effects of after-school programs on school attendance and externalizing behaviors for at-risk primary and secondary students.
Moreover, methodological flaws and high risk of bias on most of the domains assessed in this review were found across included studies, which is consistent with findings from past reviews of after-school programs. Given these findings, a reconsideration of the purpose of after-school programs and the way after-school programs are designed, implemented, and evaluated seems warranted. After-school programs are expected to affect numerous outcomes, but attempt to do so without being intentional in the program elements and mechanisms they implement by using empirical evidence or theories of change in program design to affect those outcomes.
It is clear that if our priority is to spend limited resources to provide supervision and activities for youth after school, we should also be investing in studying and implementing programs and program elements that are effective and grounded in empirical evidence and theory.
Improving the design of the programs as well as the evaluations of after-school programs to examine specific elements and contexts that may affect outcomes could provide valuable information to realize the potential of after-school programs.
The content is solely the responsibility of the authors and does not necessarily represent the official views of the supporting entities. Kristen P. Her major research interests include school engagement, academic achievement, cognitive development, youth violence, and research with at-risk students. Brandy R. Her major research interests include the etiology and developmental course of academic risk and externalizing behavior problems; evidence-based practice and knowledge translation; and research synthesis and meta-analysis.
Joshua R. His major research interests include systematic review and meta-analytic methods, prevention science, predictive modelling using statistical programs, bullying and disruptive behaviors, and academic achievement.
Michael G. Vaughn is a Professor at Saint Louis University. His major research interests include school dropout, antisocial behavior over the life course, cell-to-society approaches to the study of human behavior, youth violence prevention, and drug use epidemiology. Christine M. Sarteschi is an Assistant Professor at Chatham University. She received her doctorate in Social Work from the University of Pittsburgh. Her major research interests include mental health courts, schizophrenia, incarceration of those with serious mental illness, women and youth offenders, research synthesis and meta-analysis, mass murder, and homicidal ideation.
Electronic supplementary material. The online version of this article doi National Center for Biotechnology Information , U.
J Youth Adolesc. Author manuscript; available in PMC Oct 8. Kremer , Brandy R. Maynard , Joshua R. Polanin , Michael G. Vaughn , and Christine M. Author information Copyright and License information Disclaimer. Corresponding author. Kremer: ude. Copyright notice. The publisher's final edited version of this article is available at J Youth Adolesc. See other articles in PMC that cite the published article.
Associated Data Supplementary Materials 1. Abstract The popularity, demand, and increased federal and private funding for after-school programs have resulted in a marked increase in after-school programs over the past two decades.
Keywords: After-school programs, Attendance, Externalizing behaviors, Systematic review, Meta-analysis. Introduction Over the past two decades, the number and types of after-school programs have increased substantially. Purpose of the Present Study This systematic review and meta-analysis aimed to synthesize the available evidence on the effects of after-school programs with at-risk primary and secondary students on school attendance and externalizing behavior outcomes. Materials and Methods Systematic review methodology was used for all aspects of the search, selection, and coding of studies.
Study Eligibility Criteria Experimental and quasi-experimental studies examining the effects of an after-school program on school attendance or externalizing behaviors with at-risk primary or secondary students were included in this review.
Information Sources Several sources were used to identify eligible published and unpublished studies between and May, Study Selection and Data Extraction Titles and abstracts of the studies found through the search procedures were screened for relevance by one author. Assessing Risk of Bias The extent to which one can draw conclusions about the effects of interventions from a review depends on the extent to which the results from the included studies are valid Higgins et al. Statistical Procedures Several statistical procedures were conducted following recommendations of Pigott Results A total of 2, citations were retrieved from electronic searches of bibliographic databases, with additional citations reviewed from reference lists of prior reviews and studies and website searches.
Open in a separate window. Table 1 Summary of included studies. Frazier et al. Hirsch et al. Roth et al. James-Burdumy et al. LaFrance et al. Lauer et al. Schinke et al.
Tebes et al. Weisman et al. Zief et al. Welsh et al. Scott-Little et al. Characteristics of Included Studies and Programs Design Table 2 summarizes the characteristics of the included studies. Table 2 Study and sample characteristics.
Participants A total of , students participated in the studies. Interventions Table 3 summarizes the characteristics of the interventions for the included studies. Table 3 Intervention characteristics. Risk of Bias of Included Studies Overall, there was a high risk of bias across the included studies see Fig. Table 4 Risk of bias summary table. Table 5 Moderator analyses. Discussion Despite the popularity of after-school programs and the substantial resources being funneled into after-school programs across the United States, surprisingly few rigorous evaluations have been conducted to examine effects of after-school programs on behavior and school attendance outcomes.
Recommendations and Directions for Future Research Although past after-school program reviews have consistently suggested improvements in the rigor of after-school program research, limited progress has been made. Conclusion After-school programs in the United States receive overwhelming positive support and significant resources; however, this review found a lack of evidence of effects of after-school programs on school attendance and externalizing behaviors for at-risk primary and secondary students.
Supplementary Material 1 Click here to view. Footnotes Author contributions KK participated in the conception and design of the study, acquisition of data, and drafting of the manuscript; BM participated in the conception and design of the study, acquisition and analysis of data, and drafting of the manuscript; JP participated in the acquisition and analysis of data and revision of the manuscript; MV participated in the conception and design of the study and revision of the manuscript; CS participated in the acquisition of data and revision of the manuscript.
All authors read and approved the final manuscript. Contributor Information Kristen P. America after 3PM: Afterschool programs in demand. After-school programs for adolescents: A review of evaluation research.
A randomised control trial evaluation of a literacy after-school programme for struggling beginning readers. International Journal of Educational Research. Social and behavioral factors associated with high-risk sexual behaviors among adolescents. Journal of Behavioral Medicine. Journal of Feminist Family Therapy. Capturing unique dimensions of youth organized activity involvement theoretical and methodological considerations.
0コメント