I am writing to correct Lisbeth Schorr’s characterization of the Nurse-Family Partnership (NFP) as “frozen in time,”1 and to address a few of the issues Ms. Schorr raises regarding the use of evidence to guide policy and practice.
My colleagues and I think of the NFP as a work in progress designed to respond to our own internal evaluation of its implementation and outcomes and to take into account new research and practice standards that are emerging in all aspects of maternal and child health.2,3 Evidence-based preventive interventions are rarely final products. They have reached a stage of development and testing that warrant public investment, but inevitably require additional research and development to strengthen their impacts. Before I address our approach to program improvement, I will summarize the program and its approach to replication.
The program consists of prenatal and infancy home visiting for low-income mothers (not only teenagers as Ms. Schorr indicates) bearing first children, and has three goals: 1) to improve the outcomes of pregnancy by helping women improve their prenatal health; 2) to improve children’s subsequent health and development by helping parents provide more competent care of their children; and 3) to improve families’ economic self-sufficiency by helping develop a vision for their future and helping them make appropriate choices about staying in school, finding work, and planning subsequent pregnancies. Nurses involve other family members in the program (especially fathers), and systematically link families with other needed health and human services.4
The NFP consists of a whole package:
- targeting a segment of the population in need and likely to respond to the provision of nurse home visiting;
- provision of services at a stage in human development that will increase engagement and leverage long-term impact;
- the provision of program content, methods and service providers that align with pregnant women’s and new parents’ sense of need and aspirations;
- excellent education and consultation of nurses, supervisors, and agency administrators;
- three volumes of program visit-by-visit guidelines that provide structure to nurses in their work with families and that are adapted to families’ needs and aspirations;
- an information system from which reports are generated, benchmarked against maternal and child health goals derived from Healthy People 2010 goals, and outcomes and implementation features achieved in the original trials; and
- support of communities and agencies for quality implementation of the program prior to their becoming a program site. Organizations are guided through a process of self-assessment and development to ensure that the program is planted well in communities committed and capable of implementing the program with quality.2,3
In community practice, the NFP is being improved regularly in response to our own internal evaluation of its implementation and in response to external standards of care as the field of maternal and child health progresses, and in a way that protects its strong evidentiary roots:
- NFP program guidelines are updated quarterly to take into account the most recent practice standards recommended by the American College of Obstetrics and Gynecology, the American Academy of Pediatrics, and the American Nurses’ Association. As the program is implemented in societies outside of the US (e.g., the UK, Canada), NFP guidelines are adjusted to align with practice standards in the new society.3
- All implementing sites employ an electronic data system in which maternal and child health outcomes and features of implementation are benchmarked against standards achieved in the original trials. Reports produced on outcomes and implementation benchmarks are used to guide continuous quality improvement at local and sometimes national levels.2,3
- Variations in outcomes or implementation are used to identify sites that are performing particularly well or that need assistance. Qualitative and quantitative studies are invoked to understand this variation in greater depth. This information is then used to guide continuous quality improvement or the development of program augmentations following rigorous procedures for developing new components of the program to determine whether they actually improve NFP performance, or reduce costs.3 This usually has taken the form of testing with RCT’s, but not always. Here are some examples:
o After finding that program participants were dropping out of the program at higher rates than participants had in the original trials, we conducted a mixed-methods, multi-level study to understand factors associated with participant attrition.5 Nurses at sites with higher rates of participant retention were adapting the program more completely to the needs of families, while nurses at low-retention sites were expecting families to adapt to the program. 5 The program is designed to be adapted to families’ needs.4 We formatively developed an intervention to support nurses’ more complete adaptation of the program (in both dose and content) to families’ revealed needs and tested it in a quasi-experimental design, and then tested it in a cluster-based RCT funded by the WT Grant Foundation, with both studies showing impacts on retention and completed home visits. 6,7 The results of this program of research led to reformatting the home visit guidelines, revised technical assistance to sites, and refined nurse education.
o After finding that the 48% reduction in state-verified reports of child abuse and neglect produced by the program in the first RCT 8 was attenuated in households with moderate to high levels of Intimate Partner Violence (IPV),9 we searched the literature and found that there were no interventions for the prevention or management of IPV that met high evidentiary standards10 and that might be integrated well into the NFP. Harriet MacMillan, Susan Jack and their team at McMaster University decided to formatively develop and test a new IPV intervention designed for the NFP by aligning it with the underlying theories of the program and its operational model to assure greater uptake in community practice. 11 That intervention was piloted, refined, and is now being tested in a 15-site RCT, with randomization conducted at the level of sites, funded by the CDC, and is being examined as part of a 17-site RCT of the NFP in British Columbia.
o After finding that nurses were spending less time in promoting parents’ care of the their children in US community replication sites than had nurses in the original trials, Nancy Donelan-McCall conducted survey work with NFP nurses in practice settings and learned that they found the existing tool they had been taught to use in observing parent-infant interaction was cumbersome to learn and that it provided insufficient guidance to them in promoting competent parental care.12 This led to a multi-year program of research focused on development of a new tool (DANCE) and set of practice guidelines (DANCE STEPS) designed for NFP nurses to use in home visits.12 The goal was to design a tool that cost no more than the existing one, that had at least equivalent predictive validity, and that nurses found more useful in guiding their practice. The new tool has surpassed these expectations, and is now the new standard for NFP nurse education, without being tested in a separate RCT, as it is a simple substitution for an existing program element, and it is almost certainly superior. All existing sites in the US, the UK, and Canada are being educated in using the new tool.
o One of the most consistent findings across the three trials of the NFP is that nurse-visited mothers had reductions in the rates of closely spaced subsequent pregnancies compared to women assigned to control groups. Closely spaced subsequent pregnancies are associated with increased rates of low birth-weight and infant mortality in subsequent births,14 they make it hard for parents to return to school or work following the birth of the second child because of increased challenges with child care,15 and they make it hard for vulnerable mothers to provide good care for both first and later-born children because of mothers’ having to divide their limited caregiving resources among several young children at once.16 Dr. Alan Melnick, Dr. Teresa Gipson, and Marni Storey in the Department of Family Medicine at Oregon Health Sciences University have been leading a randomized trial of an innovation in the NFP program in which nurses are given the resources to administer hormonal contraception to NFP mothers in the context of their home visits. If effective, we will work with sites to expand nurses’ roles to include administration of hormonal contraception, which will be challenging in some settings. The potential for enhancing the public health impact of the program, however, is substantial.
o In an effort to increase the efficiency and effectiveness of the NFP, we are engaged in the development of a framework for nurses to use in calibrating the levels and types of risks and strengths exhibited by families they serve. With support from the Annie E. Casey Foundation, we are working with NFP nurses and supervisors in five sites to develop, refine, and pilot this framework for making decisions about families’ needs, possible adjustments to the NFP home visit schedule, and increasing teams’ effectiveness in delivering the program. This work is connected to earlier work we conducted in which we found that NFP nurses in the trials focused their scarce resources on families in which mothers had greater levels of need.17 We are likely to implement this intervention without the conduct of an RCT as it represents an elaboration of the NFP model as currently designed and tested in the original trials.
o In the original trials of the NFP, we found no program effects on maternal reports of symptoms of depression and anxiety, although we did find consistent improvements in care-giving and child development, and reductions in injuries among children born to mothers with limited psychological resources, including more symptoms of depression and anxiety.4, 13, 18-19 We developed a refined mental health screening tool and piloted it in Los Angeles County and New York City. Nurses and families found the tool acceptable, but nurses reported that it did not increase their ability to work with families, as most mothers were reluctant to use specialty mental health services.20 We continue to search for effective methods of addressing parental mental health issues.
o As the NFP is replicated in new societies (e.g., the UK, Canada, with Australian Aboriginal families, Native Americans, Alaskan Natives), the program first goes through a process of careful formative adaptation to the new context and populations served. The focus of this early work is on feasibility of delivering the program well in these new contexts. If the preliminary evidence looks promising, investments are made in the conduct of RCT’s in the new context, conducted by local researchers, to provide assurance that the program is producing its intended outcomes, to understand possible population or contextual moderators, and to determine whether the program is a good investment in the new context. If it is, we support careful replication of the adapted program in the new society, with the likely integration of new evidence-based augmentations as they are developed.
Ms. Schorr suggests that there is a polarization of those committed to using data from RCT’s to guide policy and practice, the Experimentalists, and those who wish to use a broader array of evidence, the Inclusionists.1 She suggests that the way to bring together the Experimentalists with the Inclusionists is to focus on core components of effective interventions. This approach minimizes the importance of careful intervention development and corresponding provider education. In fact, one of the likely reasons that so few of the parenting interventions tested in randomized controlled trials have worked well is that insufficient effort appears to have gone into intervention development before they were tested.18 The focus on core components runs the significant risk of trivializing the importance of intervention design and integrity. In this regard, it is important to note that program model clarity in specification of its elements is associated with better program implementation.22
Ms. Schorr suggests that complex interventions cannot be evaluated with RCT’s, but there are excellent examples of community-based interventions that have been tested in RCT’s, with randomization conducted at the level of communities, in which the interventions produced impacts on outcomes of public health importance. They include trials of the Communities that Care (CTC) model and Promoting School-university Partnerships to Enhance Resilience (PROSPER) 23-25 both of which show effects on youth substance use and antisocial behavior in randomly assigned intervention communities compared to controls; and the Intervention with Microfinance for AIDS and Gender Equity (IMAGE) community-based trial in South African communities.26 While the IMAGE intervention had no effects on unprotected sex with a non-spousal partner, it reduced Intimate Partner Violence in intervention communities.26 The point is that it is possible to conduct rigorous research of complex interventions at the levels of communities. Broad based community systems deserve to be evaluated rigorously using standards of evidence that are as rigorous as those used to evaluate individual services. At very least, systems of community services need to be developed using individual components that have strong evidentiary foundations.
I know of no developer of model interventions tested and found to be effective in RCT’s who believes that experimental designs are the only source of relevant information to guide policy and practice. The polarization Ms. Schorr refers to is hardly as severe as she suggests, at least on the part of experimentalists.
Sincerely,
David L. Olds, Ph.D.
Professor of Pediatrics and Director
Prevention Research Center for Family and Child Health
University of Colorado Denver, Anschutz Medical Campus
References
1. Schorr LB. Broader Evidence for Bigger Impact. Stanford Social Innovation Review 2012, 50-55.
2. Olds DL, Hill PL, O’Brien R, Racine D, Moritz P. Taking Preventive Intervention to Scale: The Nurse-Family Partnership. Cogn Behav Pract 2003, 10(4);278-290.
3. Hill P, Olds DL. Improving implementation of the Nurse-Family Partnership in the process of going to scale. In Applying Implementation Science to Early Care and Education Programs and Systems: Exploring a New Frontier. Part III: Aligning Stage-Appropriate Evaluation with the Stages of Implementation: Formative Evaluation and Fidelity. Baltimore, MD, Brookes Publishing, 2012.
4. Olds DL. Prenatal and infancy home visiting by nurses: From randomized trials to community replication. Prev Sci 2002, 3(3);153-172.
5. O’Brien RA, Moritz P, Luckey DW, McClatchey MW, Ingoldsby EM, Olds DL. Mixed methods analysis of participant attrition in the Nurse-Family Partnership. (submitted to Prevention Science)
6. Ingoldsby EM, Baca P, McClatchey MW, Luckey DW, Ramsey MO, Loch JM, et al. Increasing participant retention and home visits in prenatal and infancy home visiting by nurses. (2012 under review)
7. Olds DL, Baca P, Ingoldsby E, Luckey DW, Knudtson MD. Cluster randomized controlled trial of intervention to increase participant retention and completed home visits in the Nurse-Family Partnership. (under review)
8. Olds DL, Eckenrode J, Henderson CR Jr., Kitzman H, Powers J, Cole R, Sidora K, et al. Long-Term Effects of Home Visitation on Maternal Life Course and Child Abuse and Neglect: a 15-Year Follow-Up of a Randomized Trial. JAMA 1997, 278(8);637-643.
9. Eckenrode J, Ganzel B, Henderson CR Jr., Smith E, Olds DL, Powers J, Cole R, et al. Preventing Child Abuse and Neglect with a Program of Nurse Home Visitation. JAMA 2000, 284(11);1385-1391.
10. Wathen CN, MacMillan HL. Interventions for violence against women: Scientific review. JAMA 2003, 289(5);589-600.
11. Jack SM, Ford-Gilboe M, Wathen CN, Davidov D, McNaughton DB, Coben JH, Olds DL, MacMillan H. Development of a nurse home visitation intervention for intimate partner violence. BMC Health Serv Res 2012, 12;50.
12. Donelan-McCall N. Improving the NFP as it Moves toward Scale. Invited Speaker at the 2012 Edna McConnell Clark Foundation Grantee Retreat, May 2012. Terrytown, NY.
13. Kitzman H, Olds DL, Henderson CR Jr., Hanks C, Cole R, Tatelbaum R, McConnochie K, et al. Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing. A randomized controlled trial. JAMA 1997, 278(8);644-652.
14. Conde-Agudelo A, Rosas-Bermudez A, Kafury-Goeta AC. Birth spacing and risk of adverse perinatal outcomes: A meta-analysis. JAMA 2006, 295(15);1809-1823.
15. Furstenberg FF, Brooks-Gunn J, Morgan SP. Adolescent mothers in later life. In Human Development in Cultural and Historical Contexts. New York, NY, USA: Cambridge University Press, 1987.
16. Nathens AB, Neff MJ, Goss CH, Maier RV, Rivara FP. Effect of an older sibling and birth interval on the risk of childhood injury. Inj Prev 2000, 6(3);219-222.
17. Olds D, Korfmacher J. Maternal psychological characteristics as influences on home visitation program contact. J Community Psychol 1998, 26(1);23-36.
18. Olds DL, Robinson J, Pettitt L, Luckey DW, Holmberg J, Ng RK, Isacks, K, Sheff K. Effects of home visits by paraprofessional and by nurses: age-four follow-up of a randomized trial. Pediatrics 2004, 114(6);1560-1568.
19. Kitzman HJ, Olds DL, Cole RE, Hanks CA, Anson EA, Arcoleo KJ, Luckey DW, Knudtson MD, Henderson CR JR., Holmberg JR. Enduring effects of prenatal and infancy home visiting by nurses on children: follow-up of a randomized trail among children at age 12 years. Arch Pediat Adol Med 2010, 164(5);412-418.
20. Pinto F. Mental Health Screening and Referral Quality Improvement Project Report. Los Angeles Nurse-Family Partnership 2009.
21. Olds DL, Sadler L, Kitzman H. Programs for parents of infants and toddlers: recent evidence from randomized trials. J Child Psychol Psychiatry 2007, 48(3/4);355-391.
22. Rorbach LA, Grana R, Sussman S, Valente TW. Type 2 translation: Transporting prevention interventions from research to real-world settings. Eval Health Prof 2006, 29;302-333.
23. Hawkins JD, Oesterle S, Brown E, Arthur MW, Abbott RD, Fagan AA, Catalano RF. Results of a Type 2 translation research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Arch Pediat Adol Med 2009, 163;790-798.
24. Hawkins JD, Oesterle S, Brown EC, Monahan KC, Abbott RD, Arthur MW, Catalano RG. Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial. Arch Pediat Adol Med 2012, 166;141-148.
25. Spoth R. PROSPER community-university partnerships delivery system outcomes through 6 ½ years past baseline. (in preparation)
26. Prokyn PM, Hargreaves JR, Kim JC, Morison LA, Phetla G, Watts C, Burza J, Porter JDH. Effect of a structural intervention for the prevention of intimate-partner violence and HIV in rural South Africa: a cluster randomized trial. Lancet 2006, 368;1973-1983.
COMMENTS
BY ericnee-archive
ON October 4, 2012 03:29 PM
POSTED ON BEHALF OF DAVID OLDS
I am writing to correct Lisbeth Schorr’s characterization of the Nurse-Family Partnership (NFP) as “frozen in time,”1 and to address a few of the issues Ms. Schorr raises regarding the use of evidence to guide policy and practice.
My colleagues and I think of the NFP as a work in progress designed to respond to our own internal evaluation of its implementation and outcomes and to take into account new research and practice standards that are emerging in all aspects of maternal and child health.2,3 Evidence-based preventive interventions are rarely final products. They have reached a stage of development and testing that warrant public investment, but inevitably require additional research and development to strengthen their impacts. Before I address our approach to program improvement, I will summarize the program and its approach to replication.
The program consists of prenatal and infancy home visiting for low-income mothers (not only teenagers as Ms. Schorr indicates) bearing first children, and has three goals: 1) to improve the outcomes of pregnancy by helping women improve their prenatal health; 2) to improve children’s subsequent health and development by helping parents provide more competent care of their children; and 3) to improve families’ economic self-sufficiency by helping develop a vision for their future and helping them make appropriate choices about staying in school, finding work, and planning subsequent pregnancies. Nurses involve other family members in the program (especially fathers), and systematically link families with other needed health and human services.4
The NFP consists of a whole package:
- targeting a segment of the population in need and likely to respond to the provision of nurse home visiting;
- provision of services at a stage in human development that will increase engagement and leverage long-term impact;
- the provision of program content, methods and service providers that align with pregnant women’s and new parents’ sense of need and aspirations;
- excellent education and consultation of nurses, supervisors, and agency administrators;
- three volumes of program visit-by-visit guidelines that provide structure to nurses in their work with families and that are adapted to families’ needs and aspirations;
- an information system from which reports are generated, benchmarked against maternal and child health goals derived from Healthy People 2010 goals, and outcomes and implementation features achieved in the original trials; and
- support of communities and agencies for quality implementation of the program prior to their becoming a program site. Organizations are guided through a process of self-assessment and development to ensure that the program is planted well in communities committed and capable of implementing the program with quality.2,3
In community practice, the NFP is being improved regularly in response to our own internal evaluation of its implementation and in response to external standards of care as the field of maternal and child health progresses, and in a way that protects its strong evidentiary roots:
- NFP program guidelines are updated quarterly to take into account the most recent practice standards recommended by the American College of Obstetrics and Gynecology, the American Academy of Pediatrics, and the American Nurses’ Association. As the program is implemented in societies outside of the US (e.g., the UK, Canada), NFP guidelines are adjusted to align with practice standards in the new society.3
- All implementing sites employ an electronic data system in which maternal and child health outcomes and features of implementation are benchmarked against standards achieved in the original trials. Reports produced on outcomes and implementation benchmarks are used to guide continuous quality improvement at local and sometimes national levels.2,3
- Variations in outcomes or implementation are used to identify sites that are performing particularly well or that need assistance. Qualitative and quantitative studies are invoked to understand this variation in greater depth. This information is then used to guide continuous quality improvement or the development of program augmentations following rigorous procedures for developing new components of the program to determine whether they actually improve NFP performance, or reduce costs.3 This usually has taken the form of testing with RCT’s, but not always. Here are some examples:
o After finding that program participants were dropping out of the program at higher rates than participants had in the original trials, we conducted a mixed-methods, multi-level study to understand factors associated with participant attrition.5 Nurses at sites with higher rates of participant retention were adapting the program more completely to the needs of families, while nurses at low-retention sites were expecting families to adapt to the program. 5 The program is designed to be adapted to families’ needs.4 We formatively developed an intervention to support nurses’ more complete adaptation of the program (in both dose and content) to families’ revealed needs and tested it in a quasi-experimental design, and then tested it in a cluster-based RCT funded by the WT Grant Foundation, with both studies showing impacts on retention and completed home visits. 6,7 The results of this program of research led to reformatting the home visit guidelines, revised technical assistance to sites, and refined nurse education.
o After finding that the 48% reduction in state-verified reports of child abuse and neglect produced by the program in the first RCT 8 was attenuated in households with moderate to high levels of Intimate Partner Violence (IPV),9 we searched the literature and found that there were no interventions for the prevention or management of IPV that met high evidentiary standards10 and that might be integrated well into the NFP. Harriet MacMillan, Susan Jack and their team at McMaster University decided to formatively develop and test a new IPV intervention designed for the NFP by aligning it with the underlying theories of the program and its operational model to assure greater uptake in community practice. 11 That intervention was piloted, refined, and is now being tested in a 15-site RCT, with randomization conducted at the level of sites, funded by the CDC, and is being examined as part of a 17-site RCT of the NFP in British Columbia.
o After finding that nurses were spending less time in promoting parents’ care of the their children in US community replication sites than had nurses in the original trials, Nancy Donelan-McCall conducted survey work with NFP nurses in practice settings and learned that they found the existing tool they had been taught to use in observing parent-infant interaction was cumbersome to learn and that it provided insufficient guidance to them in promoting competent parental care.12 This led to a multi-year program of research focused on development of a new tool (DANCE) and set of practice guidelines (DANCE STEPS) designed for NFP nurses to use in home visits.12 The goal was to design a tool that cost no more than the existing one, that had at least equivalent predictive validity, and that nurses found more useful in guiding their practice. The new tool has surpassed these expectations, and is now the new standard for NFP nurse education, without being tested in a separate RCT, as it is a simple substitution for an existing program element, and it is almost certainly superior. All existing sites in the US, the UK, and Canada are being educated in using the new tool.
o One of the most consistent findings across the three trials of the NFP is that nurse-visited mothers had reductions in the rates of closely spaced subsequent pregnancies compared to women assigned to control groups. Closely spaced subsequent pregnancies are associated with increased rates of low birth-weight and infant mortality in subsequent births,14 they make it hard for parents to return to school or work following the birth of the second child because of increased challenges with child care,15 and they make it hard for vulnerable mothers to provide good care for both first and later-born children because of mothers’ having to divide their limited caregiving resources among several young children at once.16 Dr. Alan Melnick, Dr. Teresa Gipson, and Marni Storey in the Department of Family Medicine at Oregon Health Sciences University have been leading a randomized trial of an innovation in the NFP program in which nurses are given the resources to administer hormonal contraception to NFP mothers in the context of their home visits. If effective, we will work with sites to expand nurses’ roles to include administration of hormonal contraception, which will be challenging in some settings. The potential for enhancing the public health impact of the program, however, is substantial.
o In an effort to increase the efficiency and effectiveness of the NFP, we are engaged in the development of a framework for nurses to use in calibrating the levels and types of risks and strengths exhibited by families they serve. With support from the Annie E. Casey Foundation, we are working with NFP nurses and supervisors in five sites to develop, refine, and pilot this framework for making decisions about families’ needs, possible adjustments to the NFP home visit schedule, and increasing teams’ effectiveness in delivering the program. This work is connected to earlier work we conducted in which we found that NFP nurses in the trials focused their scarce resources on families in which mothers had greater levels of need.17 We are likely to implement this intervention without the conduct of an RCT as it represents an elaboration of the NFP model as currently designed and tested in the original trials.
o In the original trials of the NFP, we found no program effects on maternal reports of symptoms of depression and anxiety, although we did find consistent improvements in care-giving and child development, and reductions in injuries among children born to mothers with limited psychological resources, including more symptoms of depression and anxiety.4, 13, 18-19 We developed a refined mental health screening tool and piloted it in Los Angeles County and New York City. Nurses and families found the tool acceptable, but nurses reported that it did not increase their ability to work with families, as most mothers were reluctant to use specialty mental health services.20 We continue to search for effective methods of addressing parental mental health issues.
o As the NFP is replicated in new societies (e.g., the UK, Canada, with Australian Aboriginal families, Native Americans, Alaskan Natives), the program first goes through a process of careful formative adaptation to the new context and populations served. The focus of this early work is on feasibility of delivering the program well in these new contexts. If the preliminary evidence looks promising, investments are made in the conduct of RCT’s in the new context, conducted by local researchers, to provide assurance that the program is producing its intended outcomes, to understand possible population or contextual moderators, and to determine whether the program is a good investment in the new context. If it is, we support careful replication of the adapted program in the new society, with the likely integration of new evidence-based augmentations as they are developed.
Ms. Schorr suggests that there is a polarization of those committed to using data from RCT’s to guide policy and practice, the Experimentalists, and those who wish to use a broader array of evidence, the Inclusionists.1 She suggests that the way to bring together the Experimentalists with the Inclusionists is to focus on core components of effective interventions. This approach minimizes the importance of careful intervention development and corresponding provider education. In fact, one of the likely reasons that so few of the parenting interventions tested in randomized controlled trials have worked well is that insufficient effort appears to have gone into intervention development before they were tested.18 The focus on core components runs the significant risk of trivializing the importance of intervention design and integrity. In this regard, it is important to note that program model clarity in specification of its elements is associated with better program implementation.22
Ms. Schorr suggests that complex interventions cannot be evaluated with RCT’s, but there are excellent examples of community-based interventions that have been tested in RCT’s, with randomization conducted at the level of communities, in which the interventions produced impacts on outcomes of public health importance. They include trials of the Communities that Care (CTC) model and Promoting School-university Partnerships to Enhance Resilience (PROSPER) 23-25 both of which show effects on youth substance use and antisocial behavior in randomly assigned intervention communities compared to controls; and the Intervention with Microfinance for AIDS and Gender Equity (IMAGE) community-based trial in South African communities.26 While the IMAGE intervention had no effects on unprotected sex with a non-spousal partner, it reduced Intimate Partner Violence in intervention communities.26 The point is that it is possible to conduct rigorous research of complex interventions at the levels of communities. Broad based community systems deserve to be evaluated rigorously using standards of evidence that are as rigorous as those used to evaluate individual services. At very least, systems of community services need to be developed using individual components that have strong evidentiary foundations.
I know of no developer of model interventions tested and found to be effective in RCT’s who believes that experimental designs are the only source of relevant information to guide policy and practice. The polarization Ms. Schorr refers to is hardly as severe as she suggests, at least on the part of experimentalists.
Sincerely,
David L. Olds, Ph.D.
Professor of Pediatrics and Director
Prevention Research Center for Family and Child Health
University of Colorado Denver, Anschutz Medical Campus
References
1. Schorr LB. Broader Evidence for Bigger Impact. Stanford Social Innovation Review 2012, 50-55.
2. Olds DL, Hill PL, O’Brien R, Racine D, Moritz P. Taking Preventive Intervention to Scale: The Nurse-Family Partnership. Cogn Behav Pract 2003, 10(4);278-290.
3. Hill P, Olds DL. Improving implementation of the Nurse-Family Partnership in the process of going to scale. In Applying Implementation Science to Early Care and Education Programs and Systems: Exploring a New Frontier. Part III: Aligning Stage-Appropriate Evaluation with the Stages of Implementation: Formative Evaluation and Fidelity. Baltimore, MD, Brookes Publishing, 2012.
4. Olds DL. Prenatal and infancy home visiting by nurses: From randomized trials to community replication. Prev Sci 2002, 3(3);153-172.
5. O’Brien RA, Moritz P, Luckey DW, McClatchey MW, Ingoldsby EM, Olds DL. Mixed methods analysis of participant attrition in the Nurse-Family Partnership. (submitted to Prevention Science)
6. Ingoldsby EM, Baca P, McClatchey MW, Luckey DW, Ramsey MO, Loch JM, et al. Increasing participant retention and home visits in prenatal and infancy home visiting by nurses. (2012 under review)
7. Olds DL, Baca P, Ingoldsby E, Luckey DW, Knudtson MD. Cluster randomized controlled trial of intervention to increase participant retention and completed home visits in the Nurse-Family Partnership. (under review)
8. Olds DL, Eckenrode J, Henderson CR Jr., Kitzman H, Powers J, Cole R, Sidora K, et al. Long-Term Effects of Home Visitation on Maternal Life Course and Child Abuse and Neglect: a 15-Year Follow-Up of a Randomized Trial. JAMA 1997, 278(8);637-643.
9. Eckenrode J, Ganzel B, Henderson CR Jr., Smith E, Olds DL, Powers J, Cole R, et al. Preventing Child Abuse and Neglect with a Program of Nurse Home Visitation. JAMA 2000, 284(11);1385-1391.
10. Wathen CN, MacMillan HL. Interventions for violence against women: Scientific review. JAMA 2003, 289(5);589-600.
11. Jack SM, Ford-Gilboe M, Wathen CN, Davidov D, McNaughton DB, Coben JH, Olds DL, MacMillan H. Development of a nurse home visitation intervention for intimate partner violence. BMC Health Serv Res 2012, 12;50.
12. Donelan-McCall N. Improving the NFP as it Moves toward Scale. Invited Speaker at the 2012 Edna McConnell Clark Foundation Grantee Retreat, May 2012. Terrytown, NY.
13. Kitzman H, Olds DL, Henderson CR Jr., Hanks C, Cole R, Tatelbaum R, McConnochie K, et al. Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing. A randomized controlled trial. JAMA 1997, 278(8);644-652.
14. Conde-Agudelo A, Rosas-Bermudez A, Kafury-Goeta AC. Birth spacing and risk of adverse perinatal outcomes: A meta-analysis. JAMA 2006, 295(15);1809-1823.
15. Furstenberg FF, Brooks-Gunn J, Morgan SP. Adolescent mothers in later life. In Human Development in Cultural and Historical Contexts. New York, NY, USA: Cambridge University Press, 1987.
16. Nathens AB, Neff MJ, Goss CH, Maier RV, Rivara FP. Effect of an older sibling and birth interval on the risk of childhood injury. Inj Prev 2000, 6(3);219-222.
17. Olds D, Korfmacher J. Maternal psychological characteristics as influences on home visitation program contact. J Community Psychol 1998, 26(1);23-36.
18. Olds DL, Robinson J, Pettitt L, Luckey DW, Holmberg J, Ng RK, Isacks, K, Sheff K. Effects of home visits by paraprofessional and by nurses: age-four follow-up of a randomized trial. Pediatrics 2004, 114(6);1560-1568.
19. Kitzman HJ, Olds DL, Cole RE, Hanks CA, Anson EA, Arcoleo KJ, Luckey DW, Knudtson MD, Henderson CR JR., Holmberg JR. Enduring effects of prenatal and infancy home visiting by nurses on children: follow-up of a randomized trail among children at age 12 years. Arch Pediat Adol Med 2010, 164(5);412-418.
20. Pinto F. Mental Health Screening and Referral Quality Improvement Project Report. Los Angeles Nurse-Family Partnership 2009.
21. Olds DL, Sadler L, Kitzman H. Programs for parents of infants and toddlers: recent evidence from randomized trials. J Child Psychol Psychiatry 2007, 48(3/4);355-391.
22. Rorbach LA, Grana R, Sussman S, Valente TW. Type 2 translation: Transporting prevention interventions from research to real-world settings. Eval Health Prof 2006, 29;302-333.
23. Hawkins JD, Oesterle S, Brown E, Arthur MW, Abbott RD, Fagan AA, Catalano RF. Results of a Type 2 translation research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Arch Pediat Adol Med 2009, 163;790-798.
24. Hawkins JD, Oesterle S, Brown EC, Monahan KC, Abbott RD, Arthur MW, Catalano RG. Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial. Arch Pediat Adol Med 2012, 166;141-148.
25. Spoth R. PROSPER community-university partnerships delivery system outcomes through 6 ½ years past baseline. (in preparation)
26. Prokyn PM, Hargreaves JR, Kim JC, Morison LA, Phetla G, Watts C, Burza J, Porter JDH. Effect of a structural intervention for the prevention of intimate-partner violence and HIV in rural South Africa: a cluster randomized trial. Lancet 2006, 368;1973-1983.