Yes! And… I would urge those seeking to evaluate programs and policies in complex systems, to read Michael Quin Patton’s book “Developmental Evaluation.” Patton brings a keen understanding of traditional methods of evaluation, and an equally deep understanding of the dynamics of complex systems and challenges. He recommends distinguising between types of problems, so that simpler, more linear systems, can be evaluated with traditional methods. Developmental evaluation is recommended for the complex systems and challenges- an approach in which we constantly observe, reflect, and learn from the outcomes of our actions in the system. This leads to adaptation both of our interventions, and at times, of our objectives.
It is the dimension of time that is becoming a critical element of program evaluation - closing the gap from policy development, to execution, to evaluation with the results feeding back into the policy development/re-alignment process. The complexity you refer to is a function of the compression occurring within policy making, on the time to act and to demonstrate results. This is driven by our rapidly changing political and economic cycles. This compression requires agility and flexibility in program evaluation methods including automation. While complexity rightly demands new methods as outlined in this brief, more importantly - and perhaps paradoxically, it demands information on program performance to be made available as close as possible to real-time - ie. as it is happening in the field.
Brian - I agree with your categorization of “close to real-time” feedback and data being necessary in today’s world. We explore this idea, as well as the role of technology, in a previous publication titled “Next Generation Evaluation” http://www.fsg.org/tabid/191/ArticleId/964/Default.aspx?srpush=true
COMMENTS
BY Bruce Waltuck
ON March 5, 2015 05:28 PM
Yes! And… I would urge those seeking to evaluate programs and policies in complex systems, to read Michael Quin Patton’s book “Developmental Evaluation.” Patton brings a keen understanding of traditional methods of evaluation, and an equally deep understanding of the dynamics of complex systems and challenges. He recommends distinguising between types of problems, so that simpler, more linear systems, can be evaluated with traditional methods. Developmental evaluation is recommended for the complex systems and challenges- an approach in which we constantly observe, reflect, and learn from the outcomes of our actions in the system. This leads to adaptation both of our interventions, and at times, of our objectives.
Very highly recommended.
BY Srik Gopal
ON March 5, 2015 07:02 PM
Thanks, Bruce! Yes, Michael’s book is indeed a great resource. We are fortunate to be able to collaborate with him on occasion.
BY Brian Lee-Archer
ON March 8, 2015 06:05 PM
It is the dimension of time that is becoming a critical element of program evaluation - closing the gap from policy development, to execution, to evaluation with the results feeding back into the policy development/re-alignment process. The complexity you refer to is a function of the compression occurring within policy making, on the time to act and to demonstrate results. This is driven by our rapidly changing political and economic cycles. This compression requires agility and flexibility in program evaluation methods including automation. While complexity rightly demands new methods as outlined in this brief, more importantly - and perhaps paradoxically, it demands information on program performance to be made available as close as possible to real-time - ie. as it is happening in the field.
BY Srik Gopal
ON March 9, 2015 10:37 AM
Brian - I agree with your categorization of “close to real-time” feedback and data being necessary in today’s world. We explore this idea, as well as the role of technology, in a previous publication titled “Next Generation Evaluation” http://www.fsg.org/tabid/191/ArticleId/964/Default.aspx?srpush=true