Learning and Improvement in Social Systems: Continuous Improvement Insights from Learning Health Systems
Social sector organizations face a persistent structural challenge: the gap between how interventions create change and how those interventions are evaluated. Nonprofits, foundations, and public agencies routinely construct logic models, track outputs, and submit evaluation reports. Yet, impact evaluation remains largely retrospective, treating outcomes as post-hoc verdicts of programmatic effectiveness delivered at grant-end rather than real-time data for optimizing programs underway.
This retrospective orientation mirrors the structural inefficiencies that characterized healthcare delivery two decades ago. Historically, clinical research operated on a linear, long-horizon model, where the translation of empirical evidence into routine practice took an estimated 17 years.¹ Meanwhile, vast amounts of observational data generated during daily clinical practice remained unanalyzed. In response, the Institute of Medicine formalized the concept of the Learning Health System (LHS) in 2007.
An LHS aligns infrastructure, informatics, governance, and organizational culture to embed knowledge generation directly within service delivery. Every client interaction operates simultaneously as a delivery of care and an empirical data point. By compressing the feedback loop between data collection and operational adaptation from years to weeks, the organization learns as it operates.
Given that social sector interventions operate within similarly complex adaptive systems, the LHS framework offers a strategy for shifting away from static, retroactive compliance reporting toward continuous, evidence-based improvement processes that improves beneficiary outcomes, more effective use of resources, and increases systemic impact of programs, organizations, and funders all at once.
We call this a Learning Social System (LSS) - an operational framework that embeds continuous research, rapid data feedback, and iterative design directly into social service delivery. For leaders of nonprofits, foundations, and public agencies, adopting a Learning Social System transforms monitoring, evaluation, research, and learning (MERL) from a passive compliance exercise into an active engine for strategic adaptation.
Complexity Science and the Learning Social System
The intellectual foundation of the Learning Health System is rooted in complexity science. Health and social outcomes, such as long-term housing stability, economic mobility, or family safety, are emergent properties of the system. They result from non-linear interactions among individual behaviors, systemic inequities, environmental conditions, and service delivery interventions.
Like the healthcare environment, cause-and-effect relationships in complex social systems are rarely linear or predictable across different settings. An intervention that succeeds in one context may fail in another due to subtle shifts in underlying causal variables like economic conditions, culture, or resource access.
Traditional practice often treats planning tools like the logic model as a rigid, one-time artifact created for communicating with various stakeholders and for demonstrating compliance to funders. We affirm these uses. Yet, when actual results of a program diverge from intended outcomes delineated in advance in a logic model, we have seen first hand how organizations shift into defensive mode, sensing their need to explain away the gap between intended and actual outcomes. Instead, a learning system approach can enable both service providers and funders to view these gaps positively as opportunities to adapt and align service delivery to directly target conditions “on the ground” and in real time. Rather than viewing outcomes variance negatively, a learning perspective can actually help organizations and funders better target population needs and use human, financial, and material resources more effectively to reach outcome goals.
Using Logic Models as Living Documents in a Learning Social System
When paired with Plan-Do-Study-Act (PDSA) cycles, structured frameworks like our Nonprofit Logic Model Worksheet become the operational compass for continuous learning:
Rationales & Assumptions (Plan): Ground your initial intervention and outcome goals in evidence, community voice, and clear assumptions.
Leading Indicators (Do/Study): Track short- and medium-term outcomes (e.g., changes in knowledge, skills, or immediate behaviors) as monthly or quarterly data rather than dormant metrics.
Iterative Course Correction (Act): Use divergence between expected and actual metrics as empirical data to adjust activities, refine assumptions, and update the inputs and outputs of the logic model in real time.
How Key Social Sector Stakeholders Can Apply and Benefit from Rapid Learning Cycles
Embedding rapid PDSA cycles into routine operations offers concrete, practical value across
the social sector ecosystem. Expand the accordions below to learn more about how different types of social sector organizations can benefit from PDSA cycles.
Nonprofit & NGO Leaders
Traditional Bottleneck | LHS / PDSA Practical Application |
End-of-year evaluations reveal missed targets when it is too late to adjust operations or reallocate staff. | Run 90-day PDSA process on individual friction points (e.g., month-two client drop-off) to improve retention before annual reporting. |
Foundation Program Officers & Funders
Traditional Bottleneck | LHS / PDSA Practical Application |
Grant milestones penalize operational pivots, forcing grantees to execute outdated strategies or defend new ones to protect funding. | Structure grants around adaptive learning milestones, requiring grantees to report on what was tested, what was learned, and how the model adapted. Provide support resources to grow learning capacity of grantee orgs. |
Government Agency Officials
Traditional Bottleneck | LHS / PDSA Practical Application |
Multi-year public service contracts rely on lagging compliance data that masks implementation bottlenecks. | Mandate quarterly leading-indicator reviews across contracted providers to identify systemic barriers early and scale proven adjustments. |
5 Steps to Build a Continuous Improvement Architecture
Transitioning from retrospective evaluation to a continuous Learning Social System does not require abandoning existing systems or adopting costly or demanding new approaches. Rather, we advocate for creative and dynamic shifts in the way current tools and data systems are used. We are convinced that most organizations already have a lot of the data they need to evolve into an LSS.
Deploy Your Logic Model as a Living Blueprint: Use our Logic Model Worksheet to clearly delineate short-term, medium-term, and long-term outcomes. Treat short-term metrics as immediate indicators of program effectiveness within current conditions rather than end-of-year checkboxes.
Focus on One, Maybe Two, High-Stakes Questions: Resist overhauling your entire measurement apparatus at once. Identify one pressing operational question, test a targeted change for 60 to 90 days, and review results in your programmatic data during this period. Then, adjust your strategy. Use the data to help you identify opportunities to increase the effectiveness of your program for participants.
Deploy Dedicated Outcomes Measurement Tools: Integrate pragmatic tracking tools alongside your logic model to gather continuous, low-burden operational data from frontline staff and service participants. Review your existing data collection tools first to determine whether simple shifts in the way you analyze data or small revisions to the types of questions you ask can deliver the results you need to become an LSS.
Put Practitioners Inside the Inquiry: We are strong advocates for participatory processes that capture lived experience and innate knowledge of communities. In an LSS context, this means hosting regular data reviews where participant representatives, community leaders, and frontline staff examine leading indicators alongside organizational leaders. Individuals benefitting from the services and staff who gather the data must be empowered to co-design the operational solutions.
Treat Data as a Strategic Asset, Not a Compliance Trap: Make sure a team member clearly owns data quality, and build a culture where staff feel safe reporting bad news. Cultivate a culture where "negative numbers" (i.e. your program did not meet its goals) is neutral. It provides information you can use to increase the effectiveness of your programs, to understand outside influences that impact how your programs impact participants (or don't), and do not reflect solely on your team. In complex social programs, a missed target usually isn't a team failure, but rather valuable proof that unexpected real-world factors are at play, showing you exactly where your program design needs to adapt. And that is a good thing!
How Kohli+Associates Can Help
Building a culture of continuous learning is a powerful opportunity to help social sector organizations stay effective in a rapidly changing world. At Kohli+Associates, we help nonprofits, grantmakers, and government agencies build measurement systems that help teams learn and adapt.
We work with leaders in three key ways:
Evaluation & Data Audits: e look closely at how your team currently tracks data and reports results, spot where the process gets stuck, and help you launch your first 90-day learning cycle.
Program Design & Outcomes Toolkits: Through hands-on team workshops, we build clear logic models and custom tracking tools that help you monitor progress continuously, month-by-month, and not just for grant report deadlines.
Flexible, Outcomes-oriented Strategic Planning: We help executive teams, boards, and public agencies create realistic strategic plans and funding proposals that show grantmakers you know how to adapt, improve, and deliver on your mission.
Ready to transform your evaluation process into a driver of continuous improvement?
Contact Kohli+Associates today to schedule a strategic consultation.
Footnotes
Balas, E. Andrew, and Suzanne A. Boren. "Managing Clinical Knowledge for Health Care Improvement." Yearbook of Medical Informatics 9, no. 01 (2000): 65–70.
Learn More about PDSA Cycles
Learn more about PDSA Cycles through video tutorials from the Institute for Healthcare Improvement: https://www.ihi.org/library/tools/plan-do-study-act-pdsa-worksheet
Kohli + Associates works with nonprofits, foundations, and government agencies to build theory of change frameworks, outcome measurement systems, and evaluation strategies grounded in research methodology.
