Measuring what matters: Building impact pathways to actionable information for the weADAPT platform

Summary

• Online platforms can help bridge the climate adaption knowledge-to-action gap. 
• Understanding climate adaptation outcomes and impacts is key to effective knowledge management. 
• A knowledge management Theory of Change must evolve to meet changing user needs. 
• Trust and transferable, place-based knowledge supports research, policy and practice. 
• Online platforms increasingly serve as hubs for learning, fostering connections, collaboration and long-term partnerships. 

Introduction

The potential of online knowledge platforms to support urgent climate action is increasingly recognised; however, their effectiveness is often hindered by the fragmentation and overabundance of information, which can impede learning and contribute to misinformation, redundancy and erosion of trust. Despite their proliferation, few platforms have undergone systematic evaluation of their impact on research, policy, and practice. This study addresses this gap by assessing the usability and impact of the weADAPT online platform through a mixed-methods approach, combining a user survey and semi-structured interviews. The findings reveal clear pathways linking knowledge management (KM) aims and activities to outputs, outcomes, and longer-term impacts. Users reported that the platform effectively promoted climate change adaptation awareness, supported capacity development, influenced policy and planning, and facilitated knowledge exchange and collaboration. Further analysis identified six core KM activities—enhancing usability, inclusivity, trust, transferability, connectivity, and alignment with FAIR principles—as central to platform effectiveness. These findings informed a recent platform upgrade (2022–2024), the refinement of weADAPT’s Theory of Change, and the development of a tailored monitoring, evaluation, and learning (MEL) framework featuring custom progress indicators. The study underscores the importance of aligning KM practices with user needs and evaluating platform impact in meaningful ways—measuring what we value, rather than merely what is easy to quantify. These insights offer practical guidance for knowledge managers and platform developers working to enhance learning and support evidence-based climate adaptation.

Objectives

Thus far there have been limited systematic evaluations of the impact of online climate adaptation services and the degree to which they address users’ needs and values emphasise this MEL research “gap” in wider climate services literature, with their review finding that evaluations tend to focus on “intermediate success variables”, such as use, access and “perceived variability, and rarely assess the services’ uptake, impacts or outcomes, particularly regarding adaptation decisions and actions. 


This gap could be explained by cases where the claimed outcomes were unintended and therefore not initially monitored in the evaluation, or by shortcomings in evaluation approaches, methods, and measures. Approaches to evaluating the use and success factors of online climate adaptation knowledge brokering platforms have included: participatory observation to identify active users and assess platform inclusivity and user motivations; content and database analysis to evaluate user involvement and content quality; document analysis to understand user demographics ; statistical analysis to examine platform popularity, usage factors, and links between science and policy; interviews and surveys to explore user needs, preferences, and behaviours; and user feedback through meetings, workshops, contact options, social media, and newsletters to assess platform usability. Such studies have highlighted users that are not being targeted or reached, as well as examples of how platforms and their content are used in practice.

This study engages with the weADAPT community to critically assess whether existing knowledge management (KM) processes can be enhanced more effectively to support and accelerate climate action. This research places particular emphasis on understanding and responding to user needs. In doing so, it aims to explore the platform’s core impact by drawing on insights from a user survey and a series of semi-structured interviews.

Methodology

This study uses key questions identified as vital in MEL approaches to assess the effectiveness of adaptation activities.  This approach has been applied to assess a knowledge platform’s alignment with core KM criteria, related to usability and usage (“Are we doing things right?”), meeting user needs and values and achieving outcomes (“Are we doing the right things?”), and verifying these findings through a survey and interviews (“Can we verify that we know what the right things are?”). This approach provided insights into improving functionality, guiding the upgrade process, refining KM practices, and identifying indicators for a MEL framework.

This paper uses the weADAPT platform as a case study to examine how evaluating knowledge management (KM) effectiveness—particularly through the lens of “measuring what we value”—can support progress toward platform objectives and broader climate adaptation goals.

A survey was sent to 5523 weADAPT newsletter subscribers and it received 379 responses, yielding a response rate of almost 7 %. The survey provided information on the demography and role of weADAPT users, how the platform is used, how it supports adaptation work, and how it can be improved. Twenty-one selected survey respondents then participated in focussed interviews with the research team (see Figure 1).

 

Figure 1: Geographic distribution of weADAPT survey respondents (379, one black pin per country represented) and interviewees (21, red pins).

Discussion

This study’s findings contribute to advancing the field of KM offering guidance to knowledge managers of platforms like weADAPT, to better understand and respond to user needs, by developing Theories of Change (ToC) and tailored Monitoring and Evaluating processes. 

This research explored the transition from useful to usable knowledge by examining the uptake, outcomes, and impacts of weADAPT through a ToC. The results showed that users found the platform valuable in four key areas: 1) raising awareness, 2) developing capacity, 3) supporting policy, planning, and implementation, and 4) fostering new connections and knowledge exchange. Enabling factors emerging to support these outcomes have been organized into six key KM aims and activities: 1) increasing usability through translation, tailoring, syntheses and capacity development; 2) enhancing inclusivity through just and equitable sharing of multiple knowledges; 3) building trust through collaborative KM processes; 4) creating transferability through sharing multi-scale, multi-sectoral cases place-based knowledge; 5) improving connectivity through cross-fertilization of knowledge, users, and networks; and 6) promoting FAIR and decolonized search and discovery. Insights from users allowed us to refine the baseline ToC, revealing KM activities and user outcomes that strengthen the connection between KM and impact pathways. The ToC identifies barriers faced by the weADAPT community in accessing and applying knowledge and outlines key KM activities to address these challenges. It also clarifies the outcomes and impacts that are achieved, valued, and should be targeted. To operationalize this ToC (see Figure 2), a set of indicators has been proposed to track progress towards desired outcomes and impacts (see Supplementary Material on the top right).

In the case of weADAPT, many of the KM activities that could support user outcomes align closely with the priorities already identified for platform improvement, e.g. improving collaboration spaces; presenting geo-referenced case studies more effectively; promoting better content search and discoverability; enhancing the user interface; and expanding the content to include both failures and successes, alongside ‘good practice’ information. However, this research underscores the importance of developing indicators that extend beyond standard website analytics (e.g., visits, citations, downloads) to capture the diverse and unique ways in which platforms and their content are used. By complementing basic analytics with indicators derived from regular user engagement, platforms can better assess what holds value and challenge assumptions about the inherent value of traditional metrics. Over time, the resources needed for such efforts can become more efficient, as the right questions to ask and the key elements to monitor become clearer.

Figure 2: Updated theory of change – new users, KM activities and outcomes identified from the survey and interviews, resulting in knowledge to action impact pathways.

Conclusion

This paper reveals how the weADAPT platform is utilized by its users, shedding light on outcomes and moving towards the impacts that users value most for a just, sustainable and well adapted present and future. Through insights gathered from a user survey and interviews, key KM aims and activities were identified, which contribute to supporting these valued outcomes. These findings also guided the platform in revising its ToC, and in establishing a monitoring framework to effectively track progress toward achieving desired outcomes and impacts. The integration of user feedback has played a crucial role in enhancing the platform’s strategy and ensuring that its activities align with the needs of its audience.